Retiring xunit — what the in-mesh migration removes, and what it does not

Measured 2026-09-18 on MeshWeaver.Plugins main (b0d32552d), read-only, with scripts/xunit1069-sites.py over the tree and scripts/generate-in-mesh-suites.py as the conversion ledger. Both are committed; re-run them rather than trusting this page's numbers.

The question this answers (maintainer, 2026-09-18): of the 2,308 xUnit1069 sites in #2031, how many live in suites the migration deletes or converts — N of 2,308 — and what is the residue?

The denominator, reproduced from source

xUnit1069 fires on a [Fact(Timeout = …)] / [Theory(Timeout = …)] method whose body does not reference TestContext.Current.CancellationToken. #2031 counted 2,308 sites from one CI run's log (35199866577). That log is partial by construction — it is attempt 2, and 12 of the 40 projects with sites (MeshWeaver.AI.Test among them, 270 sites) did not run in it — so the denominator has to come from the tree:

count
timed tests (Timeout = on a Fact/Theory) in src/*.Test* 2,752
… that already pass the token (not a site) 419, in 145 files
xUnit1069 sites 2,317, in 546 files, 40 projects

The rule is checked against the run: for the 28 projects the log does carry, source and log agree within the commits landed since (MeshWeaver.Security.Test 218 = 218; Hosting.Monolith.Test 387 = 387; the rest ±1…5). 2,317 today is #2031's 2,308 plus a day of merges.

N of 2,317: what the two migration PRs remove

Zero. Neither PR deletes an xunit test file:

PR files added modified deleted lines
core #4185 (testing/core-in-mesh-estate) 327 308 19 2
Plugins #1792 (testing/in-mesh-suites) 606 605 1 3

Both lay Testing/<Suite>/Source/*.cs down BESIDE src/*.Test; the xunit project stays, is still built by Build + test the portal hosts, and still emits every warning. A site disappears only when its xunit PROJECT is deleted — a project, not a file, because CI builds and declares tests per project (.github/ci-tests.json, scripts/project-closure.py).

So the honest question is which projects could be deleted once their in-mesh suite is green, and the generator's own ledger answers it:

sites share
in files the generator CONVERTS 370 16%
in files the generator REFUSES 1,947 84%
… refused because the class overrides ConfigureMesh 1,586 68%
… needs Postgres 76 3%
… inherits HubTestBase/OrleansMeshTestBase/… (ConfigureHost) 66 3%
… acts as an arbitrary identity (declined by design) 50 2%
… uses a type declared in a refused file, test-only package, FileSystem, MemberData, … 169 7%

And the project view is stricter still. 18 of the 83 test projects that carry test files convert completely, and every one of them carries 0 sites. Every project that has a site has a refused remainder (Hosting.Monolith.Test 53 of 184 files, AI.Test 107 of 241, Security.Test 3 of 50, Hosting.PostgreSql.Test 0 of 145), so none can be deleted by this converter.

Deleting everything the converter can convert today removes 0 of 2,317 sites. The 370 "convertible" sites sit in projects that cannot go.

The residue is real — and converting does not fix it, it hides it

Two findings that change what "convert" means for this warning:

  1. The in-mesh runner has the defect xUnit1069 names. MeshTestRunner.RunCase (core src/MeshWeaver.Testing.InMesh) runs a case as Observable.FromAsync(...).Timeout(bound). The timeout emits ❌ FAIL — no verdict within Ns and moves to the next case; the awaited work keeps running, inside the gate's one portal process, with no token to observe. MeshTestContext exposes a Deadline and no CancellationToken. That is "a timeout that fires while the work it started keeps running" — the shape #2031 calls the live correctness risk — with no analyzer left to say so.
  2. The converter un-fixes the fixed ones. convert-xunit-to-inmesh.py rewrites TestContext.Current.CancellationTokenCancellationToken.None. The 419 timed tests in 145 files that already pass the token would lose it on conversion.

So the count would fall by analyzer absence, not by behaviour. That is the outcome the task rules out ("not because 2,308 tests were decorated" applies in both directions: not decorated, and not undecorated).

What follows — the slice plan

The two directives reconcile as: fix the sites in a way that SURVIVES conversion, and retire suites by deleting projects as the facility allows. Passing the token is not decoration when the in-mesh runner honours it — it is the one change that is correct on xunit today and correct in-mesh tomorrow.

# slice removes status
1 this page + scripts/xunit1069-sites.py (the reproducible denominator and ledger join) #2078 merged
2 core: MeshTestRunner cancels a case at its bound — a per-case CancellationTokenSource linked to the timeout, MeshTestContext.CancellationToken exposed; converter maps TestContext.Current.CancellationToken → the context's token instead of None fixes 1 and 2 above core #4719 (green on build + tests; its required Automatic review answered context is red because Copilot's monthly quota is exhausted — fleet-wide) · Plugins #2079 merged
3 per-project token pass in src/*.Test, following the token into the helpers the tests wait in (core did this in MeshWeaver#4388) — one project per PR, largest first: Hosting.Monolith 387, AI 270, Security 218, Hosting.PostgreSql 153, PluginCatalog 151, … 2,317 → 0 warnings in flight — see the ledger below
4 remove xUnit1069 from WarningsNotAsErrors in src/Directory.Build.propswarning == error the hatch when 3 reaches 0
5 core: the per-area service-substitution facility (InMeshTestMigration facility 1) unblocks 1,586 sites' files (68%) separate design
6 per-project retirement: in-mesh suite green in the gate AND the xunit project deleted in the SAME PR (ci-tests.py --update, project-closure, test-suite-lanes.allow untouched) — starting with the 18 projects that already convert completely (0 sites, so no warning moves, but 18 fewer xunit projects) xunit projects → 0 after 2; the site-bearing projects after 5

Slice 3 is the large one and it is the one that cannot be skipped: 84% of the sites live in files no converter reaches until the facility exists, and the facility is a core design, not a Plugins PR. A site fixed in slice 3 stays fixed through slice 6 because slice 2 makes the converter carry the token instead of dropping it.

What the plan does NOT do: decorate a site with the token while its awaited work ignores it (that is the analyzer's approval without the behaviour — core measured "707 tests satisfy xUnit1069; 271 helper-level waits still outlive the verdict" after its own pass), add a NoWarn, or delete a project whose in-mesh suite has not rendered green.

Slice 3 ledger (2026-09-18)

Every row: scripts/xunit1069-pass-token.py over ONE project, the compiler as the judge (--fix-errors turns CS1061/CS0411/CS1739 into the right shape), then the no-wait pass (a synchronous body, or an async Task that never awaits, loses a Timeout that could never have interrupted anything — the reason at each site), dotnet build -c Release -warnaserror at 0/0, and the suite run with the native xunit runner (dotnet <dll> -trx <file>; dotnet test has no working reporter flag in this repo). "removed" counts the Timeouts taken off no-wait tests.

Rows still open when the self-hosted queue reached 36 runs were folded into ONE change, #2114 (each slice one merge commit): every slice touches scripts/, which selects all 52 units, and every merge buys a full un-cancellable main run — fourteen of each was most of the queue. The PR numbers below name where each slice was reviewed and run; the closed ones point at #2114.

PR project sites → 0 removed cases
#2084 Threading 13 0 144/144
#2085 Hosting.Monolith 387 17 808/808 (+2 skipped)
#2087 AI 270 38 2014/2014 (+3 skipped)
#2089 Security 218 12 383/383
#2090 · merged PluginCatalog 151 31 738/738
#2091 · merged Hosting.PostgreSql 153 7 1141/1141 (+1 skipped), real Postgres
#2092 · merged Markdown.Collaboration 122 88 422/422 (+1 skipped)
#2098 Query 118 11 354/354
#2099 Cornerstone 106 55 120/120
#2100 GitSync 103 2 199/199
#2101 Content 98 63 142/142
#2102 Autocomplete 79 4 146/146
#2104 Acme 66 3 65/65
#2105 FutuRe 47 0 59/59
#2106 Markdown.Export 39 3 207/207
#2108 twenty smaller suites 287 2 4032/4032
#2112 AI.Orleans 36 · Snowflake 19 · Cosmos 11 · Indexing.PostgreSql 5 · Collaboration.Orleans 2 73 4 all green (Snowflake 55 ran / 103 skipped without credentials)

All 2,317 sites are covered by the rows above; scripts/xunit1069-sites.py reads 0 on #2114's tree. Slice 4 (#2113, draft until main reads 0) removes xUnit1069 from WarningsNotAsErrors in src/Directory.Build.props; with it applied to today's main, Threading.Test fails with exactly its 13 sites as error xUnit1069 — the exception is load-bearing until the slices land.

What the rewriter learned along the way, each pinned in its --self-test: a comma inside a generic argument list, a [Fact(Timeout…)] quoted in a comment or held in a raw string (C# source under test), a brace inside a literal, await X() as T, expression-bodied act lambdas handed to Should().ThrowAsync (the token goes inside the lambda; the ThrowAsync await is left alone), a hand-rolled var ct = new CancellationTokenSource(x).Token linked to xunit's token instead of racing it, and a private Ct => TestContext.Current.CancellationToken accessor named directly (the analyzer looks for the member access, and var ct = Ct; hides it).

Found on the way and reported rather than papered over: Auth.Test's DeleteToken_NonexistentPath_Completes (no Timeout, untouched) answers True on one run and False on the next on identical code — #2109.

Both PRs as they stand

Neither is restarted here. Slice 2 lands the converter fix on top of #1792's tooling (on main since #1789); slice 6 draws suites from #1792's branch one package at a time.

Reproduce

python3 scripts/generate-in-mesh-suites.py 2> /tmp/refused.txt     # lays Testing/<Suite>/Source down (untracked)
python3 scripts/xunit1069-sites.py --refusals /tmp/refused.txt      # sites, converted/refused, per project, per reason
git clean -fdq Testing                                               # the generated tree is not committed here
Reconnecting…
The connection to the server was interrupted. Trying to restore it…
Trying again…
The connection could not be restored. Reloading the page…
The server was updated. Reloading the page to pick up the latest version.