Enumerating from a Survivor

A recycle tears a hub down. Anything the recycle needs to work out — which other addresses to reach, what the network of dependents is — has to be derived from somewhere, and the obvious somewhere is the hub being recycled: it is the one holding the request, it knows its own address, and its services are right there.

That works for as long as the derivation is synchronous, and the derivation is not synchronous.

The window

MessageHub.HandleDispose runs the RecycleCascade seam and then calls Dispose():

HandleDispose
 ├─ CascadeRecycle(request)     ← the seam. Returns as soon as the pipeline is SUBSCRIBED.
 └─ Dispose()                   ← RunLevel walks down to Dead
       └─ DisposalCompleted
             └─ HostedHubsCollection.CloseScopeWhenDisposed → the hub's Autofac scope is CLOSED

The comment at the seam says the network is derived "while this hub is still whole enough to compute it". True — of the prologue. The seam's pipeline is a chain of cross-hub queries composed as Concat, so only the first leg's Subscribe happens on the recycle's turn. Every leg after it subscribes on a pool thread, milliseconds later, after Dispose() has completed and the scope has been closed. There is no race to lose: by construction the later legs run against a dead scope.

What they reach for is ordinary and invisible:

Each of those is an AutofacServiceProvider.GetService on a closed LifetimeScope, which throws ObjectDisposedException: Instances cannot be resolved and nested lifetimes cannot be created from this LifetimeScope as it (or one of its parent scopes) has already been disposed.

Why it is worse than an error

The cascade is careful: a leg that cannot be read is carried to the end as incomplete rather than collapsing into an empty answer, and reported at Error. That is exactly right, and it is still not enough, because the recycle succeeds everywhere a caller can see it:

• the operator's recycle returns;
• the definition hub really does go down;
• the fan-out really does reach the addresses that were derived;
• and the instance hubs — the whole reason the cascade exists — keep serving the assembly they were born with, with no error of their own and nothing to grep.

Measured on memex.systemorph.com: a recycle of Hosting/TriageItem recycled 0 addresses with one leg incomplete. The type's dependents happened to be empty, so the one leg that failed was the type's own instances, and the entire point of the operation was silently skipped.

The rule

The set is derived once, from a survivor — and so is every READ that derives it. The cascade already applied that rule to the outbound DisposeRequests ("a dying hub cannot deliver its own last frame") and posts them from the mesh's node-operation issuing hub. The read half needs the same treatment, and its seam is the mesh's read-issuing hub:

// NodeTypeRecycleCascade.DependencyNetwork
var reader = hub.GetMeshHub().ReadIssuingHub();
var logger       = reader.ServiceProvider.GetService<ILoggerFactory>()?.CreateLogger(…);
var meshService  = reader.ServiceProvider.GetRequiredService<IMeshService>();
var accessService = reader.ServiceProvider.GetService<AccessService>();

ReadIssuingHub() is the right survivor for three independent reasons, and picking a different one gives up one of them:

• it is hosted by the mesh hub, so the recycled hub's teardown cannot touch it;
• it is off the router, so the router is never an end of a delivery;
• it is not the node-CRUD execution hub, so these reads do not queue behind every write in flight — the /api/content 503 shape documented in CQRS and Content Access and its neighbours.

hub.GetMeshHub() is safe on an already-dead hub: it walks Configuration.ParentHub, which resolves from the parent's provider (the mesh's, still alive) and is cached.

Testing it

The property is "the enumeration does not depend on the passed hub's DI scope", and it has two halves, because the hub can die in either gap:

1. composition resolves the services, and
2. subscription runs the legs.

Both are covered by ARecycleCascadeEnumeratesFromASurvivorTest, and neither is a race. The production shape is deterministic if you compose while the hub is whole, dispose it, and only then subscribe. The second case calls the method when the hub is already down.

Two things keep those cases honest:

• The precondition is asserted, not assumed — a resolve from the dead hub must throw. If the scope-closing order ever changes, the cases would otherwise pass having exercised nothing.
• Every case asserts the instance path is in the network, not merely that the answer is "complete". An empty network is complete too, and a cascade that answered "complete, 0 addresses" for every input is the silent form of the very defect being fixed.

And a third case runs the ordinary live path, so a failure that made the enumeration read nothing could not hide behind the other two.

🚨 Hoisting the resolve does NOT fix this, and that is the part worth remembering

Disposed Scopes and Dying Hubs catalogues this family and gives R3 — a continuation resolving from a scope its own pipeline closed — the correction hoist, never guard: capture the service at the top of the method, while the scope is alive, and the continuation uses the captured instance.

That correction is already applied here and is not sufficient. DependencyNetwork resolved its three services eagerly, in its own synchronous prologue, exactly as R3 prescribes. It still threw.

The reason is a property of the service, not of the call site:

MeshService takes IMessageHub in its constructor and reaches back through it on every single Query<T>: StampViewer → CaptureContext() → hub.ServiceProvider.GetService<AccessService>(). So hoisting a per-hub scoped service relocates the throw from the continuation into the service's own method body, where no amount of hoisting at the call site can reach it.

So R3's rule needs its companion clause:

Hoist when the service outlives the scope. When the service IS the scope — anything registered per hub — hoisting is not the fix; resolve from a survivor instead.

The two rules agree on the underlying question, which R3 states as "what does this resolve stand in front of". Here it stands in front of the only work the operation exists to do, and the service that would perform it is bound to the thing being destroyed.

Where else this shape lives

The generalisation is not "never resolve in a continuation" — that rule is false, and hoisting has its own failure modes (R3's own counter-example: hoisting a logger put a possible throw ahead of a collectible-ALC lease release, turning a lost log line into a guaranteed leak). The question is narrower and answerable:

Does this pipeline outlive the hub whose provider it resolves from — directly, or through a service that hub owns?

Wherever the answer is yes — a teardown that derives work, a write that reports after its caller is gone, a watcher armed on the way out — the resolve belongs to a survivor. HostedHubsCollection's own commentary names this the ObjectDisposedException straggler class, "whose one escape onto a scheduler thread is the anonymous Catastrophic failure that reds an otherwise green shard".

See also

• Hub Disposal Model — what a DisposeRequest tears down, and the cascade
• Stale State Until Recycle — why an un-recycled activation is indistinguishable from a bad fix
• Disposed Scopes and Dying Hubs — the five roots of this symptom, the tiered 204-site sweep, and two hoists that made things worse. This page is R3's companion clause
• Access Context Propagation — why a read carries an identity across Subscribe boundaries at all