Package Uninstall

The rule (policy package-uninstall-request, register). A platform admin uninstalls a package through ONE durable request, in two phases. Phase 1 needs no confirmation and destroys no data. Phase 2 — the irreversible drop of the package's partition — runs only after the requester repeats the partition name; without that the package stays uninstalled with its data retained, and the request says so. A request a feature flag issued never reaches phase 2, so its phase 1 does not run the user-data census that guards the drop: it records the partition as not measured: a flag retire never drops, and any confirmation of it is refused by name.

It sits beside Module Reload and has the same shape: a node at Admin/_PackageUninstall/{id} (PackageUninstallRequest), written only by PackageUninstall.Request as System after the caller was authorised, executed by PackageUninstallExecutor on the request node's own hub, every step a stream.Update.

Phase 1 — uninstall, retain the data

Refused by name, before anything is touched:

The last two refusals are the census, and the census exists only to guard phase 2's drop. A request a feature flag issued (RetiredByFlag) skips it: it can never reach phase 2 (see below), so its preview names the partition and records not measured: a flag retire never drops (PackageUninstallExecutor.FlagRetireNotMeasured) instead of counting it, and the request offers no ConfirmationRequired. Before this, the census refused a flag retire of a partition above the cap on every boot, forever, while touching nothing: on memex-cloud Hosting (about 26 800 nodes) reached attempt -a170 (#8020). A person's uninstall is measured exactly as before. The tests are ADisabledFlagRetiresAnInstalledPackageTest.AFlagRetire_OfAPartitionOverTheCap_IsNotMeasured_AndRetires and its negative control APersonsUninstall_OfThePartitionOverTheCap_IsStillRefused.

The module's activation is read whole or not at all (ModuleLandingService.GetCheckedActivation). An unreadable activation record would make a landed module look like it was never landed, and phase 1 would remove the install record with the generation still enabled. So an incomplete read, or a removal that fails, fails the request and keeps the install record. An entry that is already disabled counts as done: a phase 1 resumed after its removal records ModuleDisabled, never "never landed".

Then, in order: the module is retired (its landed generation disabled; unloaded live through IModuleLiveActivation.Retire where the loader can, else by the ONE restart the instance's restart coordinator dispatches, see "One restart for every retirement" below); the partition's hubs are closed (DisposeRequest to every hosted hub at or under it, from the off-router issuing hub); the install record is removed; and the request moves to AwaitingConfirmation with the preview — per partition: whether a storage provider reports a per-partition store (Postgres: the schema), rows per table (mesh_nodes, and each satellite segment _Access, _Thread, …), whether a sync configuration ({partition}/_GitSync) is present, and what cannot be counted here, named:

From this point no unattended pass installs the package again — not the seed, not the platform baseline, not a feature flag (InstanceAutoRegistrationService.InstallAll consults PackageUninstallExecutor.UninstalledHere). A person installing it again lifts that: the block only applies while no install record exists. A request a feature flag issued (RetiredByFlag) blocks nothing — the flag's own exclusion governs, so the record decides.

The install record decides who may write it

An uninstall and the unattended lanes write the same node, the install record Plugins/{package}, from different hubs. Every writer that is not the record's own hub runs its stream.Update lambda on its own copy of the node, so a check like "is this package being uninstalled?" made there can be stale (Conditional Writes Across Hubs). So no writer decides. Each writer states a fact, unconditionally and under its own key (PackageManifest.RecordFacts). The record's own hub folds every pending fact against current state (InstallRecordGate.Fold, serialised there) and answers each one under the same key (FactOutcomes). The writer reads its answer and removes it.

fact stated by the owner
Begin phase 1, after its refusals and the preview grants RetiringBy (this request) unless another request holds it; phase 1 proceeds only when granted, and touches nothing otherwise
Remove phase 1, just before it deletes the record marks the retirement Removing; refuses when this request no longer holds it (a person installed the package meanwhile), and phase 1 then stops and keeps the record
End phase 1, when it faults after Begin — and when Begin's answer never arrived (the owner may have granted it) releases the retirement, so the package is repairable again, and remembers the request for the retention window so a LATE Begin of it is refused
Stamp every install-record write: the install funnel, the repair, the unattended update applies it, or refuses it while a retirement is held. A person's stamp (PackageRecordWriteIntent.Person, the catalog click) clears a retirement that is not yet Removing, and is applied

| PersonClear | the install funnel, at the end of every exit of a person's install — including the up-to-date skip, which writes no record | clears a retirement that is not yet Removing; refused while the record is being removed |

Every submission correlates its answer under a FRESH key, so an answer left behind by an earlier attempt is never read as this one's; the owner is idempotent on the request, not on the key. A write intent the gate does not know is refused by name — by the writer before anything is created, and by the owner for a stamp — never given the default's semantics. A delete of the record racing a submission (an ended reader, a routing NotFound, a failed stating write) is judged by whether the record is gone: gone is the typed refusal, present is an unrelated fault and propagates.

A refused stamp reaches its writer as PackageRecordWriteRefusedException, carrying the owner's words; the boot install records it as a skip, never as a silent success. A write through the gate is durable when it completes. The writer removes its answer with a cross-hub stream.Update, which acknowledges only after the owner has written the node to storage, fold included, and the writer waits for that before it returns. So every later storage read of the record sees the stamp, as it did when the record was written with a full-replace CreateOrUpdateNodeRequest. The gate issues every write from the mesh's off-router execution hub, never from the record's own hub, even when that hub is the caller: an own-hub update acknowledges the in-memory commit, not the durable write. If that durability step fails for an applied stamp, the write faults instead of reporting it as applied. A refused stamp stays a refusal. If no answer arrives within InstallRecordGate.AnswerBudget, the writer re-reads the record. Absent means it was removed and the write is refused; present means a fault naming the owner that did not answer.

Re-creation is closed one level down. A stamp is a stream.Update, and a delete wins over an update that commits after it (UpdateNeverResurrectsADeletedNodeTest). An absent record is created (create-only, CreateNodeRequest) only by CreateAllowed (the default install of a package the instance does not hold) or Person. A create that collides falls back to a stamp. A Restamp (the repair, the unattended update) is refused when the record is gone. This is how this page answers the create-if-missing-plus-fold gap (#4928): the record is never written with a full-replace CreateOrUpdateNodeRequest.

The partition's content is still written before an install stamps the record, and a stamp refused at the owner does not undo it. The module no longer shares that gap; see the next section.

The module lands through the install record

An install writes its record, then downloads and lands the module. A download that started before an uninstall could finish after phase 1 had written the module's tombstone. Its landing then re-enabled the module on every replica, because the module volume is shared. The decision to land was taken on a snapshot read before the download and carried out after it.

So nothing outside the record's own hub lands or removes a package's module (ModuleLandingOwner):

A person's install waits for a removal in flight. A stamp or a person's clear is refused while ANY removal of the module is Executing, including one whose uninstall has already released its retirement (a writer that timed out and stated End). That removal may be waiting on the shared landing lane behind another module, and would disable the module after the person's install read it as current. Clearing the retirement then would keep the record and lose the module. The person installs again once the uninstall finishes.

Phase 1 always queues the removal. When phase 1 finds no activation entry, the module may still be landing for the first time on the owner. So phase 1 queues the removal anyway. The owner decides whether a generation exists once every earlier landing has finished, and answers ModuleLandingOwner.NothingToRemove when none does.

No record path means the canonical one. An adoption that names no install-record path belongs to Plugins/{package}, so it never lands around that record's owner by omission.

A landing that loses its record is refused. The adoption notes whether the install record exists when it STARTS, before the plan reads the registry index and the activation, and so before the download. If the record existed then and is gone when the bytes arrive, an uninstall removed it while the module was downloading, and the landing is refused. This holds even for a module that never landed, where no tombstone exists to refuse it.

A landing planned against an earlier install is refused. Knowing that a record existed is not enough: an uninstall followed by a re-install leaves a record at the same path, with no retirement and the module enabled. So each install record carries an Incarnation, minted when the gate creates the record and kept by every stamp. The adoption reads it when it starts, and the landing carries it on its operation. The owner refuses, at promotion and again at execution, a landing whose incarnation is not the record's. Without that, a download held across the uninstall and the re-install would replace the new install's module with the old install's bytes. A record created before incarnations existed has none, and a landing planned against it is not checked this way.

A package with no install record has no owner. The boot's modules-first wave lands a fresh package's module before any record exists. A retirement lives on the record and ends when the record is deleted, so a package with no record has no retirement in flight. That landing runs where it is, through ModuleLandingService.LandUnlessUninstalled. It is refused when the module's newest event is an uninstall, which covers a download released after an uninstall that has completed. It never creates a record. It is never a fallback for a package that does have one: the writer reads the record first and, if it exists, goes through the owner.

What the fresh path does not close: its uninstall check and its write are one work item on one replica. A removal another replica writes inside that work item, between the read and the write, would be ordered before the landing. For that to happen, a record must be created, a retirement granted and the removal executed within that interval. The owned path has no such window, apart from one case: when an owner is torn down after a landing has passed its commit rename, that landing finishes its record write. On Orleans a successor activation starts only after the old one has been disposed, and on a single process the successor's landing queues behind it on the same cap-1 lane. A staged landing whose writer died before taking its answer, or whose submission's outcome is unknown, is not collected by the modules GC.

One restart for every retirement

A module that some process still holds is unloaded by a restart. A request does not issue that restart itself any more. Two uninstalls running at once used to restart the instance twice, and a request that resumed after the restart it had asked for could not tell whether to ask again. So the instance has ONE restart coordinator, a node at Admin/_PackageUninstall/RestartCoordinator (PackageRestartCoordinator, content PackageRestartCoordinatorState). It is written only through its own stream, and it follows the same facts-to-owner shape as the install record (Conditional Writes Across Hubs): requesters state facts under fresh keys, and the coordinator's own hub decides.

fact stated by the coordinator
Pending phase 1, after the retirement is granted and before the module is touched, when a restart can follow (a declared module this process holds, and a restart path on the host); for a flag-issued request the boot pass, before it writes the request, and phase 1 again whether or not this process holds the module (see below) records the request as pending. While any member is pending (for at most PendingBudget, 15 minutes), no NEW restart is dispatched, so the restart that follows covers this request too
Ready phase 1, once the module's generation is disabled and it could not be retired live; for a flag-issued request, once the generation is disabled durably, whether or not it was retired live here (a sibling replica may hold it) records the request as ready. A request that is already covered stays covered: stating Ready again (a resumed phase 1) never dispatches again
Withdraw phase 1, when no restart is needed after all (retired live, not loaded here, no restart path, a request that already asked its restart before the coordinator existed) or when phase 1 stops removes a pending member. A ready or covered member keeps its place, because its module IS disabled
Retry the plugin catalog's reconcile pass (PackageRestartCoordinator.RetryFaulted), once a faulted dispatch's backoff is due dispatches that faulted dispatch again. With no dispatch named, it only decides again, so a ready member held back by a pending member that went stale is dispatched

The decision (Decide, pure, on the coordinator's own hub): every ready member joins a dispatch that faulted, because that dispatch restarted nothing and its retry covers them too. Otherwise, while no dispatch is in flight and no member is pending, ONE new dispatch covers every ready member.

The dispatch follows ModuleReloadExecutor.IssueRestart:

Resume-safe across a process restart. Everything lives on the node. A dispatch that another activation left Dispatching (most often because the restart it asked for rolled that very process before the answer was written) becomes Interrupted and is never asked again: a restart is asked at most once. There is no leader election: the coordinator is one node, so one hub decides.

Phase 1 waits for the answer of the dispatch that covers it (up to AnswerBudget) only to state it in ModuleOutcome (one restart requested (Restarted: …) — dispatched by the restart coordinator as restart-N). If no dispatch answers in time, for example because another uninstall is still pending, phase 1 continues and says so; the coordinator dispatches all the same.

Retired per-request field. PackageUninstallRequest.RestartRequestedAt is [Obsolete] and never written. A request persisted by an earlier build may carry it: such a request asked its one restart itself. When its phase 1 resumes on this build, it finds the stamp and withdraws, so it is never restarted again. Requests without the stamp go through the coordinator.

Flag retirements: one restart for the whole batch. The boot pass that retires every installed package a feature flag declared off (DeclaredDisabledRetirement, Environment Composition → "A disabled flag retires what is installed") walks its packages one at a time, each phase 1 settled before the next is decided. So before it writes the FIRST request, it states Pending for every request it will run phase 1 for: a new attempt, or the install's active attempt that is still in phase 1, of a package that declares a module, on a host with a restart path. An attempt already past phase 1 is not enrolled again, because nothing would ever resolve that pending member. Each phase 1 (RetireOnEveryReplica) then states Ready once the module's generation is disabled durably, or Withdraw when there was nothing to disable or it failed. It does not wait for the dispatch, because the dispatch waits for the batch's last request and an earlier one would only spend AnswerBudget (PackageRestartCoordinator.StateReady). When the walk ends, the boot pass withdraws every enrolment whose request was never written: refused by validation, a create that faulted, or a package that a staying package requires. The coordinator dispatches ONE restart once no member of the batch is pending. A request carries no restart field of its own, and a later boot that finds every request settled enrols nothing, so it never restarts again.

The batching is pinned by ADisabledFlagRetiresAnInstalledPackageTest.TwoPackagesDeclaredOff_RestartTheInstanceOnce: two packages declared off, each with a landed module, give one dispatch covering both requests and one restart. Without the boot pass's enrolment (negative control, measured) the first request dispatches alone and the second dispatches again, so the instance restarts twice.

Phase 2 — drop the data, on confirmation

ConfirmationRequired is the partition name. The requester sends it back (PackageUninstall.Confirm, recorded with who and when). A confirmation is accepted only while the request AWAITS one — it answers the preview, so one sent before the preview exists is refused and records nothing, and phase 1 clears any confirmation it finds and stamps awaitingConfirmationAt. The executor is the authority: a confirmation of a request a feature flag issued (checked first — its partition was never measured), a different string, a confirmation from anyone but the requester, a request that names no requester, or a confirmation older than the preview is refused by name (confirmationRefusal) and nothing is dropped. A matching one runs the platform's governed whole-partition teardown, as System, after closing the partition's hubs again: PartitionTeardown.TearDownPartition — the store dropped on every storage provider (Postgres DROP SCHEMA … CASCADE, satellite tables with it, so the _GitSync configuration and the partition's NodeType nodes go too), the cached queries anchored to it evicted, and the Admin/Partition/{partition} registry record deleted. The request records each partition's teardown sentence and ends Done. Never raw SQL.

Surfaces

surface how
MCP uninstall_package (MeshWeaver.Plugins) MeshOperations.UninstallPackage(package, reason) answers the preview and the exact confirmation string; UninstallPackage(requestPath: …, confirmation: …) confirms. Platform admins only.
the package page (MeshWeaver.Plugins Store/Plugin InstanceUninstall) Uninstall from this instance, shown to a platform admin only (everyone else gets an empty, hidden control). The click authorises the caller by id, fail-closed, then files phase 1 as System; the status line links the request node, which carries the phase-2 preview and its typed confirmation.
the platform's own code PackageUninstall.Request / PackageUninstall.Confirm; PackageUninstall.RequestAt writes at a caller-chosen id so several writers deciding the same thing collide on one node
a feature flag declared off the boot pass runs phase 1 for every installed package the flag names (Environment Composition → "A disabled flag retires what is installed"). The request carries RetiredByFlag (so it blocks no re-install) and RetireOnEveryReplica — the module's landed generation must be disabled durably or the request FAILS with the install record kept, and phase 1 never restarts by itself: it records ModuleDisabled and states Ready on the restart coordinator, which dispatches ONE restart for the boot pass's whole batch ("One restart for every retirement" above). Its phase 1 skips the user-data census and records the partition as not measured: a flag retire never drops; it names no requester and offers no confirmation, and the executor refuses any confirmation of it and any phase 2 by name, so phase 2 can never run.

What is NOT established