Storage connections

Maintainer, 2026-10-03: "pls implement … the e2e full scenario for connecting storage" — the first step of the document-ingestion end vision (core Doc/Architecture/DocumentIngestionEndVision, gaps G4, G18, G5 and G6). This page is the design of record; the executable proof is the storage-connection spec.

The scenario

Step What happens Where it lives
C1 The instance's Storage section lists its pre-configured stores — its own PostgreSQL and blob account — and their containers, live. core StorageSettingsTab (#6014) over the IInstanceStores the hosts register (MeshWeaver.Plugins #2733).
C2a Path A: a platform admin binds an existing container to partition P. AttachStorageRequest → Admin
C2b Path B: the admin names a new container; the store creates it with the instance's own identity (IInstanceStore.EnsureContainer). same request, createContainer: true
C3 The binding is a typed StorageBinding node in P ({P}/_Storage/{name}, purpose Originals) — store id and container, never a secret. core StorageBinding
C4a P's doc-part storage is not provisioned, so the connection parks: it opens a governed activity (storage.provision-doc-parts) awaiting the approver the user named, and waits for exactly one event — a write to {P}/_Storage/{name}-parts. StorageConnection hub + Governance
C4b The approver signs; the activity's executor writes the doc-part binding; core's binding hub creates the container; that write resumes the connection, which records the event and the node version it resumed at. Governance executor → core validation → StorageConnection hub
C5 The connection reads Active; a document placed in the container is listed and readable through the connection, read-only. the connection's originals collection
C6 Access follows P's grants: a user with no grant on P sees neither the binding nor the documents. SatelliteAccessRule / StorageBindingAccessRule, the content route's Read check
C7 Code that knows nothing of connections is unaffected: other partitions resolve their storage as before, and core alone validates the binding. core IStorageBindingResolver
C8 Detach stops access; the originals stay byte-identical. DetachStorageRequest → Admin

The nodes

{P}/_Storage/{name}          StorageBinding    purpose Originals — the container, typed, no secret (core)
{P}/_Storage/{name}-parts    StorageBinding    purpose DocParts — written by the approved activity (core validates + creates)
{P}/_Ingest/{name}           StorageConnection the connection: state, the event it is parked on, what resumed it
{P}/_Ingest/{name}/provision Governance/Activity  the governed provisioning, in the partition it governs

Everything lives in the partition: P's grants decide who reads it, and a global admin's role grants nothing here (Doc/Architecture/AccessControl). Attaching is a platform action: the Admin hub checks the caller is a global admin and writes the two nodes as System on the admin's behalf (attachedBy records who). It grants nobody anything.

Attach — what is checked, in order

StorageAttach.Attach answers on the Admin hub and writes nothing until every check passed:

  1. the request's shape — a plain-id name, a single-segment partition that is not Admin, a container, an approver, and not writable (a connection is read-only; the flag is spelled so its type-default is the safe value, because the hub serializer drops a value equal to its type default — a readOnly: false with a true initializer would arrive as true);
  2. the caller is a global admin;
  3. the store exists and the container name passes the store's own rule and the partition's ownership rule (StorageContainerOwnership: a partition binds only containers named for it);
  4. the partition exists;
  5. the approver holds Update on the partition — the right the provisioning needs;
  6. the binding does not exist yet;
  7. the container exists (path A) or the store created it (path B).

Park and resume — by event, never by polling

The connection's own hub decides its state from three nodes — itself, the doc-part binding and the provisioning activity — each watched through a synced query (StorageConnectionNodeType.Watch). The decision (StorageConnectionNodeType.Decide) is pure and a fixed point: once its step is written, the same inputs answer nothing to do, so a pass never feeds on its own writes, and a recycled hub re-reads the nodes and lands on the same answer.

Connection Doc-part binding Step
Attached missing / not validated Park: open the activity, record parkedOn = node-write:{P}/_Storage/{name}-parts
Parked exists, Pending Wake it: a read starts its hub, whose watcher validates (and creates the container)
Parked / Attached usable (Valid for its current target) Activate, recording resumedBy = {kind, key, stream, checkpoint = the binding's version}
Parked activity Rejected / Failed / Refused record why, once; stays parked — detach and attach again to propose anew
Active / Detached anything nothing

The standard storage.provision-doc-parts (Governance) is proposed only by the platform (system-security) and signed by {inputs.approver} — Governance admits a signer named by an activity input only as exactly that ONE plain identity: an input can never widen to * or a prefix pattern, and a missing input admits nobody (AuthoritySpec.Allows(entries, identity, inputs)).

🚧 The resume is a node-stream event today, not yet a durable stream. The connection's hub is subscribed to the binding node; a recycle re-subscribes and re-reads, so nothing is lost and nothing polls. When durable streams land (core #6018, gap G3), the subscription moves onto the stream and resumedBy.stream names it.

Reading the documents

The connection serves its container as the node's originals collection — read-only, unpublished, not exposed to children (DefaultOriginalsCollectionFactory.ReadOnly). A reader resolves {P}/_Ingest/{name}/originals/{file}: the collection's configuration is fetched from the connection's hub under a Read check, so the partition's grants decide; the bytes are then read from the store in the reader's process. A blob container maps to the instance's own account (the default blob client), a directory to {Storage:BasePath}/content/{container}.

Detach marks the connection Detached, deletes the Originals binding and recycles the connection's hub, whose fresh activation serves nothing. The container and its files are never touched; a store offers no delete.

What is proven where

Open