Classifying an issue that arrived from outside

Triage already exists, and this is not a second one. TriageIntake turns three signed event kinds — ci-failure, ci-green, feedback — into one Hosting/TriageItem and one thread with the triage agent. Every one of those is submitted internally: a portal's Feedback plugin, or a fleet repository's own node-repo-ci-failure.yml, each reaching the control instance through the pooled signed inbox (Hosting/PlatformBuilds/_Inbox) with one URL and one HMAC per portal.

The gap is the issue nobody submitted. An issue opened directly on GitHub — by a person, by an agent working a repository, by meshweaver-cloud[bot] filing an incident — never passes the inbox, so nothing classifies it. Measured 2026-09-18 on Systemorph/MeshWeaver.Plugins: of 23 open issues, zero carried a label naming the module they belong to, and the repository's label set contains no such label at all. In a repository that ships 72 modules, "which plugin is this" is answered by reading the issue.

What this is: stages, not a script

The framework is four stages with a registry at the classify step, because the first rule — which plugin owns this — will not be the last one wanted.

Adding a rule is a new classifier registered at stage 3. It is not a new pipeline, not a new inbox and not a new credential.

Stage 1 — Ingest

A fleet repository posts issues.opened to the inbox it already posts ci-failure to: same URL, same HMAC, same X-Hub-Signature-256 verification, one more kind. That reuse is the point — the pooling is the inbox, one connection per portal, never one per repository, and no repository gains a credential it did not already hold.

🚨 Core is excluded by construction. Systemorph/MeshWeaver is not a monorepo of plugins; it has no module to name. The classifier's input set for core is empty, and an empty input set is a REFUSAL to classify, never a default label.

Stage 2 — Validate, before a model reads anything

An issue body is text a stranger wrote. The moment a model reads it to decide something, it is an indirect prompt-injection surface — OWASP LLM01 — and the decision it drives is the excessive-agency surface, LLM08.

The standard checker is Azure AI Content Safety — Prompt Shields (Azure.AI.ContentSafety), which scores both jailbreak attempts and indirect injection in documents. It is the right choice here for a reason beyond its accuracy: core already pins Azure.AI.Inference, Azure.AI.OpenAI and Azure.AI.Agents.Persistent, so this is the provider family the fleet is already on, not a new vendor, a new key store and a new failure mode. OWASP's LLM Top 10 is a taxonomy — it says what to test for; it is not a library.

🚨 But the detector is defence in depth, and it is not what makes this safe. What makes it safe is that stage 3 cannot emit anything except a member of a closed set — the repository's actual module list, derived from its */manifest.lock files. A successful injection can therefore make the classifier pick the WRONG module. It cannot make it write prose into an issue, call a tool, open a pull request, or reach a portal, because stage 4 accepts an enumeration member and nothing else. Constraining the action space is the mitigation; the detector reduces how often the wrong member is chosen, and records the attempt.

Three rules this stage carries:

1. A refusal is a verdict, not a failure. Text that trips the shield is quarantined: the item is marked, the attempt is recorded with its score, and the issue is left UNLABELLED. It is never silently dropped, and it is never labelled on a guess.
2. The shield failing to answer is not a pass. No answer means no classification — the same posture as every gate here, where a check that could not run must not read as a check that passed.
3. Most issues never reach a model at all (below), so the exposure is a fraction of the volume rather than all of it.

Stage 3 — Classify: deterministic first, model only as fallback

The first classifier answers which module owns this issue, and it answers it without a model wherever it can:

• an explicit path — src/MeshWeaver.Observability/… or Observability/… — names its module;
• an assembly or namespace — MeshWeaver.Observability — maps to the module that ships it;
• the module list itself comes from the repository, not from a hand-kept table: every */manifest.lock carries a module field, which is the same source the build uses.

Only an issue whose text names no path, no namespace and no module reaches a model, and when it does the model is asked a multiple-choice question over the enumerated list, never an open one. Two answers are always available to it and both are honest outcomes: ambiguous and none.

🚨 Never invent a label. A value that is not a current module is a bug in the classifier, and the executor refuses it rather than creating the label. This is why the module list is read per run and never cached across runs.

Stage 4 — Act

One label, plugin:<Module>, applied with the caller's own GITHUB_TOKEN — the same grant node-repo-ci-failure.yml already uses for its issue writes, never a GitHub App token. Idempotent by construction: the label is derived from the issue, so re-running changes nothing, and an issue that already carries a different plugin: label is reported as a CONFLICT for a human rather than silently relabelled.

The governed action for the legacy backlog

The same four stages, run over issues that already exist, as a Governance activity — Hosting/Activity/ClassifyLegacyIssues — not a script somebody runs from a laptop. That buys three properties the backfill needs and a script cannot have:

• It is gated. The activity declares its gates the way every other governed action does (ActivityGates); nobody relabels a repository's whole backlog without the approval the gate requires.
• It is a dry run until it is not. Preview is the default and it writes nothing: it reports what it WOULD label, what it would refuse, and what conflicts with an existing label. Applying is a separate, explicit run.
• It reports as mesh data. Inputs, progress and outcome live on the activity node, so the result is queryable and auditable afterwards — not a terminal buffer somebody scrolled past.

Its inputs are the repository, an optional issue-state filter, and Preview | Apply. Its output names, per issue, the module chosen and WHICH stage chose it — deterministic path match, namespace match, or model fallback — because a classification nobody can attribute is one nobody can correct.

🚨 The backfill is where the injection surface is largest, and it is the reason stage 2 exists rather than being a later refinement: a legacy backlog is a corpus of text written by anyone who could open an issue, read in bulk, unattended, by an activity that holds a token. The closed action space is what keeps that safe; the shield is what tells us it was tried.

What this deliberately does not do

• It does not re-triage. An issue already handled by TriageIntake keeps its item and thread; this adds a label and nothing else.
• It does not route, assign or close. Those are further classifiers if they are ever wanted, and each would be judged on its own action space before it is registered.
• It does not run on core. See stage 1.

What was built, and where the design turned out to be wrong

Stages 3 and 4 and the governed backfill ship as Hosting/IssueClassificationRun (the closed set, the registry and the first rule, the executor, the run record, the watcher), with the governed half as Governance/Standards/hosting.classify-issues. Stage 1 (the webhook kind) and stage 2 (Prompt Shields) are not built; the seam for stage 2 is ValidatedIssue, which has no public constructor, so stage 3 is unreachable except through a verdict. Seven things did not survive contact with the implementation, and this section is the record of them rather than a quiet edit above.

1. Hosting/Activity/ClassifyLegacyIssues cannot be a node path. A backfill runs more than once and over more than one repository, so a fixed id collides with its own previous run. It is a NodeType — Hosting/IssueClassificationRun — whose instances live in the Hosting/Activity namespace, one node per run, and the standard's CreateNode executor mints the id.

2. "Gated" and "Preview is the default" cannot both hold of one node. A gate on a run that writes nothing spends a signature on a read. They are split: a Preview run is filed directly and writes nothing, and the governed standard exists only for the run that writes — its executor files an Apply run. The preview IS the standard's acceptance criterion.

3. An ambiguous answer is evidence, and the stage that produced it must survive. The design names ambiguous as an outcome but not as something with a stage. A rule that gets as far as the path stage and finds TWO modules knows more than "nothing matched" — it knows which two. The registry keeps the first rule's ambiguous verdict, with its stage and its candidates, instead of replacing it with a generic one. This was found by a failing test, not by reading.

4. The closed set has a third state the design does not have: UNANSWERED. "Empty ⇒ refuse" is right, but a set that could not be read IN FULL — an unreadable lock, a truncated tree answer, no App on this instance — must not arrive as an empty one, or a fetch failure against any repository reads exactly like core. ModuleSet therefore carries Answered alongside the values, one unread lock makes the WHOLE set unanswered, and both states refuse with different words.

5. The module FIELD and the directory name agree on all 72 modules today, which is precisely why the directory must not be used: the two are indistinguishable now and would diverge silently later. A lock carrying no module field contributes nothing at all rather than falling back.

6. The token is the GitHub App, not GITHUB_TOKEN. Stage 4's "the caller's own GITHUB_TOKEN" describes stage 1's webhook lane, which runs in a repository's own Actions. The governed backfill runs on the control instance, which holds no GITHUB_TOKEN and does hold the App installation token TriageIntake already reads through. An instance without a configured App refuses, naming itself.

7. none is a model answer. It is reachable in the type and honoured by stage 4 (skipped, like ambiguous), but no deterministic rule produces it: "this issue belongs to no module" is a judgement about prose. Until the fallback lands, the deterministic rules answer a module or ambiguous.