Wrong answers — reporting the chat's mistakes
/feedback is for formal defects: a broken link, an error card, a timeout, a button that does
nothing. It hands anything that reads as technical to the Systemorph triage agent, which may file
a GitHub issue. That is the right addressee for a defect and the wrong one for this:
"The coach said it doesn't know my hourly rate. It is in the Firmenprofil."
Nothing is broken. The chat gave a poor answer — and whose problem that is depends on why it
was poor. Sent through /feedback, the words "doesn't know" and "wrong" match the technical
classifier (FeedbackHandover.LooksTechnical) and the report becomes a GitHub issue on a code
repository, where nobody can act on it. /wrong-answer (Feedback/Skill/wrong-answer) exists so
the report lands with the right person, carrying the evidence that person needs.
What the chat can know about its own answer
Every assistant message is a ThreadMessage node under the thread, and it already records what a
reviewer needs: agentName, modelName (and requestedModelName when a different model answered
than was asked for), harness, every toolCalls entry with its arguments and result, and
updatedNodes. The skill reads that cell — the user does not have to describe it.
It also runs under the user's identity. A node the skill cannot get is one the answering agent
could not read either. That single check separates "the chat is broken" from "the user lacks a
grant", and it is the reason the skill classifies by checking rather than by what the user
believes.
The kinds, and who each one belongs to
| Kind | What happened | Decided by | Addressee | What they do with it |
|---|---|---|---|---|
fabricated |
asserted a fact, number, name or path the mesh does not hold | a search for the claim finds nothing | the agent's owner — the partition hosting the agent node ({Module}/Agent/x → the module's maintainers; {viewer}/Agent/x → that user) |
tighten the instructions (a grounding rule: cite the node or say you cannot find it), reconsider the ModelTier, add the case to the agent's eval set |
no-access |
said "I don't know", the user says it is in the mesh, and the skill cannot read it either | get of the named node is refused |
the space owner of that node | grant, or explain why not. Nothing for the AI team — the chat could not have known |
retrieval |
the node is readable, a Search/Get in the answer's tool calls should have found it and did not |
get succeeds AND a missing search is on record |
the AI engine (src/MeshWeaver.AI — search, indexing, chunking), via the normal triage hand-over to GitHub |
fix, and pin it with a test — this is the one kind that is a platform defect |
not-found |
the node is readable and the answer never looked | get succeeds AND no such tool call |
the agent's owner | an instruction that says search before you say no |
wrong-source |
faithful to a node whose content is wrong or stale | the cited node says what the answer says | the content owner of that node | fix the node; the agent was right |
unasked |
created, changed or deleted something the user did not ask for | updatedNodes names nodes the request did not cover |
the agent's owner | a scope rule in the instructions; possibly a plugin removed from the agent |
other |
none of the above | — | the Inbox reviewer | classify by hand |
Only retrieval is filed as category: bug and handed over after Submit. Every other kind is
filed as category: answer, which FeedbackHandover.ShouldHandOver keeps in this instance's Inbox
regardless of how technical the message reads — the addressee is a person on this instance, not a
repository.
What the report carries
The same Feedback/Feedback node /feedback files, so preview, Submit, the Inbox and the review
lifecycle are shared. The difference is the content: the user's account verbatim in message, the
thread and message in sidePanel, and a fixed labelled block in extraContext:
Kind: not-found
Thread: sglauser/_Thread/2026-09-18-1319
Message: sglauser/_Thread/2026-09-18-1319/a7f3
Agent: OfficeCoach
Model: claude-sonnet-4-5 (requested: same; harness: native)
Tools called: 1 — Get(AgenticOffice/01-Willkommen) → ok
Answer (excerpt): "Ich weiss leider nicht, welchen Stundenansatz du verrechnest…"
Expected: "Konzept-Ansatz CHF 150 steht im Firmenprofil"
Evidence: get sglauser/AgenticOffice/Firmenprofil → readable; no Search in toolCalls
Addressee: agent owner (AgenticOffice/Agent/OfficeCoach — the course's maintainers)
Labelled lines rather than typed fields, deliberately: the volume does not yet justify a schema
change, the card renders extraContext as-is, and a later aggregation can parse Kind: / Agent:
/ Model: without a migration. When it does justify one, the fields to add to FeedbackContent
are exactly those five.
What to do with the reports
Routing is the first use, not the only one. In order of value:
- Route to the addressee (the table above). Today the Inbox reviewer does this by reading
Kind:andAddressee:off the card and moving the item to Triaged;retrievalalone routes itself. The step to automate next: raise aNotificationsatellite on the agent node for the agent-owner kinds, so a course author sees their coach's misses without watching a shared Inbox. - Turn every confirmed case into an eval case. A report is a labelled example — this prompt,
in this context, must ground its answer in node Z / must not assert F. Kept beside the agent
(
{Module}/Agent/{x}/Eval/{case}) and replayed whenever the instructions or the model change, these are the only defence against fixing one hallucination and introducing another. This does not exist yet; the report format above is designed so it can be lifted into one without re-asking the user anything. - Aggregate by model and by agent. A count of
answerreports perModel:over a window is the cheapest quality signal the platform has, and it should informProvider/AiModelTiers— a model that fabricates on theUtilitytier is doing exactly the job that tier must never fail at. PerAgent:, the same count says which instruction sets need work. - Close the loop. The reviewer's Detail view and the Triaged / Resolved / WontFix lifecycle already exist; a Resolved report whose addressee changed something should say what, so the reporter learns the chat got better because they said so.
What this is not
- Not a place to fix things. The skill never edits the node, never grants access, never re-runs the search and presents that as the answer. It records and classifies; a person acts.
- Not a verdict. The agent filing the report is usually the agent being reported. Its classification is a hypothesis with its evidence attached; the reviewer decides.
- Not a substitute for
/feedback. A broken preview, an error card, a timeout:/feedback.
Tests
Feedback/Feedback/Test/FeedbackTests.cs—HandOver_BugAndIdeaAlways_PraiseNever_QuestionWhenTechnicalpins thatcategory: answeris never handed over, however technical the message reads, and that aretrievalreport filed asbugis.- Follow-up: an e2e beside
e2e/feedback-skill.spec.tsthat drives/wrong-answerin a thread and asserts the draft'sextraContextcarries every labelled line.