Ordered context compaction

The model re-reads the whole conversation on every round, so a long thread costs more each round and eventually stops fitting the model at all. Until 2026-11 the engine had one answer to that: AgentChatClient.TruncateToContextBudget kept the system prompt plus the newest messages within the round's budget and dropped the oldest. The agent then forgot how the conversation began: the decisions, the paths it had been given, the task it was asked to do.

Ordered compaction reduces the history in a fixed order, cheapest and least lossy first, and stops as soon as it fits (owner directive, 2026-11-12: "an ordered compression of context when it's too big").

The budget

The budget is the same as before: AgentChatClient.HistoryBudget, which is 21,000 estimated tokens (MaxContextTokens) or what the model's declared context window leaves after the output cap and the tool definitions, whichever is less. Tokens are estimated by ContextBudget (every content kind, three characters per token). Compaction starts only when the round's messages exceed that budget, so a history exactly at the budget is sent unchanged.

The order

Step	What it does	Loses
1. Old tool results	Replaces the payload of OLD tool results, oldest first, with a stub that names the tool, the call id, the size and the first 160 characters of what it returned. The newest two results stay verbatim. Only the payload changes, so every `FunctionCall` keeps its `FunctionResult` and call id.	Bulk the agent can fetch again
2. The persisted summary	Puts the thread's stored summary in place of the messages it covers (everything up to `ToMessageId`).	Nothing new
3. A new, incremental summary	Summarises the stored summary plus the NEXT-oldest span into ONE new summary, and stores it.	Wording, never facts
4. Drop the oldest	As before (`TruncateToContextBudget`), with the system prompt and the summary pinned. Logged as `[Compaction] TRUNCATED`.	Messages

Each step runs only if the steps before it left the history over budget. The system prompt and the newest user message are never touched: a span never reaches the newest user message, and the summary never covers the newest message.

Step 1 also runs inside the tool loop. ContextWindowGuardChatClient sits below FunctionInvokingChatClient and sees every iteration. When an iteration no longer fits the declared window it first shrinks that loop's oldest results, and refuses the request only if it still does not fit. It uses the same declared KeepRecentToolResults and ToolResultPreviewChars as the round: the round stamps its options on the request (ContextCompactionOptions.OptionsKey), and the guard removes that stamp before the provider sees the request. A tool-heavy round therefore runs longer before it hits ContextWindowExceededException, and every request it sends still fits the window.

The summary

One message, in chronological position. It is sent as a single user-role message right after the system prompt, prefixed with how many messages it replaces and its generation. It is marked (ContextCompaction.SummaryMarkerKey) so truncation keeps it pinned.
What it keeps. The summariser's instructions (ContextCompaction.SummaryInstructions) ask for decisions and who took them, established facts, open tasks and questions, commitments, and every identifier verbatim (paths, URLs, ids, issue numbers, names, numbers, dates). They forbid inventing, inferring or embellishing: anything unclear is reported as unclear. An empty answer counts as a failure, not as a summary.
Incremental. Each compaction folds the previous summary plus the next-oldest span into one new summary (generation + 1). The whole thread is never re-summarised. A summarising pass aims at 60% of the budget (TargetFill), so the following rounds have headroom and usually need only step 2, with no summary call.
Never split a tool exchange. A span that would end on a tool call is extended over that call's results, so the kept part never starts on an orphaned result.

Durable on the thread

The record is stored in Thread.Compaction (ThreadContextCompaction), a field the thread owns:

Field	Meaning
`summary`	The text that replaces the covered messages
`fromMessageId` / `toMessageId`	The oldest and newest cell the summary covers
`coveredMessages`	How many messages it stands for, over every generation
`generation`	1 for the first summary, +1 per incremental fold. A write never replaces a higher generation (`ContextCompaction.Merge`).
`model`, `compactedAt`	Who wrote it and when (UTC)
`lastFallback`, `lastFallbackAt`	Why the last attempt dropped messages instead, or null

The round reads the record with the history (ThreadExecution.ReadThreadCompaction, 3 s bound; a failed read means compacting without it and never blocks the round). The summary is located by cell id: the history loader stamps each message's ChatMessage.MessageId with its ThreadMessage cell id. The round writes the record back through the thread's own stream, as System, in the same way as the history witnesses. Because it lives on the node, a recycled thread hub reads it back and does not re-summarise.

Visible in the thread. The thread's menu has a Context page (ThreadNodeType.CompactionArea). It is a template bound by pointer to Thread.Compaction, so it updates the moment a round records a compaction. It shows how many messages were compacted, the first and last covered message, the generation, the model, the summary itself, and the reason when the last attempt fell back to dropping messages.

Failure never blocks the round

A summariser that throws, answers with nothing, or does not answer within Timeout (45 s by default) falls back to step 4. The summariser call is bounded by WaitAsync as well as its cancellation token, so a provider that ignores cancellation cannot hold the round. The fallback is reported in three places:

a [Compaction] FALLBACK — <reason> warning in the log;
lastFallback on the thread record;
the warning on the Context page.

A stored summary that was valid stays applied. A cancelled round propagates its cancellation instead of falling back.

Configuration (declared, never hard-coded)

Section AI:ContextCompaction. Every key is optional.

Key	Default	Meaning
`Enabled`	`true`	`false` skips step 3. Steps 1, 2 and 4 still run.
`Model`	(unset)	The model selection (wire id or LanguageModel node path) that writes summaries. Unset means the thread's own model, called tool-free on the round's own client.
`TimeoutSeconds`	`45`	The bound on one summarisation call
`KeepRecentToolResults`	`2`	How many of the newest tool results are never shrunk
`ToolResultPreviewChars`	`160`	How much of a shrunk result its stub keeps
`SummaryMaxOutputTokens`	`4000`	The output cap of a summarisation call
`TargetFill`	`0.6`	The share of the budget a summarising pass aims for (0.1–1)

🚨 A single-slot local model. On the thread's own model, the summary call runs before the round's main call, never beside it, so it does not compete with the round for a single local slot. If a deployment declares Model, it should be a cloud utility model. Pointing it at a local Ollama model that other threads also use queues summaries behind their rounds, which is the saturation that once took search down. A declared model that resolves to no client counts as a summariser failure and is reported as such. It is never replaced by another model. The summary call's token usage is logged ([Compaction] summary call on …) but is not charged to the round's ledger.

Code

src/MeshWeaver.AI/ContextCompaction.cs: the order (CompactAsync), the steps (ShrinkOldToolResults, ApplySummary, SelectSpan), the summary request, the record and the options
AgentChatClient.CompactToBudgetAsync: the round's call site, the summariser and the persistence
ContextWindowGuardChatClient.Compact: step 1 inside the tool loop
ThreadExecution.ReadThreadCompaction and the history loader's MessageId stamp
ThreadLayoutAreas.CompactionView: the Context page

Tests: src/MeshWeaver.AI.Test/ContextCompactionTest.cs covers the order, pairing, span boundaries, incremental folds, the fallback on failure, timeout and empty answers, the budget boundaries, the newest-message guarantee, config, the Context page and the tool loop. ContextCompactionPersistenceTest.cs covers the record surviving a real recycle and the history carrying cell ids.