When the model's stream ends badly

An OpenAI-compatible gateway can end a streaming response in ways the wire protocol does not describe. When it does, the round ends — but it ends naming the condition, in a sentence the person can act on, with everything that was streamed before the fault kept. This page records how, and the one thing the design deliberately does not do.

The three endings, and the guard that recognises each

ProviderStreamFault (src/MeshWeaver.AI/ProviderStreamException.cs) names them:

Fault What the transport did Found by
Stalled The connection stayed OPEN and the bytes stopped StreamStallGuardChatClient — an idle bound (AiStreamingLimits)
Faulted A payload arrived that the protocol cannot represent OpenAIWireStreamGuard — the SDK's own deserializer threw
Truncated The body ended before the response did StreamStallGuardChatClient — end-of-stream where a chunk was owed

A stall is found by a bound and a truncation by an exception precisely because they are opposite transport states; conflating them would make one of the two undiagnosable.

The canonical Faulted case is OpenRouter's finish_reason: "error", which the family uses to signal a fault at the upstream model provider. That value is not in the OpenAI specification, so OpenAI.Chat.ChatFinishReasonExtensions.ToChatFinishReason throws ArgumentOutOfRangeException: Unknown ChatFinishReason value. … Actual value was error. while deserializing the SSE chunk.

Why the payload guard lives in MeshWeaver.AI.OpenAI and not in the engine. The trigger is one SDK's specific throw, and that module owns that SDK. A general "an unexpected exception during deserialization means the provider is at fault" rule in MeshWeaver.AI would relabel our own deserialization bugs as provider faults. The match is narrow for the same reason — the "Unknown <Enum> value." message family only — so an ordinary range violation keeps reporting itself.

Why every guard sits BELOW FunctionInvokingChatClient. The scope has to be the model call and never a tool invocation, or a tool's own ArgumentOutOfRangeException would be reported as a gateway fault. The two guards compose: the engine-wide stall guard wraps the OpenAI-wire one, and each catches only what it can actually recognise.

One detail in OpenAIWireStreamGuard looks like defensiveness and is not: disposing a provider stream that faulted mid-parse makes the SDK's transport try to re-buffer content it has already consumed, and the exception coming out of that would replace the ProviderStreamException on its way up — the round would report a cleanup artefact instead of what killed it. The dispose failure is therefore logged and not allowed to become a second way for the round to fail.

What the round does with one

ThreadExecution's terminal-error path (FindStreamFault) names the condition ahead of the HTTP-status switch, because a stream fault carries no status and would otherwise fall through to ex.Message — SDK internals standing in for the thread's account of why it failed. It then:

The localized string is taken only when the loaded platform's catalog defines the key; otherwise the exception's own purpose-written English stands. That check is what lets this module ship a condition before the platform image carrying the string does, instead of rendering a raw chat.modelStream… token at a user.

The live reading (issue #2131)

#2131 was filed automatically from incident Admin/_LogIncident/9b70b639c4e77af3 and reported that the round "dies … as an unhandled ProviderStreamException" with "no response and no user-readable explanation, only a server-side error". The incident's own second occurrence falsifies that. Read on memex.meshweaver.cloud at rbuergi/_Thread/read-only-test-do-not-draft-send-reply-m-d748/15111171, the cell the 2026-09-17 04:44:44Z occurrence produced:

status      : Error
modelName   : qwen/qwen3.8-27b
text        : *Error: The language model 'qwen/qwen3.8-27b' ended its response with a provider
               error. This is a fault at the upstream model provider, not a problem with your
               request — submit again to retry, or pick a different model.*
toolCalls   : 2 × SearchMail, both isSuccess:true, results intact
timing      : modelCalls[1].outcome = "Faulted"
tokens      : 14761 in / 142 out / 14903 total

The thread root carries no status, i.e. Idle (Thread.Status's default), and its summary is that same sentence. So the turn ended as a named, localized, reportable outcome with the partial work preserved — not as a lost round. The [ThreadExec] ERROR log line the bot filed on IS the handled path; the guard and the round's branch both landed in the #2632 work on 2026-08-29, before either occurrence, and core's two catalog keys landed 2026-09-07.

The decision: the round does NOT retry automatically

#2131 asked for one — "retry the round once (the gateway already told us it is transient)". That is deliberately not done, and this is the record of why:

  1. A retry is a bound spent, not a defect fixed. The house rule is root cause only; an upstream gateway's health is not something this process can change, and a silent retry converts a visible upstream condition into an invisible one. The number of retries then becomes the knob, which is exactly the shape the no-band-aids rule refuses.
  2. The round is not idempotent. The 2026-09-17 reading above shows two SearchMail calls that had already succeeded when the stream faulted. A retried round re-runs the tool calls. For a read that is waste; for a write — a draft, a mail, a mesh node — it is a second side effect with nothing to deduplicate it, and the Executive Assistant's tools are precisely that kind.
  3. The tokens are already spent and already charged. A retry doubles the cost of a round the user may not want repeated on a model that is currently failing; "pick a different model" is often the better remedy and only a person can choose it.
  4. The remedy the message names is one click. hub.ResubmitMessage is wired to the errored cell (ThreadMessageLayoutAreas), so "submit again to retry" is an affordance and not advice.

A retry that is genuinely wanted is therefore an explicit, per-agent policy decision with a bound and a ledger — not a catch in the execution loop. Nothing in this design prevents that; it just is not what "handle the fault" means.

Where the code is