One provider key per function
The incident it answers. One OpenRouter key served every model call of the control instance —
reviews, bug-fix attempts, triage and chat — under one $900/day limit. Dispatched work spent it,
OpenRouter answered 403 Key limit exceeded (daily limit), and every review round was refused with
it until the next UTC midnight: no pull request could pass its gate. A deployment can now give each
function its own key, so a function's limit is its own.
The functions
CredentialFunctions — an open vocabulary of string constants, not an enum:
| Function | Whose rounds (the lane the shared admission seated them on — AI/AgentAdmission) |
|---|---|
Review |
reviews — the internal review (internal-review, the PR steward's review rounds and the review ensemble's halves) |
Dispatch |
bugfix, express (a red main), babysitter (the babysitter, the PR fixer, platform builds), triage — and every sub-thread of those rounds, which the admission seats on the parent's lane |
Chat |
interactive — a person's own chat, and any round whose lane is unknown: the default |
A deployment re-maps a lane with AI:CredentialFunctions:Lanes:{lane} (env
AI__CredentialFunctions__Lanes__babysitter=Babysitting) and may name functions of its own; a function
with no key of its own uses the shared key.
The configuration keys
| Key | env (pod) | What |
|---|---|---|
OpenRouter:Review:ApiKey |
OpenRouter__Review__ApiKey |
the key the review rounds spend |
OpenRouter:Dispatch:ApiKey |
OpenRouter__Dispatch__ApiKey |
the key dispatched work spends |
OpenRouter:Chat:ApiKey |
OpenRouter__Chat__ApiKey |
the key chat (and everything else) spends |
OpenRouter:ApiKey |
OpenRouter__ApiKey |
the shared key — unchanged; every function WITHOUT a key of its own uses it |
The same {Section}:{Function}:ApiKey shape works for any provider section; only OpenRouter's usage
is read on the status page.
How a key gets there. Create the keys in OpenRouter (one per function, each with its own daily
limit), then enter each through the instance's write-only Set Key Vault secrets… on its
Deployments/<id> record page and declare it in the record's keyVaultSecrets.secrets by its env key
({ "key": "OpenRouter__Review__ApiKey" } — the vault object follows the fleet naming rule,
<prefix>OpenRouter-Review-ApiKey). Nothing here is ever typed into Key Vault by hand. On the next
boot ProviderCredentialSeed carries each one onto the provider node, encrypted at rest, as
ModelProviderConfiguration.FunctionKeys[{Function}] on Provider/OpenRouter (the EU route,
Provider/OpenRouterEU, borrows them through credentialFrom exactly as it borrows the shared key).
Until a function's key is entered, that function keeps using the shared key — nothing breaks before
the keys exist.
🚨 A function key converges on its configured secret. The shared key is administered on the node
(Settings → Models) and configuration only fills it when absent. A function key has no node-side
editor, so its one administered home is the vault secret: a changed secret IS the rotation, and the
next boot re-writes the node (ProviderCredentialSeed.DecideFunctionKey). A function the
configuration stops naming is LEFT on the node — removing a key because a pod booted without its
secret would silently move that function back onto the shared key. To retire one, set its secret to
the shared key's value.
How a round is served its function's key
- The shared admission seats the round on a lane (
ThreadDispatchPool);ThreadExecutionturns that lane into the round's function (CredentialFunctions.OfLane) and sets it on the round's chat client BEFORE the agents are built (AgentChatClient.SetCredentialFunction). - The agent build hands it to the provider factory as an ARGUMENT (
ChatClientAgentFactory.CreateChatClient(AgentConfiguration, string?)— never instance state: a factory is a singleton concurrent rounds share) and the factory resolves the key OF THAT FUNCTION:ChatClientCredentialResolver.ResolveForFunction(model, function)— the provider's own key for the function, then (for a route) the referenced provider's, and only then the shared key.CredentialResolution.Functionsays whether a function key served;CredentialIdentitynames the credential (Provider/OpenRouter#Review, orProvider/OpenRouterfor the shared key). The OpenAI-wire factory (OpenRouter) honours it, including its configuration fallback while the catalog is still cold. - A client built outside an agent's round (a utility call) has no function and uses the shared key.
The admission closes ONE key, not the provider
A refusal no load change cures — a key's daily limit, credit, a rejected key — used to close every
pool of the provider until midnight. With separate keys that would still couple the functions: a
spent Dispatch key would close the very pools the reviews run in. So each round's seat carries the
credential it runs with (Seat.Credential), and such a refusal closes that credential
(AdmissionState.ClosedCredentials): its rounds wait (or move to an alternative on another key),
ONE probe goes through when nothing runs on that key, and a clean probe reopens it — while every other
key's rounds go on starting on the same provider and the same models. A closed key with work waiting
is a provider-closed starvation signal. With one shared key (no function keys) every lane carries the
same credential, so the refusal still holds every lane — the coupling only separate keys remove. When
the credential cannot be told, the old rule applies: every pool of the provider closes.
Where the spend is shown
The Daily budget & health section (/Provider/AiProviders, platform admins only —
Per-provider daily budget) lists, beside each key holder's row, one row per
function key: OpenRouter 🔑 Review, with the key's own limit, today's usage and what remains as
OpenRouter reports them (GET /api/v1/key → limit, usage_daily, limit_remaining, read with that
key, every five minutes while the section is open). A spent key, or one whose usage cannot be read
(the HTTP status, never the key), is red.
What this does not establish
- The platform's own daily budget (
DailyLimiton the provider node,ProviderBudgetGuard) still counts the key holder's spend across every function; the per-function limits are OpenRouter's, set on each key. - The in-memory catalog path (a deployment serving
Providerfrom configuration, not the DB — local development) projects no function keys: there every function uses the shared key. - Other factories (Anthropic, Azure Foundry, …) resolve the shared key; the function rung is in the resolver for all of them, but only the OpenAI-wire factory passes the function today.
- The census on
Admin/Threadsshows a closed key only as a starvation entry, not as a row of its own.
Where the code is
src/MeshWeaver.AI/CredentialFunctions.cs— the functions, the lane mapping, the configuration keys.src/MeshWeaver.AI/ModelProviderConfiguration.cs—FunctionKeys.src/MeshWeaver.AI/ProviderCredentialSeed.cs—DecideFunctionKey, the function-key pass.src/MeshWeaver.AI/ChatClientCredentialResolver.cs—ResolveForFunction,CredentialResolution.Function/CredentialIdentity.src/MeshWeaver.AI/ThreadExecution.cs,AgentChatClient.cs,ChatClientAgentFactory.cs,IAgentChat.cs— the round's function, to the factory.src/MeshWeaver.AI.OpenAI/OpenAIChatClientAgentFactory.cs— the key of the function, node and configuration fallback.src/MeshWeaver.AI/Supervision/AdmissionState.cs,ThreadDispatchPool.cs— the credential on a seat, the per-key closure.src/MeshWeaver.AI/ProviderBudget/ProviderBudgetView.cs,ProviderModelLister.cs— the per-key rows,ReadKeyUsage.- Tests:
CredentialFunctionsTest(lanes, configuration, the seed decision, the per-key closure with its negative control, the rows),CredentialFunctionRoundTest(a round's function reaches the factory),OpenRouterFunctionKeyMeshTest(seed → encrypted → resolution, the fallback, a rotation).