Attributing an Actions-budget refusal
Hosting/RepoHealth exists to stop ONE misreading: a 2-second red with zero steps is an org
GitHub-Actions budget refusing work, not a broken workflow, and the next reader must not start by
doubting the diff. It shipped able to see exactly one shape of that refusal, and the org bills two.
The two meters
GitHub bills hosted-runner minutes and shared/artifact storage as separate SKUs of the same
actions product, so ONE budget refuses both. Measured live 2026-09-22 on
organizations/Systemorph/settings/billing/usage?year=2026&month=9 โ 448 lines:
product |
sku |
unitType |
net for the month |
|---|---|---|---|
actions |
Actions Linux |
Minutes |
4,189.75 |
actions |
Actions storage |
GigabyteHours |
100.77 |
actions |
Actions macOS 3-core |
Minutes |
0.00 |
actions total |
4,290.52 โ 71.5 % of the actions budget's 6,000 cap |
The two meters refuse work in shapes that look nothing alike:
| Minutes meter | Storage meter | |
|---|---|---|
| When it bites | before a runner starts the job | inside a job that is already running |
steps |
[] โ none recorded |
every step it executed |
| Duration | ~2 s | as long as the job took |
/logs |
BlobNotFound |
a real log |
| Where GitHub says so | the job's annotation: The job was not started because an Actions budget is preventing further use. | the failed step's annotation: Failed to CreateArtifact: Artifact storage quota has been hit. |
| Looks like | a workflow the runner could not resolve | an ordinary red |
What was measured, 2026-09-17 08:50โ10:05Z. A $0 actions budget with stop-on-exhaustion armed
made every upload-artifact step in every private repository of the org fail with GitHub's storage
sentence. MeshWeaver.Manufacturing's and MeshWeaver.Crm's main runs died uploading the gate log
and the bake, and it cascaded into the platform resolver, which walks main runs looking for a
successful one and found none. The instrument built for exactly this event reported an ordinary red,
because its reading was Failed() && Steps == 0 and a storage refusal has steps.
Absence of ONE marker is not absence of the event
That predicate is the absence-of-one-marker shape, and it is the failure this page is written against. The remedy is not a second predicate of the same kind but an enumeration: every field or signature a refusal can carry, each labelled with what it is good for.
๐จ A marker that PLACES a failure in the population is not a marker that ATTRIBUTES one. Placing decides what is worth reading. Attributing decides what the finding says. Confusing the two is how a 403 on a 492-byte upload becomes "the budget refusal has recurred".
| # | Marker | Read from | Places? | Attributes? |
|---|---|---|---|---|
| M1 | job concluded failure with zero recorded steps |
/actions/runs/{id}/jobs |
yes (minutes) | no |
| M2 | the job's annotation carries "Actions budget is preventing further use" | /check-runs/{jobId}/annotations |
โ | yes โ the minutes meter |
| M3 | a failed step exists at all, with its name and conclusion | the steps array of the same jobs payload |
โ | no |
| M4 | a failed step's name names an artifact operation (artifact, upload) |
same | yes (storage) | no |
| M5 | an annotation carries the uploader's own failure wording (Failed to CreateArtifact, Failed to FinalizeArtifact, โฆ) |
annotations | yes (storage) | no |
| M6 | an annotation carries GitHub's storage sentence "Artifact storage quota has been hit" | annotations | yes | yes โ the storage meter |
| M7 | a budget on the actions product (or a SKU beneath it) with prevent_further_usage |
/organizations/{org}/settings/billing/budgets |
yes โ see the gate below | no โ see the trap below |
| M8 | the spend against that cap, summed per product AND per SKU | /organizations/{org}/settings/billing/usage |
โ | exclusion only, and only inside the window it describes |
| M9 | either billing read refused, unparseable, or never answered; the budgets page was not the last one; or a budget row is missing a required classification field | the same two calls | yes | no |
| M10 | an annotation that could not be read (non-200, unparseable, past the per-run cap) | annotations | โ | no โ UNKNOWN |
M4 and M5 are deliberately wide: the fleet's own steps are named Upload the receipts, Upload the gate log and Upload the bake, none of which contains the word artifact, so matching the
action's name alone would miss every one of them. A wide match only widens what is READ; attribution
still comes from M2 or M6, so a false positive here costs one REST call and can never produce a wrong
verdict.
๐จ prevent_further_usage is the POLICY, not the STATE
Measured live 2026-09-22 on organizations/Systemorph/settings/billing/budgets โ all five budgets
carry "prevent_further_usage": true, including actions, on a day when nothing was being refused
at all:
{"budget_type":"ProductPricing","budget_product_sku":"actions","budget_scope":"organization",
"budget_amount":6000,"prevent_further_usage":true,"budget_entity_name":"โฆ"}
The flag says the budget WILL stop further usage once exhausted. It does not say anything is being
refused. Reading it as "the budget is refusing" would have confirmed the wrong conclusion two
readers reached on 2026-09-21 โ which is worse than having no numbers at all, because it wears the
authority of a measurement. consumed_amount is documented but was absent from every live budget, so
the spend has to come from the usage report (M8), and the comparison is the whole answer:
Refusing(product):
budgets unread, or not known to be the last page โ UNKNOWN
any row whose budget_product_sku is unreadable โ UNKNOWN (for EVERY product)
a row on this product whose armed flag is unread โ UNKNOWN
no budget meters this product โ NO
every budget on it only ALERTS โ NO
an armed budget's cap is unreadable โ UNKNOWN
an armed budget's spend cannot be mapped โ UNKNOWN
otherwise โ spend >= cap, for any armed budget
A cap of zero with the flag armed is refusing from the first byte, which is the shape the
2026-09-17 window had. Two spellings meet in exactly one place: the budgets endpoint targets SKUs as
actions_storage (snake_case) while the usage report returns Actions storage (Title Case), and the
two endpoints do not even share a casing convention โ budgets is snake_case, usage is camelCase. A
target that matches no usage line reads as UNKNOWN spend, never as zero, because an unmapped name
and a quiet month are not the same fact.
๐จ The same trap, one level down: a malformed row must not default to benign
prevent_further_usage is read as three values โ true and false only from a real JSON
boolean, null for absent or any other kind โ and budget_product_sku likewise distinguishes "does
not meter this product" from "could meter anything". Letting either absence take a benign default
would be the marker-absence trap reproduced inside the fix for the marker-absence trap: exactly the
move Failed() && Steps == 0 made, one level down. A missing armed flag reading false, or a missing
sku reading "targets nothing", would let a malformed row silently exclude a real refusal.
The two clauses are kept apart because their reach differs. An unreadable sku could meter any
product, so one such row makes Refusing UNKNOWN for every product. An unreadable armed flag on
a row targeting this product makes it UNKNOWN for this one. Neither can any longer produce false, and
both are named in Describe, so a short read is visible rather than quietly benign.
๐จ The numbers describe ONE instant, so they can only exclude inside it
A cap is mutable, and this page's own evidence is the proof: the actions cap was $0 on
2026-09-17 and $6,000 on 2026-09-22. So a reading taken now cannot exclude a refusal then, and
OrgBudgetState.CanExcludeFor(instant) permits an exclusion only when the failure's own instant is at
or after ReadAt.
Billing-month alignment is not enough, and that same pair refutes it โ both readings are the same month. Within a month the spend only rises, so "spend < cap now" does imply the earlier spend was lower; it says nothing at all if the cap was lower then, which is what happened. Neither endpoint exposes budget history (measured), so the validity window is an instant, not a month.
The consequence, stated plainly: with a scan-time-only reading, budget-not-exhausted is unreachable
for any past run, so every absent-sentence artifact failure is unattributed and carries an
Unmeasured line. That is noisier and it is correct โ a board saying "nobody established this" is
worth more than a confident benign verdict over an event nothing measured. The value stays in the
vocabulary because it is what a historical reading would legitimately produce, and CanExcludeFor is
the seam that would start returning true if GitHub ever exposed that history.
Refusing() keeps its scan-time meaning for the read gate โ "is the org refusing now" is a fair
question to ask of a now-reading, and it only ever widens which annotations are fetched. What it no
longer does is exclude.
๐จ An optional signal may never fail the audit
Both billing fetches are wrapped: a timeout, a DNS failure or a reset becomes (status 0, null body)
plus a named problem, never an exception. An escaping transport failure would reach the scan's outer
Catch and replace the entire fleet audit โ six repositories, six checks โ with a skipped report,
turning an optional exclusion signal into a single point of failure for everything else. And it would
do so in a correlated way: the org's billing API is most likely to be unwell during exactly the event
this instrument exists to detect. Cancellation of the whole scan is the one thing that still
propagates.
The attribution rule
Precedence, and each step's reason for existing:
- The evidence could not be read โ
unread. Nothing is known, which is not the same as "not the budget". - GitHub's storage sentence is present โ
storage-quota. The sentence attributes and nothing overrides it โ not the numbers, not the step name, not how plausible another story is. - The numbers say a refusal IS in force at scan time โ
budget-exhausted-no-sentence. The strongest unattributed case: a refusal is armed and exhausted, an artifact step failed, and GitHub printed no sentence โ which is what a MALFORMED refusal looks like. Reported loudly, with the numbers that raise it, and still not attributed: a budget that could refuse is not a budget that did. - The numbers exclude a refusal AND they describe this failure's own instant โ
budget-not-exhausted. The budget question is ANSWERED, with a no. The finding then says nothing about what DID cause the failure, because it does not know. - Otherwise โ
unattributed. The sentence is absent and nothing available could exclude a refusal either โ the numbers are UNKNOWN, or they describe an instant that is not this one โ so the budget can be neither attributed nor excluded. Each entry carries why.
๐จ The numbers are taken at SCAN time, so they can never establish that a refusal was in force when the run ran. That asymmetry is why they may only ever exclude, and why an unknown or in-force reading widens the evidence read rather than narrowing it. Clause 4's second condition is where that asymmetry is enforced rather than merely observed โ see below.
Cases 1, 4 and 5 also add a line to the report's Unmeasured, which forces its verdict away from
clean. Cases 2 and 3 do not: both are answers.
The vocabulary is OPEN
The five values live in ArtifactFailureAttribution as const strings, never a C# enum (policy
open-vocabulary-string-constants). They are persisted inside a Hosting/RepoHealth node's content
and a deployment may add its own; an enum would break every exhaustive switch the day one is added,
and would deserialise an unknown value into its zero member โ a meaningful value, and therefore a
silent wrong answer. A value outside ActionsBudgetRules.KnownAttributions is counted as none of the
five, reported by name in the finding, and folded into nothing.
The read gate is a conjunction
One failed job's annotations are fetched when ANY of these holds โ and the benign answer needs ALL of them absent at once:
- the job never started (M1), or
- a failed step's name places it in the artifact population (M4), or
- the org's budget state is not positively excluded โ
trueand UNKNOWN both read the evidence (M7 + M9).
So the only job whose evidence goes unread is one that ran, failed at a step nothing places in
the artifact population, in a window the org's own numbers say was not refusing. The probe and
the rules share the single predicate MainBranchRules.NeedsAnnotation, so the set that is fetched can
never drift from the set the rules reason about โ which is precisely how the original defect survived
review: the probe's Where(j => j.NeverStarted()) and the classifier's own filter agreed with each
other and were both wrong.
An ordinary red therefore still costs no extra read, and a green branch costs nothing at all.
The four controls
A rule that CHOOSES among members of a set cannot be tested by one member, and a control that cannot
fail on the path being changed is not a control over it. All four live in Hosting/RepoHealth/Test:
| Control | Input | Must read |
|---|---|---|
| A real storage refusal | two jobs that RAN for ~15 min, each failing at Upload the โฆ, annotation carrying GitHub's sentence; actions at 71 % of cap |
actions-budget, Critical, title says refused at an artifact upload and NOT no failing job ran |
| A 403 with no sentence | the same jobs, annotation Failed to FinalizeArtifact: โฆ (403) Forbidden; measured 2026-09-21 |
main-health in every arm. Numbers read AFTER the jobs (the realistic case): unattributed + an Unmeasured line. Numbers ALIGNED with the failure: budget-not-exhausted, nothing unmeasured. Numbers UNKNOWN: unattributed. Numbers REFUSING: budget-exhausted-no-sentence. Never actions-budget |
| An ordinary red | one job that ran and failed at Build the solution; numbers say not refusing |
main-health, no budget clause, no artifact clause, nothing unmeasured, and the annotation lookup is never consulted (the test's lookup throws) |
| An unreadable billing answer | the state a failed transport produces | UNKNOWN for every product, the failure named, unattributed rather than excluded, an Unmeasured line, and the per-repository audit still producing its findings for EVERY repository โ while a green branch stays green |
Two more cases pin the narrowings rather than the choice: a malformed budget row is UNKNOWN for every product (and the unchanged live payload still reads a definite NO, so the rule has not simply been replaced by "unknown"), and a scan-time reading cannot exclude an earlier run (with the aligned reading as the other side).
What this does NOT cover
- A malformed refusal in a degraded window. If GitHub prints neither its storage sentence nor the uploader's wording, the failed step's name is unrecognisable, AND the billing reads are refused, nothing here sees it. The gate widens on an unknown billing read precisely to shrink this to that three-way conjunction, but it does not close it.
- Whether a refusal was in force when the run ran. The numbers describe one instant, and the code
now acts on that rather than merely noting it (above) โ so the gap shows up as
unattributed+Unmeasuredinstead of as a wrong exclusion. It is a stated hole, not a closed one: neither endpoint exposes budget history. - The probe's own transport
catch. Its consequences are controlled (above), but the in-node harness reflects over public static methods and the fetch is async, so no in-mesh case executes the HTTP hop itself. - Cross-repository simultaneity. A budget refusal hits every private repository in the same window, and the scan reads six repositories in one pass โ so red runs with artifact failures in two different repositories inside one window would be org-wide evidence no single annotation can give. Deliberately not implemented: the finding rules are per-repository, and a report-level inference is a different shape of change.
- Whether the fleet's GitHub App can read billing at all. The two calls were verified live with a person's credential, not with the App's installation token. If the App holds no billing grant the numbers are simply UNKNOWN on every scan, which widens the evidence read and is said in the report โ it is not a silent degradation, but it is a real reduction in what can be EXCLUDED.
- Firing the scan at all. RepoHealth is still entirely human-driven; see /Hosting/RepoHealthScheduling.
Where it lives
| The budget reading, the vocabulary, the attribution rule | Hosting/RepoHealth/Source/ActionsBudgetRules.cs |
The step reading, the read gate, Classify, the finding wording |
Hosting/RepoHealth/Source/MainBranchRules.cs |
| The two billing calls, once per org per scan | Hosting/RepoHealth/Source/RepoHealthProbe.cs |
| Every case above | Hosting/RepoHealth/Test/RepoHealthTests.cs |
Related: /Hosting/RequiredStatusChecks ยท /Doc/Architecture/ReadingCiSignals ยท Memex#275.