Which portal an incident fired on

Systemorph/MeshWeaver#7334.

An incident is one node per fingerprint (Admin/_LogIncident/{fingerprint}), and the fingerprint deliberately leaves the namespace and the pod out — the same defect on two portals is one ticket. So a single incident folds bursts from every deployment the fault fires on.

What was measured

Read on the control instance: one pod name was attributed to memex-cloud on one incident and to memex on two others. The incident carried

Both namespaces run a deployment named memex-portal-deployment, so only the pair namespace + pod identifies a replica, and the incident had thrown one half of that pair away. Deciding whether an occurrence was before or after a fix rolled needs exactly that half: rolls are staggered, so the same wall-clock instant is post-roll on one portal and pre-roll on the other.

What changed

The watcher queries Loki one namespace at a time, so every report names exactly ONE namespace and all of its pods belong to it. The ingest now keeps that pairing in content.namespaces[], one row per namespace:

field meaning
namespace the Kubernetes namespace — one deployment
pods the pods of THIS namespace the fingerprint fired on
firstSeen / lastSeen the first and last occurrence in THIS namespace
occurrences the lines folded in from THIS namespace

A first sighting starts the ledger; a recurrence adds to its namespace's row, or adds a row; the corpus migration unites the ledgers of a legacy incident and its successor. A retried delivery is recognised before the fold, so it is not counted twice in its row either.

The filed issue prints one row per namespace instead of the old Namespace + Pods pair, and the recurrence comment says how often and how recently each namespace fired.

Reading an incident