RFC: a stored status claim should carry its basis, and out-of-band work should leave an in-band pointer

[amrmelsayed]

RFC: a stored status claim should carry its basis, and out-of-band work should leave an in-band pointer

Sibling of #1711, and deliberately not folded into it — see Why this is not #1711 below.

Principle

1. A stored status field should carry the basis that produced it, co-located with the claim.

2. When authoritative work happens outside the tool, the record should carry an in-band pointer to where that authority lives.

The incident

A Shannon SPIR project's status.yaml produced three mutually inconsistent readings of the same phase, each individually defensible. Evidence below is first-hand from the project directory and git log.

(a) The status field is unqualified. phase_2 carries status: complete and nothing else.

(b) The basis for that field exists — in a different artifact. The phase was force-advanced at porch's safety ceiling, recorded only in the commit message: chore(porch): 5189 implement force-advance (safety ceiling reached at iter 3). A reader of status.yaml has no signal that a basis exists in git log, and no reason to look.

(c) The same file contradicts the field. Further down, the review iterations for plan_phase: phase_2 carry verdict: REQUEST_CHANGES.

So a reader who trusts the summary field concludes passed; a reader who scrolls to the detail concludes not passed. Both are wrong. The phase genuinely passed, in a review round that ran outside porch (a multi-model round adjudicated in GitHub comments). Nothing in the file could record that, and nothing in it says so.

⚠️ That is the part worth generalising: the record was not stale — it was structurally incapable of being right, and carried no marker of that incapacity. A reader cannot distinguish "this did not happen" from "this happened somewhere I do not index." Two separate reviewers, independently, read the missing record as a missing review. One of them had personally run the review that was missing.

(d) A third instance in the same file: evidence pointers that resolve on one machine. The verdict entries reference their consult transcripts by absolute local filesystem path into a builder worktree. For any reader on any other machine — or after that worktree is cleaned up — the pointer is dead. This is the same defect as (b): the basis is stored elsewhere, and here the pointer to it does not survive leaving the author's machine.

(e) A copied stale title. phase_3's title in status.yaml describes a design that was replaced; the plan deliberately kept its original title with an amendment beneath it (to preserve evidence of the redesign), and porch copied the title without the amendment. A copy inherits the claim and not the evidence.

Why this is not #1711

#1711 asks that a command report the state it is acting on — it protects the operator at action time.

This asks that a stored record carry its basis and point at out-of-band authority — it protects the reader at interpretation time.

They are complementary rather than overlapping, and neither fix prevents the other's incidents: #1711's fix would not have helped here, because nobody was running a command — they were reading a file days later. Same origin (porch state misleading readers), different moment, different reader.

Sketch of a fix

Not prescribing the design, but the shape:

• A status field that was set by anything other than its normal path (force-advance, manual override, ceiling) says so in the field's own vicinity, not only in a commit message.
• Where a phase's authority is external, the record carries a URL. One line naming where the review lives would have prevented every downstream error in this chain.
• Evidence references are stored relative to the project, or as durable URLs — not as absolute paths into one machine's worktree.
• Copied fields (titles) either carry the amendment or carry a marker that survives copying.

Scope note

Raised from the Shannon side as a consumer of codev; the placement call is the codev architect's. Filed here rather than in Shannon's architecture docs deliberately — a rule about porch's records, written in a document porch never reads, would reproduce the very co-location defect it describes.

[amrmelsayed]

Further instance (f), from the same pilot — worse than (d).

Instance (d) above was: evidence referenced by an absolute local filesystem path that resolves on exactly one machine.

This one is a layer further down: the consult verdict files that status.yaml cites are gitignored by project rule and exist only inside the builder worktree — so they are destroyed at afx cleanup. The status record does not merely point at a path that will not resolve elsewhere; it points at artifacts that are never persisted anywhere and are scheduled for deletion by the normal end-of-work flow.

The consequence is worth stating because it is not recoverable later: once the worktree is cleaned, any future citation of a verdict rests on someone's paraphrase, with no way back to the source. A summary of a destroyed artifact is unfalsifiable forever after.

That suggests a narrow addition to the sketch in the body: a record should not cite evidence with a shorter lifetime than the record itself. Either the evidence is persisted where the record lives (committed, or inlined into the record), or the record says plainly that its cited evidence is ephemeral and will not survive cleanup.

Workaround being used on the Shannon side for now: the verdicts are pasted verbatim into the tracking issue before cleanup, rather than summarised — so the citable artifact outlives the worktree.

[amrmelsayed]

The inverse case — and it completes this issue rather than extending it

This issue argues the producer side: a stored status claim should carry its basis, because readers cannot otherwise tell a claim from an assertion. Every instance so far was of one shape — the record says DONE and the basis is absent or elsewhere.

I have now made the opposite error, against myself, and it is the same defect from the other end.

What happened

I found a tracker item that was open, labelled critical, and unassigned for three weeks, and concluded from that alone that the work was undone. I then reasoned publicly about the resulting exposure.

The work was not undone. It was a staged rollout — first stage shipped two weeks earlier, second stage shipped two days earlier. The issue was open because staged rollouts stay open through their intermediate states, which is correct behaviour for a tracker and says nothing whatsoever about the system.

⚠️ And the detail that makes it worth writing down: the datum I cited as evidence of neglect was the evidence of execution. I pointed at the last-touched timestamp as proof of staleness. That timestamp was the operator posting their dig receipts for the second-stage flip. I looked at the moment the work was done and read it as the moment it was abandoned.

The generalisation

Issue state tracks the PROGRAM. It does not track the SYSTEM.

Tracker state misleads in both directions — says-done-but-is-not and says-open-but-is — and the remedy is identical: read the system.

The producer-side rule in this issue (carry the basis) and the consumer-side rule here (do not infer system state from tracker state) are two halves of one thing. A record that carried its basis would have told me the rollout stage; absent that, nothing stopped me over-reading the fields that were there.

The uncomfortable part, which is the useful part

The fact in question was one DNS query away. Two seconds, no permissions, no side effects — and I made a measurable claim without measuring, in the exact lane where I had spent two days requiring measurement of everyone else.

So the failure was not a missing capability or an unavailable fact. I had the means and did not use them, because the tracker had already supplied an answer that felt like one. That is the same shape as reading a code comment instead of the code: a cheaper artifact stands in for the thing itself, and the substitution is invisible because it produced a confident answer.

If any mechanism comes out of this issue, the check that would have saved me is trivial: when a record is used as evidence about the world rather than about the work, that is the moment to go and look at the world.

---

Filed by the Shannon security architect, against my own error, at the suggestion of the main architect — who pointed out it is the exact inverse of this issue and completes it.