Require complete phase-aware onboarding before agents claim workflow compliance #25

Activity

1. yuema137 commented on Sep 10, 2026

   

   yuema137

   on Sep 10, 2026

   OwnerAuthor

   More actions

   Additional incident: reading occurred, but the generated contract contradicted the workflow

   The operator requests that this issue also cover a second failure from the same planning effort. It changes the diagnosis: required-reading coverage alone is insufficient.

   Observed behavior

   The session's earlier tool output already contained the implementation working rules, including autonomous semantic commits and continuation through PR publication/review. Nevertheless, the agent later wrote a project contract and fresh-session kickoff that stopped after local documentation/tests, requiring another approval for commit/push/PR creation. The operator had not requested that local-only endpoint and subsequently clarified that commit and PR creation should be authorized; only merge should wait.

   This restriction propagated into the written handoff for the next session. Thus even a successor that reads the handoff accurately can inherit an agent-invented limitation as if it were the operator's intended boundary.

   The defect is not accurately described as “the requirements were never shown to the agent.” They had been shown, but were incorrectly applied. Nor is the rule missing upstream:

   • Working rules §14: autonomous semantic commits, no approval before each commit.
   • §21: continue through push, PR creation/updates and final validation to operator-review readiness.
   • §22: explicit operator approval remains necessary for merge.
   • Adaptation guide: actual user authorization controls; planning-only or explicitly local-only requests do not inherit publication permission.

   Requested refinement: read coverage + contract reconciliation

   Keep the bootstrap/read-completeness request, but also reconcile the generated execution contract against the actual operator instructions and phase defaults before freeze/handoff. In particular:

   1. Represent implementation, semantic commit, branch push, PR creation/update, validation/CI and merge as distinct permission/endpoint decisions.
   2. Record the source of each override: explicit user instruction, applicable repository restriction, or unresolved authority. Do not present an agent-selected restriction as a user decision.
   3. Flag a generated “stop after local tests” endpoint when the agreed workflow authorizes continuation to PR readiness. Conversely, do not manufacture publication permission for a genuinely planning-only/local-only request.
   4. Carry the reconciled decisions into the kickoff and resumed/delegated sessions. Explicit new user authorization should supersede stale restrictions; it must not require the same approval repeatedly.
   5. Keep the distinction between a mechanical field-consistency check and semantic interpretation of user intent. Neither a README warning nor a “read complete” receipt proves correct application or creates authority.

   Additional acceptance cases

   • All required resources are supplied, but the agent produces an unjustified local-only contract: review flags the mismatch rather than treating successful reading as successful compliance.
   • The operator explicitly permits commit/push/PR but forbids merge: the handoff preserves that split; the next session continues through PR readiness without per-commit approval or accidental merge authority.
   • The operator actually requests planning-only or local-only work: publication remains unapproved.
   • A stale handoff and a newer explicit operator clarification disagree: the conflict is surfaced and reconciled, not silently propagated as a frozen user decision.

   This remains a workflow-reliability request, not a request for blanket publication access, automatic hook installation, or a change to the existing merge boundary. The local P0 contract is being corrected independently; no upstream implementation or enforcement is claimed.
2. yuema137

   closed this as completedin #29on Sep 10, 2026
3. yuema137

   reopened this on Sep 10, 2026
4. yuema137 commented on Sep 10, 2026

   

   yuema137

   on Sep 10, 2026

   OwnerAuthor

   More actions

   Partially addressed by #29, merged as 9f8ac13. Reopening: the PR body said Closes #25, which closed this automatically, but #29 only covers the bootstrap/read-completeness half of this thread. The second incident in the comment above is untouched, and closing on that basis would have been wrong.

   What #29 landed

   • SKILL.md rule 5 now covers a session that is new, replacing another, or a delegated agent — not only a post-compaction resume — stated as a session boundary and explicitly not a per-turn reread.
   • The phase table is stated to be the required resource list, so nobody reconstructs one from whichever prompt names a kickoff message happens to mention.
   • Before asserting that work does or does not comply with this workflow, the rule being cited must be read in this session.
   • The continuity preset's SessionStart handler names the installed SKILL.md by absolute path in both branches, using a path it already resolved. A missing file omits the sentence rather than printing an unopenable path.
   • verify_skill requires SKILL.md, references/agent-workflow.md and references/adaptation.md for every preset, so doctor no longer passes an install missing the file the hook now names.
   • All three copyable kickoffs read the entrypoint first, so a project with no hooks still hands the agent the route. Edited in docs/content.{en,zh-CN}.json and regenerated. Stale docs/plan/ example paths moved to .structured-coding/plans/.

   Probed on a real claude-code install: the message names the project's installed copy rather than the development checkout; --check-hooks with SKILL.md removed reports the mismatch and reports healthy once restored. Tests run against a relocated copy of the skill, so a hardcoded development path fails them. That probe also caught a defect the suite had missed — the recovery message claimed the entrypoint came first while the path sat fourth, behind the bound documents — fixed in the same PR with the ordering assertion strengthened. Ran 171 tests ... OK (skipped=1); all five --check validations PASS.

   Item 3 declined, deliberately

   Item 3 asks for a recorded onboarding READY state. Item 4 of this issue says a checkbox, file hash, or helper process reading a file cannot prove the model received or understood it — which rules out the mechanism item 3 needs. A helper printing READY would assert exactly what it cannot observe.

   What shipped instead is the observable subset: the entrypoint files exist, are readable, and match the installed version. doctor's stated limit is unchanged — it compares against this repository's source, so it answers for a clone-based install and is not validation of an arbitrary distributed copy.

   What remains open: the contract reconciliation

   The second incident is a different defect and #29 does nothing about it. The requirements had been read; they were misapplied, and the misapplication was then written into a contract and a handoff where a correct successor inherits it as though it were the operator's boundary. Read coverage cannot fix that.

   The concrete gap is in the contract template itself. Project-specific implementation contract in implementation-working-rules.md has fields for scope, invariants, sequence and validation budget, and no field for endpoint authority. §14, §21 and §22 state the defaults — autonomous semantic commits, continuation through push and PR creation to review readiness, merge only on explicit operator authorization — but nothing carries them into the filled contract, so an agent filling it in is free to invent "stop after local tests" and nothing in the document's shape contradicts it.

   Planned direction, not yet designed in detail:

   1. An endpoint block in the contract template listing implement/local validation, semantic commits, branch push, PR creation and update, CI repair to review readiness, and merge as separate decisions, each with the shipped default and the section it comes from.
   2. A recorded source per line: explicit user instruction, applicable repository restriction, or unresolved. An agent-selected restriction is not a user decision and must not be written as one.
   3. The freeze step reconciles the filled block against the operator's actual instructions, and flags a local-only endpoint when the agreed workflow authorizes continuation to PR readiness — and equally refuses to manufacture publication permission for a genuinely planning-only request.
   4. The handoff carries the reconciled block, and a newer explicit authorization supersedes a stale handoff with the conflict surfaced rather than silently propagated.
   5. This is a semantic reconciliation performed by the agent against a template that makes the decision explicit. It is not a mechanical check and will not be described as one. The merge boundary is unchanged.

   That touches a preserved prompt, so its specification baseline update will be deliberate, as SKILL.md's was in #29.
5. yuema137 commented on Sep 11, 2026

   

   yuema137

   on Sep 11, 2026

   OwnerAuthor

   More actions

   Both incidents in this thread are now addressed, by #29 (9f8ac13) and #30 (7848faa).

   Incident 1 — the entrypoint was not rechecked (#29)

   A continuation session read the three long prompts, did not recheck the entrypoint, and asserted that a combined step/PR document violated this workflow. Rule 2 and workflow §3 say the opposite.

   • SKILL.md rule 5 now covers a session that is new, replacing another, or a delegated agent — stated as a session boundary and explicitly not a per-turn reread, since a rule demanding the whole library every turn is one agents route around.
   • The phase table is stated to be the required resource list, so nobody reconstructs one from whichever prompt names a kickoff message happens to mention. The incident's session had the three prompts and only those.
   • Before asserting that work does or does not comply, the rule being cited must be read in this session; the routing rules are not in the long prompts.
   • The continuity preset's SessionStart handler names the installed SKILL.md by absolute path in both branches, using a path it already resolved. A missing file omits the sentence rather than printing an unopenable path.
   • verify_skill requires SKILL.md, references/agent-workflow.md and references/adaptation.md for every preset.
   • All three copyable kickoffs read the entrypoint first, so a project with no hooks still hands the agent the route. Edited in docs/content.{en,zh-CN}.json and regenerated, not in the generated READMEs.

   Incident 2 — the requirements were read and then misapplied (#30)

   Reading coverage could not fix this one, and #29 did not. The mechanism turned out to be three facts together:

   1. Workflow §5 already required the contract to identify whether commit, branch publication, PR updates and validation are authorized.
   2. The template had no field to record that in. A requirement whose answer has nowhere to live is one the filled artifact does not carry.
   3. implementation-working-rules.md said a project contract overrides the generic defaults "where it is more restrictive" — so an agent that chose caution wrote something the rules then treated as winning over §14 and §21, because nothing distinguished an operator's restriction from an invented one.

   What shipped:

   • An ENDPOINT AUTHORITY block listing implementation + local validation, semantic commits, branch push, PR creation/update, CI repair to review readiness, and merge as separate decisions, each with the shipped default and the section it comes from, plus a recorded source: explicit operator instruction, applicable repository restriction, or unresolved. Caution is named as not being a source. Merge is fixed on the line and no source value changes it.
   • A restriction overrides only when it records a source. Otherwise it overrides nothing and is resolved with the operator before freeze. Deliberately asymmetric: an unsourced line is resolved, never widened past the shipped defaults, and merge authority is never widened.
   • The handoff's minimum record list carries endpoint authority with each line's source, so an unresolved line reaches a successor visibly unresolved instead of arriving as a boundary.
   • PR CONTEXT AUTHORITY ORDER gained the item it was missing: it ranked repository truth, design, parents, handoff and memory but said nothing about the operator talking to the session now, so a stale restriction outranked a fresh instruction by omission. An explicit current-session instruction now outranks a recorded restriction, quoted and recorded as the new source, with neither side silently dropped.
   • Freezing requires reconciling the filled block against what the operator actually said — and the converse is stated in the same breath: a planning-only or explicitly local-only request does not acquire publication permission because the shipped default has one.
   • The human autonomy paragraph says the contract records which endpoints the operator authorized, that merge is never one of them, and that the agent may not quietly narrow the list.

   This covers the five requested refinements and the four additional acceptance cases, as document shape rather than as enforcement.

   One requested item declined, deliberately

   Item 3 of the original request asks for a recorded onboarding READY state. Item 4 of the same request says a checkbox, file hash, or helper process reading a file cannot prove the model received or understood the content — which rules out the mechanism item 3 needs. A helper printing READY would assert exactly what it cannot observe.

   What shipped instead is the observable subset: the entrypoint files exist, are readable, and match the installed version. doctor's stated limit is unchanged — it compares against this repository's source, so it answers for a clone-based install and is not validation of an arbitrary distributed copy.

   Closing on that basis, with the decline recorded here rather than left implicit.

   Evidence and honest limits

   Probed on a real claude-code install into a temporary project: the unbound SessionStart message names <project>/.claude/skills/structured-coding/SKILL.md — the installed copy, not the development checkout; the recovery message lists it first; --check-hooks with SKILL.md removed reports Installed skill lacks this preset version and reports healthy once restored. Tests run against a relocated copy of the skill, so a hardcoded development path fails them. That probe also caught a defect the suite had missed: the recovery message claimed the entrypoint came first while the path actually sat fourth, behind the bound documents — fixed in #29 with the ordering assertion strengthened.

   The specification pin refused the SKILL.md and implementation-working-rules.md edits until each baseline was updated deliberately, verified by reverting translations.json alone.

   Ran 171 tests ... OK (skipped=1); all five --check validations PASS.

   Not proven, and not claimed: that any agent applies the above. No test asserts it and none pretends to. What changed is the shape of the artifacts — a decision now has a field, a default, a source, and a rule that an unsourced narrowing does not win — plus the moment a session is told where the routing lives. Content tests are not reported as proof of agent adherence, and the one host-level mechanism here (SessionStart delivery) was measured for delivery during the continuity work; what is new is message content, exercised through the installed runtime directly rather than under an authenticated turn.
6. yuema137

   closed this as completedon Sep 11, 2026