docs(examples): add public-sector risk-register assurance worked example#45
docs(examples): add public-sector risk-register assurance worked example#45antnewman wants to merge 2 commits intoSingularityAI-Dev:mainfrom
Conversation
📝 WalkthroughWalkthroughIntroduces a complete eight-step assurance workflow specification ( ChangesRisk Register Assurance Workflow Example
Estimated code review effort🎯 3 (Moderate) | ⏱️ ~25 minutes Poem
🚥 Pre-merge checks | ✅ 5✅ Passed checks (5 passed)
✏️ Tip: You can configure your own custom pre-merge checks in the settings. ✨ Finishing Touches🧪 Generate unit tests (beta)
Tip 💬 Introducing Slack Agent: The best way for teams to turn conversations into code.Slack Agent is built on CodeRabbit's deep understanding of your code, so your team can collaborate across the entire SDLC without losing context.
Built for teams:
One agent for your entire SDLC. Right inside Slack. Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
|
Scope looks right. All four feature stress-tests landed where requested, and the human_review_gate-as-a-step pattern is exactly the shape I was hoping to see exercised. On the three "why draft" items:
On the four spec edges you flagged: all four are real, all four are worth tracking, none belong in this PR. If you're up for it, file them as separate issues post-merge and I'll triage. Specifically:
Mark ready-for-review when you're done polishing. Squash-merge target. Closes #18. |
Closes SingularityAI-Dev#18. Eight-step plan-execute workflow for processing a project delivery artefact through extraction, outlier scan, triage, human review, and persistence with full provenance. Designed to read as a regulated-domain reference example. The four feature stress-tests requested on SingularityAI-Dev#18 are deliberately bundled: - Rubric-based self-verification (5 weighted criteria, minimum_score 0.75) on quality_gates.self_verification — not previously exercised by an example or fixture - Write-segregated per-step allowed_tools across five tool families (read-only document, schema-validator, LLM+search, read-only store, human-review, write-only store) — every step declares both allowed_tools and denied_tools where load-bearing - human_review_gate as a first-class step in the DAG, not a flag on an LLM step or an action in fallback.escalation; allowed_tools is exactly [human_review_request], LLM and store-write explicitly denied; verification.on_fail = abort guards bypass attempts - Provenance metadata (step_chain, timestamps, source_artefact_id, reviewer_id, confidence) flowing through every step's contract and collected into the terminal assurance_record output Markdown narrative documents strategy choice, tool-segregation table, the human_review_gate pattern's three benefits, the rubric's audit rationale, and four spec edges encountered during drafting (no workflow-local $ref in output_schema; no first-class human-gate step type; no canonical tool-name registry; steps.<name>.completed not in spec text). Verified locally: validate() returns ok; node spec/fixtures/run-fixtures.mjs shows 29 passed, 0 failed.
antnewman
force-pushed
the
examples/pda-risk-register
branch
from
122a1f3 to
6572ef7
Compare
![coderabbitai[bot]](https://avatars.githubusercontent.com/in/347564?s=60&v=4){kind=small}
There was a problem hiding this comment.
Actionable comments posted: 1
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Inline comments:
In `@examples/pda-risk-register-assurance.logic.md`:
- Around line 90-127: Add a stable "risk_id" to each extracted risk: update
output_schema -> properties -> risks.items to include a required string property
"risk_id" and mark it required alongside title/description/etc., and ensure the
extractor populates risk_id deterministically (e.g., hash of title +
cited_byte_offset or other stable key) when emitting risks so downstream steps
(outlier_scan, triage, human_review_gate per-risk decisions,
emit_report.top_risks) can join reliably.
🪄 Autofix (Beta)
Fix all unresolved CodeRabbit comments on this PR:
- Push a commit to this branch (recommended)
- Create a new PR with the fixes
ℹ️ Review info
⚙️ Run configuration
Configuration used: defaults
Review profile: CHILL
Plan: Pro
Run ID: ebe45d5c-48de-453c-a5b1-add27d1f906a
📒 Files selected for processing (1)
examples/pda-risk-register-assurance.logic.md
![cubic-dev-ai[bot]](https://avatars.githubusercontent.com/in/1082092?s=60&v=4){kind=small}
There was a problem hiding this comment.
1 issue found across 1 file
Prompt for AI agents (unresolved issues)
Check if these issues are valid — if so, understand the root cause of each and fix them. If appropriate, use sub-agents to investigate and fix each issue separately.
<file name="examples/pda-risk-register-assurance.logic.md">
<violation number="1" location="examples/pda-risk-register-assurance.logic.md:120">
P1: `extract_risks` does not emit a stable `risk_id`, but downstream steps require it for per-risk correlation.</violation>
</file>
Reply with feedback, questions, or to request a fix. Tag @cubic-dev-ai to re-run a review.
Closes the contract break flagged by CodeRabbit and cubic-dev-ai on PR SingularityAI-Dev#45: extract_risks did not emit a stable risk_id, but four downstream steps (outlier_scan, triage, human_review_gate, emit_report) all require it as the join key for per-risk correlation. - Adds risk_id to required[] and properties on extract_risks.output_schema.risks.items - Constrains the format with a 12-hex-char pattern (deterministic SHA-256 prefix) - Updates instructions to specify deterministic generation (hash of title + cited_byte_offset) and to make the contract-key role explicit Validates clean; node spec/fixtures/run-fixtures.mjs unchanged at 29 passed, 0 failed.
|
Good catch from both @coderabbitai and @cubic-dev-ai — the missing
Worth noting: the structural validator passed this happily, which is itself a finding worth filing post-merge as a fifth spec edge — cross-step contract integrity (output of step A claims field X but is consumed by step B as field Y) is not statically checkable today. Will queue that as a separate issue alongside the four already flagged. |
@antnewman The fix looks solid — Your observation about the structural validator being blind to cross-step contract integrity is sharp — and I think it's the most practically impactful of the five spec edges. A field-level linter that checked "step A's It fits naturally as spec edge #5 in your existing list. Post-merge as a separate issue, it'd probably be the easiest one to start on since the shape of the check is mechanical (walk |
|
Tip For best results, initiate chat on the files or code changes.
The fifth spec edge is well-scoped and distinct from the other four:
Would you like me to open the GitHub issue for finding |
There was a problem hiding this comment.
1 issue found across 1 file (changes from recent commits).
Prompt for AI agents (unresolved issues)
Check if these issues are valid — if so, understand the root cause of each and fix them. If appropriate, use sub-agents to investigate and fix each issue separately.
<file name="examples/pda-risk-register-assurance.logic.md">
<violation number="1" location="examples/pda-risk-register-assurance.logic.md:93">
P2: The suggested `risk_id` derivation is not uniqueness-safe for a field used as the join key across downstream steps.</violation>
</file>
Reply with feedback, questions, or to request a fix. Tag @cubic-dev-ai to re-run a review.
| instructions: | | ||
| From the risk register and any narrative mentioning risk, threat, issue, | ||
| dependency, or assumption, extract candidate risk items. For each: | ||
| - risk_id (stable identifier; deterministic — e.g. SHA-256 of |
There was a problem hiding this comment.
P2: The suggested risk_id derivation is not uniqueness-safe for a field used as the join key across downstream steps.
Prompt for AI agents
Check if this issue is valid — if so, understand the root cause and fix it. At examples/pda-risk-register-assurance.logic.md, line 93:
<comment>The suggested `risk_id` derivation is not uniqueness-safe for a field used as the join key across downstream steps.</comment>
<file context>
@@ -90,6 +90,9 @@ steps:
instructions: |
From the risk register and any narrative mentioning risk, threat, issue,
dependency, or assumption, extract candidate risk items. For each:
+ - risk_id (stable identifier; deterministic — e.g. SHA-256 of
+ title + cited_byte_offset, truncated to 12 hex chars. Must be unique
+ within this run and reproducible across reruns of the same artefact.)
</file context>
2 participants
Closes #18 — the regulated-domain worked example you flagged for the v1.4 ship-honest window. Opening as a draft so you can review the shape before I do any final polish; happy to take feedback on any axis.
What this is
A LOGIC.md spec at `examples/pda-risk-register-assurance.logic.md` modelling an eight-step assurance workflow for processing a UK-government project delivery artefact (periodic report, business case, PRINCE2 product description). Output: a triaged risk register written to an assurance store with a full provenance trail.
The four feature stress-tests, bundled
Per your scoping comment on #18:
The 8 steps
`ingest_artefact → validate_artefact → extract_risks → outlier_scan → triage → human_review_gate → persist → emit_report`
Each step has `description`, `instructions`, scoped `allowed_tools`/`denied_tools`, and a typed `output_schema`. `extract_risks` and `triage` carry confidence thresholds; `human_review_gate` carries a 72h timeout with abort-on-bypass verification; `persist` carries a retry on storage-write failure.
Verification
Spec edges encountered
The markdown narrative ends with four observations from drafting that may be worth considering for spec evolution. None block this example's correctness against v1.0:
Happy to file each of these as a separate follow-up issue if any are worth addressing — explicitly not asking for spec changes inside this PR.
Why draft
Three reasons I've opened this as a draft rather than ready-for-review:
Mark ready-for-review on your nod, or push fixes if you'd rather direct.
Summary by cubic
Adds a public-sector risk-register assurance worked example: an eight-step LOGIC.md workflow that ingests a project artefact, extracts and triages risks, gates via human review, and persists with full provenance. Also adds a deterministic
risk_idto keep per-risk correlation consistent across steps.New Features
examples/pda-risk-register-assurance.logic.md(eight-stepplan-executeworkflow).human_review_gatewithallowed_tools: [human_review_request], 72h timeout, and abort-on-bypass verification.allowed_tools/denied_toolsacross tool families (read-only doc, schema-validator, LLM+search, read-only store, human-review, write-only store).assurance_record.minimum_score: 0.75) and quality gates (provenance intact, decision recorded, confidence floor).persistonly runs afterhuman_review_gatecompletes.Bug Fixes
risk_idtoextract_risksoutput (12-hex pattern), used as the join key acrossoutlier_scan,triage,human_review_gate, andemit_report.Written for commit 4ef40fb. Summary will update on new commits.
Summary by CodeRabbit