An operating contract for AI-assisted software and pipeline development in regulated environments.
Most AI governance documents describe. This one gates: risk-tier the task before code is written, hold every AI-generated component to a human-owned acceptance test suite, and never let the model grade its own homework.
Status: practitioner's working protocol, v1.2. Written from working practice in regulated-adjacent bioinformatics, mapped to the standards landscape as of August 2026. It is not legal advice and it is not a compliance certification. Critique is invited — open an issue. Corrections are treated the way the protocol treats caught errors: logged, versioned, and converted into a better rule.
AI now writes a large share of production code, and a human cannot read every line. Most organizations respond with either blind trust or a policy PDF nobody opens. Regulated industries solved this class of problem decades ago — you don't trust the process, you verify the output against independently defined criteria. This protocol ports that discipline to AI-assisted development.
- Data handling pre-flight (§0.5). Before any tiering or code: what class is this data, what is its approved boundary, and does anything here constitute egress to an unapproved service? The test is domain-agnostic — not "does this feel sensitive?" but "is there a law, contract, or classification governing where it may go?" — covering health, personal, financial, export-controlled, privileged, and trade-secret data alike. Binary, not graded; the AI-specific failure mode is an agent pasting restricted data into a third-party model API. Prevents the agent from becoming the vector; does not make an organization compliant. §0.5.1 covers local/in-boundary models and routing, including the rule that a probabilistic component may never be the sole control on a binary gate.
- Untrusted-output doctrine. The agent produces evidence; a named human — the Reviewer of Record — signs. The agent can never mark its own work validated (Rules 1–9 in PROTOCOL.md).
- Risk tiering before building. Four tiers, T0 (exploratory) to T3 (clinical/regulated), each with defined controls. The tier is declared at task start, never guessed downward (§2).
- Human-owned acceptance tests — the audit boundary. Human oversight scales at the test level, not the line level. Every acceptance test carries a provenance tag:
REG-(derived from regulatory requirements),DOM-(human domain knowledge),ADV-(agent-proposed adversarial, human-approved). The agent that wrote the code is never the sole author of the tests that gate it, and cannot modify the suite (§3.4). - Evals as validation. When a model runs inside the system, its eval set is a locked, human-owned acceptance suite: mandatory contamination checks, thresholds fixed before results, LLM-as-judge inadmissible until validated against human labels, every metric shipped with dataset version and reportable range (§3.6).
- Change control. Model, prompt, or parameter changes are changes: full re-validation with before/after comparison — the same structure the FDA's PCCP guidance formalizes (§3.5).
| Path | Contents |
|---|---|
| PROTOCOL.md | The protocol itself (generic — adapt per organization) |
| ADOPTING.md | How to put it into practice — solo repo in 30 minutes, team in a week, organization with a sponsor; plus the six adoption failure modes |
| examples/worked-run.md | Start here. An abbreviated transcript of the agent's required first response to a realistic request — what the protocol actually feels like in the first ten minutes |
| examples/bioinformatics-genomics.md | Full instantiation for regulated-adjacent genomics: truth sets, domain failure modes, data classes and boundaries (maintainer's own domain) |
| examples/llm-application.md | Instantiation for systems with a model at runtime: tiering, eval design, error taxonomies, provider drift, human-oversight anti-patterns |
| examples/financial-model-risk.md | Adaptation sketch for model risk management (SR 11-7 vocabulary) — clearly labeled as derived from published guidance, not operational experience; corrections invited |
| templates/ | Copy-paste artifacts: README validation section, validation report skeleton, caught-it log |
| CHANGELOG.md | Version history — maintained under the protocol's own change-control rules |
The generic protocol is deliberately incomplete: §3.3's performance characteristics and §3.4's REG- test sources must be instantiated against your domain's authorities (CLIA/CAP for clinical labs, GAMP 5 for GxP pharma, IEC 62304/ISO 14971 for devices, SR 11-7 for financial models, ISO 26262 / DO-178C for automotive/avionics). The genomics example shows what that instantiation looks like end to end. The adaptation itself is the work — a protocol you can't map to your own regulators is shelf-ware.
Practical adoption path: drop PROTOCOL.md (or your adapted copy) into a repo and reference it from your agent instructions file (e.g., CLAUDE.md, system prompt, or equivalent) so every AI session inherits it; put the acceptance suite in a path the agent cannot write to; wire the templates into your PR checklist. ADOPTING.md walks this through at three scales, and states the governing rule: a protocol not enforced by tooling is a document, and documents are ignored under deadline.
Quality/validation: CLIA 42 CFR §493.1253 · AMP/CAP NGS bioinformatics validation guideline (Roy et al., J Mol Diagn 2018) · FDA GMLP (2021) · FDA PCCP guidance (final 2024/2025) · IEC 62304 (Ed. 2 in ballot as of Aug 2026) · GAMP 5 2nd ed. / FDA CSA · NIST AI RMF 1.0 & AI 600-1 · EU AI Act Arts. 9–17 · Federal Reserve SR 11-7. Data handling (§0.5): HIPAA · GDPR · sector regimes (PCI DSS, GLBA, FERPA, BIPA, CJIS) · export control (ITAR/EAR, CUI) · contractual instruments (NDAs, DUAs, protective orders) · 21 CFR Part 11. See §4 of the protocol for what each contributes. Standards move; verify current status before citing any of them as binding.
Two contributions are especially welcome. Corrections — if a standard is cited wrongly, a status is stale, or a domain sketch is naive, open an issue; that is how a document about validation should be maintained. New instantiations — an examples/ file for your domain (devices, pharma GxP, aerospace, legal, public sector) makes the protocol more useful to everyone. Instantiations written from operational experience should say so; those derived from published standards alone should say that too. Mislabeled expertise is the one thing this repository cannot tolerate, given what it is about.
Text licensed CC BY 4.0 — adapt freely, with attribution. If this protocol shaped your organization's practice, a citation or a link back helps others find it.
Maintained by Chathura J. Gunasekara, Ph.D. — computational scientist, regulated-adjacent genomics.