Repository navigation
docs: point every citation at the code it names, and gate citation drift - #55
Merged
Merged
Conversation
…carry The meta description of every synced page, the llms.txt summary and the JSON-LD site node said the site covers Claude Code trajectories only, while the converter also reads Codex CLI rollouts. The description now lives once in src/lib/repo.ts, in the words site/authored/index.md already uses, and astro.config.ts and Head.astro import it.
The docs cited code by `path:line`, and the site's citation pass checked only that each path exists. Of the 2,177 citations on main, 1,318 needed a change: 657 re-pointed by exact text match, 342 re-pointed after reading the code, 283 whose sentence was wrong about the code and is rewritten, and 36 removed with the claim they backed. 859 were already right. The rewritten pages add 324 citations, and the pages carry 2,466 after the second read. Claims corrected against the code include the dependency graph's package attribution (pydantic from atif-converter; pydantic-settings and pyarrow from atif-duck; sibling pins at ==0.3.0), the dead-code page's importlib sites, unused exports and dead definitions, the CLI reference's flags and exit codes, the vendored harbor version (0.24.0), the console script's manifest (the root pyproject.toml), source-removed sessions being kept rather than deleted, and the CONTRACT CLI heading. A second read of 431 citations (311 a heuristic flagged, 120 at random) found 8 still wrong; all are fixed.
scripts/docs_citations.py (`mise run docs:citations`, inside `docs:gate`, on every pre-push, and in docs.yml against the pull request's base) runs two checks over docs/ and site/authored/. The static check fails on a citation naming a file git does not track, a line past the end of its file, or a shorthand with no antecedent. On main it finds 15 past-the-end citations; on this tree it finds none. The drift check maps each citation on a doc line the change leaves alone through its cited file's diff from the base. Lines that moved fail with the new span, and `mise run docs:citations:fix` rewrites them. Lines that were removed, or rewritten and shifted, fail for a reader to re-point. An edit in place prints a note. Replayed over the 29 commits before this change, it fails 19 of them on 991 citations and names the new lines for 934. On the 5c906c5..4fccb1e range, all 21 citations it fails were among those this audit had to fix. A check that the cited line holds an identifier from its sentence was measured and not built: on the corrected docs it can judge 1,203 of 2,465 citations, and 304 of the 311 it flags are correct (a 25% false-positive rate), because a test, behavior or config citation rarely repeats the sentence's identifiers.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Every
path:linecitation indocs/now points at the code its sentence names, the claims on every published page match the code at 4fccb1e (v0.3.0), and a new gate fails the next time code moves out from under a citation.Why
The site's citation pass (
site/src/lib/citation-links.ts) checks only that a cited path exists. A moved line, a range past the end of its file, or a sentence that went stale passed every gate. On main,docs/reference/cli.mdcitedanalyzeatapp.py:1240(it's at 1634) andconvertatapp.py:282(it's at 675).Citations
All 2,177 citations on main were read against their target lines.
The pages carry 2,466 citations after the pass. A second read of 431 of them (the 311 an identifier heuristic flagged, plus 120 drawn at random) found 8 still wrong, and this PR fixes all 8. 119 of the 120 random ones were right on the second read.
insights/business-logic.mdinsights/contract-map.mdinsights/impact-analysis.mdarchitecture/module-map.mdbehavior/processes.mdinsights/debugging-guide.mdreference/cli.mdanalysis/dead-code.mddiagrams/behavioral/sequences.mdanalysis/risk-hotspots.mddiagrams/architecture/components.mdbehavior/state-machines.mdinsights/tech-debt.mddiagrams/structural/dependency-graph.mdarchitecture/data-flow.mdreference/public-api.mdarchitecture/system-overview.mdClaims corrected, by page
reference/cli.md: every command and flag cites itsdefand parameter lines. Adds the flags the sections omitted (materialize --lake/--no-lake;embed --install-extension,--prune-orphans,--lake/--no-lake;corpus slim --format), merges a duplicated exit-code paragraph, and completes the exit codes (77, thelake_mismatch,sandbox_refused,extension_missingandlake_unavailablekinds;127is retired). The console script is declared in the rootpyproject.toml;embedandsearchcall Bedrock only with the defaultcohereprovider;searchreturns acorpuscolumn; cron has acompactlane.reference/public-api.md: signatures ofbuild_plan,convert_and_audit,materialize,register,run_analyzeandscan_source_root; theConversionOutput,EmbedSettings,mainandregisterdescriptions.architecture/module-map.md,system-overview.md: the vendored harbor models are 0.24.0, and nosrc/module imports harbor; sections aren't ordered by size, and atif-converter is the largest member; four atif-cli adapter modules import siblings at module scope; atif-cli pins five siblings; thelakeandcorpussub-apps; the local EmbeddingGemma 2 provider; onlyanalyzeis a dry run by default; five members carry a layers contract; the published wheel builds with hatchling;cyclopts>=5.1.1.architecture/data-flow.md: the artifact write order, and no harbor call at materialize time.behavior/state-machines.md,processes.md: a session whose source vanished is kept and markedsource_present: false, never deleted; materialize's write step has noArtifactProduceror--no-columnar, and the DuckLake sink step is added; the loss report is built after enrichment;statuscallspreview_pass.diagrams/structural/dependency-graph.md: pydantic comes from atif-converter, pydantic-settings and pyarrow from atif-duck, the sibling pins are==0.3.0, andzstandardandimagesizeare listed.diagrams/architecture/components.md,behavioral/sequences.md:run_analyzeis atanalyze.py:37;spec_forandclassify_structuredare methods; the materialize participant is our converter, not harbor;queryregisters authorship and attaches the lake by default; the lance extension is loaded there, installed only byembed.insights/business-logic.md: drops the rule for a retired harbor drift error; side files are read in place; the embed backfill isn't a dry run by default;EXIT_CODESis the only exit-code table; only the retry queue checksPIPELINE_NAMES; a dozen smaller rules match the code.insights/contract-map.md: stored artifacts are.zst, and the lake loader writes five parquet files; the embedding ports gain the Gemma provider,LakeTextRows,discoveryandcommit;ConverterPort.converttakesarchive_dirand returns six fields; five unlisted Protocols are named.insights/impact-analysis.md:harbor_versionis the vendoredUPSTREAM_VERSION; the refresh script, notatif_cli.cron, suppresses retries on 78;convertwrites with--trajectory-out;gh release create --verify-tagruns inrelease.yml.insights/debugging-guide.md: an empty scan exits 78 withsuspicious_scan; a corrupt watermark logs a warning and re-converts;ATIF_SQL_LOG_LEVELexists; the exit-code table matchesEXIT_CODES.analysis/dead-code.md: re-measured on this tree. Seven public definitions are never used.importlibsits atapp.py:129andgemma_local.py:115, andharbor_adapter.pyhas neitherimportlibnorgetattr.parse_trajectory_step,read_trajectory_stepsandread_edges_uuidsare listed as exports nothing imports.DEFAULT_STEPS_CACHE_SIZEis left off, becausetest_resource_guards.py:177reads it.analysis/risk-hotspots.md,insights/tech-debt.md: churn, author and scanner counts re-measured (69 banditB608sites, 70 ruffS608); retired debt rows removed;FidelityGaphas six members;max-argsis set bymaterialize.CONTRACT.md,CONTRACT-V2.md: two reserved corpus slugs; themessagescolumns;DETACHis refused;--formattakestable; the CLI heading names atif-cli as the composing package (atif-analytics also imports atif-models); three cron lanes.README.md,AGENTS.md,CONTRIBUTING.md,RELEASING.md, package READMEs,site/authored/:checkhas 11 gates; a Codex query needs--agent codex; release examples use 0.3.0, and the failure table quotes the messages the preflight prints; the version-files dry run keeps the trackedCHANGELOG.md; package READMEs list their real commands, views and parameters.llms.txtsummary and the JSON-LD site node said "Claude Code agent trajectories". They now name Codex CLI too, from one constant insite/src/lib/repo.ts.The guard:
mise run docs:citationsscripts/docs_citations.pyruns insidedocs:gate, on every pre-push, and indocs.ymlagainst the pull request's base.mise run docs:citations:fixrewrites them. Lines that were removed, or rewritten and shifted, fail for a reader. An edit in place prints a note and passes.Measured before building it:
The other proposed guard, a check that each cited line holds an identifier from its sentence, was measured and not built. On the corrected docs it could judge 1,203 of 2,465 citations, and 304 of the 311 it flagged were correct (a 25% false-positive rate). Test, behavior and config citations rarely repeat the sentence's identifiers.
Verification
mise run check: 2,679 passed, 1 skipped, 1 xfailed; every gate green.mise run docs:gate:docs:citations0 problems; 20 pages and 1,980 internal links with 0 broken;astro check0 errors; the dist probes pass, including every Mermaid diagram rendering to SVG.mise run docs:prose: 0 errors. Warnings fell from 269 to 257, and.vale-baseline.jsonis lowered to match.tests/test_docs_citations.pyplants each defect. Disabling the range check or the move detection turns its tests red.Not verified here
risk-hotspots.md's coverage column: it wasn't re-derived, since that needs a coverage run.Separately, some code comments disagree with the code, and this PR doesn't touch code:
pyproject.tomlsays 15 SQL sites (there are 70), namessearchas themax-argsowner (it'smaterialize), and says the root has no[project]table.mise.tomlsays the VEX ledger ships empty; it has 2 statements.claude_code_converter.pyheader says harbor 0.23.0.