Repository navigation
Description
Nothing detects that a model's SQL changed between runs. Job carries a ProjectVersion documented as a content hash of the project files (crates/model/src/lib.rs:93-96 at a0b5e6c, the project field at :105), but no code computes one: the only value ever built is the literal "abc" in a test module (crates/jobs/src/lib.rs:30, :52). Once the second run processes only new rows (ROADMAP.md:13), an edited model appends rows from its new SQL beside rows from its old SQL. The write path already refuses an edit that changes an output column's type (crates/storage/src/lib.rs:380-381), but not one that keeps it. For example, take a model that reads amount from an append-only table where it is already decimal(12, 2), the type the RFC 0001 example casts it to (docs/rfcs/0001-project-files.md:96). If after ten runs it selects cast(amount * 100 as decimal(12, 2)) as amount instead, every later run appends amounts 100 times larger beside the old ones, and nothing flags it.
This issue computes a version hash per model and commits it under an infostash. key beside the model's per-input checkpoint, in the same commit as the rows. ADR 0006 already requires this for checkpoints (docs/adr/0006-ordered-job-commits.md:49-51) and append_if_head already supports it (crates/storage/src/lib.rs:63, :188, :209, :213), so no new ADR is needed. In Phase 0 a mismatch fails the model's run with an error that names the model, which infoctl run (#14) handles like any other failed model. Falling back to a full refresh instead (docs/rfcs/0001-project-files.md:121) needs the replace commit that #72 decides and #88 builds, so it is not part of this issue.
Schema drift on the CSV side is out of scope too: a new source column makes every later ingest of its table fail (crates/storage/src/lib.rs:373-378). #79 turns that into an early error that names the file, #94 corrects the "Handles schema changes" line in docs/architecture.md:96, and additive Raw schema evolution stays a later opt-in.
Crates: crates/compiler (the hash), crates/jobs (the check before a model runs and the key in its commit)
Done when
- Each model's version hash is computed from its normalized SQL, its directives and the current schema id of each input table, as the rule from docs: revise RFC 0001 with the incremental rules (decision D2) #73 defines them, with a fixed digest rather than
std::hash::DefaultHasher(whose algorithm may change between Rust releases), so the same files and input schemas always give the same value - Every commit to a model's target writes the hash under an
infostash.-prefixed key through theappend_if_headproperties path, in the same commit as the rows and the per-input checkpoint from In-process runner: stage, run tasks, commit once #13, so a model's first run records it and no checkpoint is ever stored without the hash of the SQL that produced it - Before a model reads any input, the hash stored on its target is taken from the same
Warehouse::stateload that supplies the head the commit is checked against (crates/storage/src/lib.rs:153-161) and compared with the computed hash, and a model with no stored hash or a matching hash runs normally - On a mismatch the model is refused before anything is committed to its target or its quarantine table (Row checks and quarantine tables #12), with an error that names the model and says that its SQL, directives or input schemas changed since its last run, and the refusal is not retried
- Test: a model selects
amountfrom an append-only input table where it isdecimal(12, 2), such as one the test creates, and runs; new rows land in the input and a second run without edits is not refused; then the model selectscast(amount * 100 as decimal(12, 2)) as amountinstead, more rows land, and the third run is refused with the target's head, rows and checkpoint unchanged, so rows from the old and the new SQL never mix - Unit tests: the same files and input schema ids always give the same hash, pinned to a literal value, reformatting the query's whitespace leaves it unchanged, and changing an expression in the query, one
@checkdirective or one input's schema id changes it
Points: 3 (M2 version hash with refuse-on-mismatch 3)
Depends on: #73 (the version-hash rule), #7 (the header directives and input tables the hash covers), #13 (the model commit that carries the per-input checkpoint)
Blocks: nothing
Source: design review of 2026-11-01, finding C16 (points from section 8.5, M2).
Activity
codecapo commentedon Nov 10, 2026
#101 adds one case to the mismatch rule: a run of a model that declares
@full_refresh: truedoes not compare the stored hash; it replaces the table and commits the new hash in the same commit. An incremental run still refuses on a mismatch. The hash covers each check's name and expression in file order. See the Model versions paragraph in that PR.docs: revise RFC 0001 with the incremental rules (#101)