Open
Labels
area/compilercrates/compilercrates/compilerarea/enginecrates/enginecrates/engineenhancementNew feature or requestNew feature or request
Milestone
Description
Rows failing a model's @check expressions go to <table>__quarantine instead of the target, and the run still succeeds.
Crates: crates/compiler, crates/engine
Done when
- Each
@checkis a SQL boolean; false or null sends the row to quarantine - Quarantine rows keep the original columns plus which check failed and the run time
- Target and quarantine rows land under the same job id: quarantine commits first and the target last, and a rerun of the job completes it without duplicates (iceberg-rust has no multi-table commit)
- Test: a crash between the two commits, then a rerun, leaves each row in exactly one table once
-
JobMetrics::rows_quarantinedis filled in - Tests for pass, fail and null cases
Metadata
Metadata
Assignees
Labels
area/compilercrates/compilercrates/compilerarea/enginecrates/enginecrates/engineenhancementNew feature or requestNew feature or request
Activity
codecapo commentedon Oct 30, 2026
Spike findings (commit semantics)
Target and quarantine can't share one atomic commit. iceberg-rust has no public multi-table commit (
TableCommit::buildispub(crate), and nothing calls the RESTV1_COMMIT_TRANSACTION).The spike tested three fallbacks. Recommended, and now in the criteria:
infostash.job-idin the snapshot summary.Alternatives (a
failed_checkcolumn in the target, or swapping both SQLite catalog pointers in one transaction) work but break ADR 0003 naming or tie us to SQLite.Points: 5 -> 8. Criteria updated below.
docs: supersede ADR 0004 with ordered per-table job commits (#41)
codecapo commentedon Nov 6, 2026
The design review of 2026-11-01 touches this issue. It keeps the quarantine split here and adds what the split needs to be correct and repeatable, while the RFC text, the job outcome type and a new Raw column move to new tickets. The issue now waits on two decisions, #73 and #74.
Suggested additions to the Done-when:
::casts and the casts that type coercion inserts (where amount > 0over text) are caught too. Raw columns read from a file are nullable text (crates/connectors/src/csv.rs:199at a0b5e6c) and DataFusion'sCASTis strict, so one bad string under the casts of the RFC example (docs/rfcs/0001-project-files.md:94-97) fails the whole run before any check runs.x is null or try_cast(x) is not null, the only way to tell a NULL in the source from a failed conversion; explicit@checks then run on the typed value, and a failed conversion is reported ascast:<col>, not as a failure of the column's own check. Test: a Clean model that castsamountand checksamount >= 0, over N Raw rows where oneamountis12,50, commits N-1 target rows and one quarantine row whose_checks_failedholdscast:amountand notamount >= 0. A bad value outside a cast (checked arithmetic, a timestamp parse with a format) still fails the run, with the non-retryableDataerror dist: write an RFC for the Job, JobOutcome and runner contracts (decision D3) #74 defines, naming the model.@checkof a model applies, not only the last:directives()collects into aBTreeMap(crates/compiler/src/lib.rs:9-16), so the RFC'sclean/orderskeeps only its second check (docs/rfcs/0001-project-files.md:90-91) and a negative amount reaches Clean unnoticed. With the list parsed in Compile a project into a dependency graph #7, a test with both checks shows both applied, and a row failing both is quarantined once with both checks in_checks_failed.CHECK(docs/rfcs/0001-project-files.md:119), or NULL accepted with required values written asis not null. A blank optional cell arrives as NULL because the reader keeps arrow-csv's default null handling (crates/connectors/src/csv.rs:106-108), so the null tests include@check: discount >= 0over a blankdiscount, quarantined under today's text and accepted under the alternative._source_file,_loaded_atand, once connectors: add _source_row to Raw tables #86 lands,_source_row) plus_checks_failed, alist<utf8>of the checks the row failed (implicit ones included, named as docs: revise RFC 0001 with the incremental rules (decision D2) #73 decides), and a per-row_job_idin place of the run time, all from one plan split on whether that list is empty. The job id has to be a column because readers that need a consistent view trust only quarantine rows whose job id is in the target's history (docs/adr/0006-ordered-job-commits.md:67-70), and the job id in a snapshot summary cannot be joined from SQL.to, the end of the range the job read, beside the job id the existing item already asks for, because under ADR 0006 a model's last processed snapshot per input is written in the same commit as the rows it describes (docs/adr/0006-ordered-job-commits.md:49-51); on the target this is the checkpoint of In-process runner: stage, run tasks, commit once #13, and the quarantine commit records the same value.@unique_keymerge of Merge rows on @unique_key #35 only sees accepted rows, and in a@unique_keymodel a NULL key fails an implicitkey_not_null:<col>check: a blankorder_idinclean/orders(docs/rfcs/0001-project-files.md:89,:94) satisfies its cast check and would otherwise collapse with every other NULL key.@check: ordered_at <= now()over a row whoseordered_atfalls between the first attempt's start and the rerun: the engine plans under the job's start time (Run a model with DataFusion and append to its table #10) and the runner keeps that time with the job id and supplies it again on a rerun (In-process runner: stage, run tasks, commit once #13), so the rerun computes the same split as the first attempt, asdocs/adr/0006-ordered-job-commits.md:71-72requires, and no row lands in both tables or in neither.Work that moved to new tickets:
try_castexample, the cast lint and implicit check, repeated and named checks, the NULL rule, the ordering rule, the pinnednow()and_job_id.JobOutcomerecording both commits of a job, the retry classification and theDataerror._source_rowto Raw tables.Suggested re-score: no change. Section 8.5 of the design review of 2026-11-01 scores the quarantine split with
try_cast, ordering and_job_idat 8, as #19 does, within milestone M2 at 50 points, and scores Raw_source_row(2, #86) and the directive parser (3, #7) as separate items; #19 decides.Source: design review of 2026-11-01, findings C14, C15 and C32 and design points S6-02, S6-04 and S6-08, with its sections 6.4 and 8.5 (D2, D3 and M2).
codecapo commentedon Nov 10, 2026
RFC 0002 (#102) fixes how this ticket reports its two commits:
JobOutcome::Committed { commits, metrics }carries oneCommitRecordper output table in commit order, so the quarantine record comes first and the target last.TaskStatsgainsrows_quarantined, which the runner sums intoJobMetrics::rows_quarantined. Values that fail a cast and still reach the engine map to the newError::Data, which is never retried. One note for the crash test: a job that loses its target commit to aConflict, or to a non-retryable error, after its quarantine commit is abandoned for a new id, which quarantines the same rows again; the "exactly once" property holds over the rows the ADR 0006 reader rule shows (job id in the target's history), and the RFC says so.codecapo commentedon Nov 10, 2026
ADR 0009 (PR #103) names this ticket, with #13 and #81, as where the rerun rule is implemented: a job that failed after any of its commits is rerun under the same job id, skipping the tables that already carry it, before any new job for the target is created. The quarantine commit must set
infostash.job-idas a table property as well as in the summary, since the ADR requires that of every commit and the runner precheck compares the two.docs: revise RFC 0001 with the incremental rules (#101)
docs: add ADR 0009 amending 0006 on position, reruns and expiry (#103)
docs: add RFC 0002 for the job and runner contracts (#102)