Skip to content

chore(deps): ignore RUSTSEC-2026-0459 for the bundled SQLite - #99

Merged
codecapo merged 1 commit into
mainfrom
chore/deny-sqlite-advisory
Nov 10, 2026
Merged

codecapo merged 1 commit into
mainfrom
chore/deny-sqlite-advisory

Conversation

@codecapo

@codecapo codecapo commented Nov 9, 2026

Copy link
Copy Markdown
Owner

What and why

cargo deny check fails on every branch since 2026-11-09, when RUSTSEC-2026-0459 (CVE-2026-119375) entered the RustSec database. SQLite before 3.54 can return uninitialised page-cache memory when a database is opened next to a crafted -wal file whose page size differs. Our copy is SQLite 3.46.0, bundled by libsqlite3-sys 0.30.1, which sqlx-sqlite 0.8.6 pins and iceberg-catalog-sql requires (cargo tree -i libsqlite3-sys). The fix is libsqlite3-sys 0.39, and no sqlx release accepts it: sqlx-sqlite 0.9.0 caps it below 0.38, and iceberg-rust main still requires sqlx 0.8.1. No dependency update clears the advisory today.

This adds an ignore with a reason to deny.toml, beside the three zstd ignores, with a comment saying when to remove it. Removal is tracked in #98.

Exposure

Only catalog.db in the warehouse directory is opened through this SQLite, and Infostash is the only writer there. Triggering the bug needs a crafted file placed next to the catalog by someone who already has write access to that directory.

About #71

crates/storage declares sqlx only to turn on the SQLite driver of the copy iceberg-catalog-sql uses. #71 moves that declaration to sqlx 0.9, which builds a second sqlx and leaves the catalog's copy without a driver; its tests fail with a pool panic for that reason, and it would not clear this advisory anyway. #71 should wait for iceberg-rust to move to sqlx 0.9, or be closed with a dependabot ignore for sqlx majors.

How it was tested

cargo deny check passes with the ignore and fails without it (one advisories error, RUSTSEC-2026-0459). typos passes. No code changes.

The advisory entered the RustSec database on 2026-11-09, so every
`cargo deny check` now fails. SQLite before 3.54 can return
uninitialised page-cache memory when a database is opened next to a
crafted -wal file whose page size differs. The bundled copy (SQLite
3.46.0) comes from libsqlite3-sys 0.30.1, which sqlx-sqlite 0.8.6
pins, which iceberg-catalog-sql requires. The fix is libsqlite3-sys
0.39, and no sqlx release accepts it yet: sqlx-sqlite 0.9.0 caps it
below 0.38. So no dependency update can clear the advisory today.

Only catalog.db in the warehouse directory is opened through this
SQLite, and Infostash is the only writer there. The ignore carries a
reason and a comment saying when to remove it, like the zstd ones.

Signed-off-by: Mayur Kale <hello@mayurkale.com>
@coderabbitai

coderabbitai Bot commented Nov 9, 2026

Copy link
Copy Markdown

Warning

Review limit reached

You've used all free OSS reviews for now. Wait for the free limit to reset to keep reviewing this public repository.

Next included review available in 58 minutes.

Check out review usage here.

View limit details

Limit details: You’ve used the included review currently available.

Learn how review limits work.

Review configuration:

Run configuration

Commits

Reviewing files that changed from the base of the PR and between 6e13aa1 and 8104c0b.

Files selected for processing: 1

  • deny.toml
  • Autofix · Keep fixing CodeRabbit findings and required CI, and resolving merge conflicts

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

Share

@codecov

codecov Bot commented Nov 9, 2026

Copy link
Copy Markdown

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 99.14%. Comparing base (6e13aa1) to head (8104c0b).

Additional details and impacted files
@@           Coverage Diff           @@
##             main      #99   +/-   ##
=======================================
  Coverage   99.14%   99.14%           
=======================================
  Files           8        8           
  Lines        1048     1048           
=======================================
  Hits         1039     1039           
  Misses          9        9           

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

@codecapo
codecapo merged commit 437a58c into main Nov 10, 2026
19 checks passed
@codecapo
codecapo deleted the chore/deny-sqlite-advisory branch November 10, 2026 15:41
codecapo added a commit that referenced this pull request Nov 10, 2026
## What

Two architecture decision records from the #72 spike on the replace
commit route (decision D1), plus their rows in the ADR index.

- **ADR 0007** chooses the hand-built route for 0.1.0: the `overwrite`
snapshot is built from public iceberg-rust APIs and committed with our
own one-row compare-and-swap on the SQL catalog's `iceberg_tables` row,
with `TableRequirement::check` on the UUID and head first and zero rows
updated mapped to `Error::Conflict` with no retry. It records the `cargo
tree -d` result (one copy each of `iceberg` and `sqlx`), the state of
upstream at the pin, how far the fork route got and why it is not taken,
and the costs of the chosen route.
- **ADR 0008** records the rule for incremental readers of a replaced
table: when any input holds a non-append snapshot or an expired `from`
snapshot, the scheduler pins every input of the model with `from =
None`, reports the fallback and replaces the target as a new job; the
refusal in `appends_between` stays as a guard that ends the job without
a retry; the compile-time rule from #73 and #84 applies until the
plan-time check from #89 lands.

Closes #72

## How

The spike built a throwaway prototype in `crates/storage` on a local
branch and committed the route with twelve tests: two replaces prepared
on one head give one commit and one conflict with the loser's files
referenced by no snapshot, and exactly one wins when they commit
concurrently; a replace racing an append that lands first is a conflict;
the table keeps its UUID, snapshot history and metadata log; a reader
holding the old metadata still reads the old files; iceberg-rust's own
`fast_append` builds on the hand-made snapshot; the incremental refusal
fires on the real overwrite; a full refresh with no rows still commits;
and a replace commits on a catalog whose `iceberg_tables` has no
`iceberg_type` column, the older layout the catalog reads and appends
to. Each head guard was then tried on its own against those tests: the
swap alone still gives one winner, the requirement check alone leaves
the window between check and swap open, so ADR 0007 asks #88 for a test
per guard. The fork route was probed alongside with a one-line `[patch]`
that makes `TableCommit::build` public; the same snapshot committed
through `Catalog::update_table`, the catalog's own conflict refused a
stale commit when the requirements were set, and without them a stale
commit given a fresh sequence number landed over the moved head. Both
ADRs record what those runs showed.

## Things to know

- The commit statement is the SQL catalog's own less its record type
filter: the table's key and the metadata location already pin the row,
so the statement fits both catalog layouts.
- The prototype is not in this pull request. #88 builds
`Warehouse::replace_if_head` to the design ADR 0007 records.
- The route keeps every untouched manifest, as #88's acceptance already
asks, and never copies an earlier snapshot's `DELETED` entries into a
manifest it writes.
- A losing replace leaves manifests, a manifest list and a metadata file
as well as data files; ADR 0007 says removal of unreferenced files must
cover them, which widens #40's current scope.
- `cargo deny check` is red on every branch until #99 lands
(RUSTSEC-2026-0459, tracked in #98). Nothing here touches dependencies.
- Only `docs/adr/` changes. No code, CI or configuration is touched.

## Tests

- `typos docs/adr/` passes.
- Every relative link in the two ADRs and the index resolves.
- Both ADRs follow the format in `docs/adr/README.md`.

---------

Signed-off-by: Mayur Kale <hello@mayurkale.com>
codecapo added a commit that referenced this pull request Nov 10, 2026
Closes #15
Closes #93

## What

`infoctl query "<sql>"` runs one read-only SQL statement against the
warehouse and prints the result.

```
infoctl query [--warehouse <dir>] [--format table|csv|json] "<sql>"
```

- Tables are named `<layer>.<table>`, either by namespace
(`gold.daily_revenue`) or by alias (`ready.daily_revenue`), as in ADR
0003.
- Writes are rejected through DataFusion's `SQLOptions`: DML, DDL, `COPY
TO`, `CREATE EXTERNAL TABLE` and `SET` all fail with "not supported",
also under `EXPLAIN ANALYZE`, and the tests check that nothing was
written.
- `--format csv` prints a header. `--format json` prints one JSON object
per row (line-delimited), with explicit nulls. An empty result still
shows its columns in table and CSV output.
- An unknown table gives `not found: table ready.missing; tables:
gold.daily_revenue`, or says the warehouse has no tables.
- Exit codes: 0 on success, 1 for a query error, 2 for a usage error.

## How

- **engine:** `infostash_engine::query(&Warehouse, sql) -> QueryResult`
registers each table as an `IcebergStaticTableProvider`. Every table is
read at the snapshot it had when the query started. Each layer's schema
is registered under both its namespace and its alias.
- **storage:** new public `Warehouse::tables()` (every table in the
three layer namespaces, sorted) and `Warehouse::load()` (previously
private).
- **model:** `Layer::ALL` and `Layer::alias()` sit next to
`Layer::namespace`, so the ADR 0003 mapping stays in one place.
- **cli:** the logic lives in `src/lib.rs` (`run(args, out, err) ->
i32`), so it can be tested in-process. `main.rs` only passes the
arguments through.
- **deps:** `arrow-json = "59.2"` added to the workspace. It was already
in the lock file through DataFusion.

## Things to know

- **`--warehouse` is a stopgap.** It defaults to `.infostash/warehouse`,
the value in the RFC 0001 draft. Once project files land (#5, #7), the
warehouse should come from the project file. The flag is not in #15.
- **A warehouse must already exist.** A missing directory, or a
directory without `catalog.db`, is an error and nothing is created.
`Warehouse::open_local` opens SQLite with `mode=rwc`, so the command
checks for the catalog file before opening (#93).
- **Scans run on one core.** That is the upstream provider's single
partition, which #38 fixes. This PR is only about correctness.
- **Follow-ups.** SQL that starts with a `--` comment is refused as an
unknown option until #96 adds an end-of-options marker. `show tables`
needs `information_schema`, which #97 turns on. A `QueryContext`, a
single-pass load and a memory limit are #95.

## Tests

- `crates/engine/tests/query.rs` (12 tests):
  - namespace and alias resolve to the same table
  - aggregates
  - an empty result keeps its schema
  - a table with no snapshot reads as empty
- writes are rejected (`COPY TO`, `INSERT`, `CREATE TABLE AS`, `DROP`,
`SET`)
- `CREATE EXTERNAL TABLE`, `EXPLAIN ANALYZE INSERT` and a file path in
`FROM` are rejected
  - the unknown-table message and the empty-warehouse message
  - parse errors and execution errors
- `crates/cli/tests/query.rs` (8 tests) runs the real binary against a
small committed table: table, CSV and JSON output, empty results, a
rejected write, an unknown table, a missing warehouse, and a directory
without a catalog (which stays empty).
- Mutation checks:
  - dropping the read-only options fails the write test
  - dropping alias registration fails 8 tests
  - dropping the table listing fails the unknown-table test
- reverting the catalog check to a directory check fails the
empty-directory test
  - dropping `with_allow_ddl(false)` fails the external-table test
- dropping `with_allow_dml(false)` fails the `EXPLAIN ANALYZE INSERT`
test
  - enabling URL tables fails the file-path test
- fmt, clippy, tests, typos, machete and `cargo doc` with `-D warnings`
all pass. `cargo deny check` fails on RUSTSEC-2026-0459, which entered
the RustSec database on 2026-11-09 and is not related to this change;
#99 handles it on main. Workspace line coverage was 96.2% at the last CI
run.

---------

Signed-off-by: Mayur Kale <hello@mayurkale.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant