[ParallelEntrepreneur]

⚠️ The final tree has not been verified. Re-run CI before reviewing. The first implementation passed every CI check (listed below). A follow-up review pass then started shortening crates/tokker-api/src/freshness.rs and fixing two small bugs in tools/freshness.mjs / tools/build.mjs. It was killed partway through by an API rate limit, and nothing could re-run the checks afterwards. Those three files may hold partial edits. If CI fails, the cause is almost certainly that interrupted edit, not the design.

What changed

• data/rules/freshness.json: the SLA classes volatile_api (3 days), standard_api (10 days) and subscription (14 days), plus the rules that pick a row's class. The first matching rule wins, and the row type is checked before volatility, so a subscription with volatility: high still gets 14 days. A key left out of a rule matches anything.
• Semantics, the same in JS and Rust:
  • age = whole UTC days from the date part of last_verified_at to "today".
  • stale = age > sla_days. A row exactly at its SLA is not stale, and a row verified today or with a future date never is.
  • stale_since = verified date + SLA + 1 day (the first stale day), otherwise null.
  • A malformed date is an error. It never counts as fresh.
• tests/vectors/staleness.json: 17 shared vectors:
  • each class at age 0, SLA−1, SLA and SLA+1;
  • medium volatility at 11 days, which must be stale;
  • low volatility;
  • a high-volatility subscription;
  • a date-time last_verified_at;
  • year-boundary and leap-day crossings;
  • a future verification date.
• tools/freshness.mjs:
  • classify, staleness and annotate.
  • annotate copies the document and adds stale, stale_since and, where it applies, stale_fields to every api_offers[] / subscriptions[] row.
  • stale_fields maps a field to its stale_since. It lists a field only when its provenance.fields.<f> override has a fetched_at older than the row and is itself past the SLA.
  • It also adds counts.stale and a top-level freshness: { as_of, oldest: [5 rows] }, ordered by oldest last_verified_at with ties broken by id.
• tools/build.mjs (npm run build): reads data/pricing.json, annotates it as of --as-of YYYY-MM-DD (default: today in UTC) and writes dist/pricing.json (--out overrides the path; tests use it). Assembling the per-provider shards is still left to Split pricing.json into per-provider and per-vendor files, with a deterministic build back to pricing.json and the CSVs #10. dist/ is already gitignored.
• crates/tokker-api/src/freshness.rs:
  • The Worker-side implementation. It embeds the same rules file with include_str! and parses it once, so no SLA is hard-coded in Rust.
  • Date math uses days-from-civil, with no new dependencies; only serde/serde_json were added, both from the workspace.
  • The Worker calls it at read time with the request date, because data goes stale between merges. No read routes exist yet, so nothing calls it so far.
• Schema (schema/pricing.v1.json): additive only. It adds optional stale, stale_since and stale_fields on both row types and an optional top-level freshness block, so dist/pricing.json validates. counts already accepted extra integer keys, so counts.stale needed no change.
• Version and docs: schema_version in data/pricing.json goes to 1.1.0, following the additive-minor policy in docs/SCHEMA.md. docs/SCHEMA.md and tools/README.md get short notes.

How it was verified (before the interrupted edit)

• Rust:
  • cargo fmt --all -- --check, cargo clippy --workspace --all-targets -- -D warnings, cargo test --workspace and RUSTDOCFLAGS="-D warnings" cargo doc --no-deps --workspace all passed.
  • cargo build -p tokker-worker --target wasm32-unknown-unknown passed.
  • cargo tree -d --depth 0 | grep cratefield printed nothing.
• Data:
  • npm ci and npm run validate passed.
  • npm test passed 22/22: every shared vector, the four stale_fields cases, counts/oldest, no mutation of the input, and the annotated real dataset passing validateDocuments.
  • npm run build built dist/pricing.json with 253 stale rows out of 583 as of 2026-10-10. That output passed node tools/validate.mjs dist/pricing.json.
• The Rust tests and the vitest suite both run every vector in tests/vectors/staleness.json, which is how the Worker and the build are kept in agreement.

Look at closely

• The interrupted edits. These are what the review pass was working on when it stopped:

  • tools/freshness.mjs: replacing Date.UTC with setUTCFullYear, so that years 0000–0099 parse literally the way Rust does. Date.UTC maps them to 1900–1999, so the two implementations would disagree on those years.
  • tools/build.mjs: printing a clean error and exiting 1 on an impossible --as-of or a malformed last_verified_at, instead of a stack trace.
  • crates/tokker-api/src/freshness.rs: a shorter rewrite of the 318-line first version.

  Check each file is complete. If one isn't, take it from the first implementation and reapply the two fixes.

• Size. About 790 lines in total, over the 400-line guideline: 8 modified files (+65/−5) and 6 new files of about 730 lines. The issue asks for two implementations, shared vectors, two test suites and the schema/docs updates.

• Stale rows today. Every row has last_verified_at: 2026-10-05, so the 253 high-volatility API rows are already past their 3-day SLA. The flags are correct; the verify loops just haven't run yet.

Closes #18

Note: this branch was 6 commit(s) behind main when this pull request was opened.

---

🤖 Generated by Colonizer in a microVM

Co-Authored-By: Colonizer Settlers 331648616+colonizer-settlers@users.noreply.github.com