Skip to content

feat(data): FX policy — data/fx.json, fx refresh, build recomputes every usd block (#12) - #68

Merged
ParallelEntrepreneur merged 1 commit into
mainfrom
colonizer/issue-12-381d7c85
Oct 10, 2026
Merged

ParallelEntrepreneur merged 1 commit into
mainfrom
colonizer/issue-12-381d7c85

Conversation

@ParallelEntrepreneur

Copy link
Copy Markdown
Contributor

Resolves #12 (part of #1).

What changed

  • data/fx.json (new): {base: "USD", date, source, rates}. This is the seed block from pricing.json, copied unchanged (2026-10-02), so no price moves in this PR beyond re-rounding.
  • tools/fx.mjs (new, npm run fx [--out path]):
    • Pure helpers: roundSig (6 significant digits), toUsd (native ÷ rate; null and "unknown" pass through; an unsupported currency throws), isSupportedCurrency, validateFx, and fxFromFrankfurter.
    • The CLI fetches https://api.frankfurter.app/latest?from=USD with a descriptive User-Agent. That URL now redirects (301) to api.frankfurter.dev/v1, and source records the URL it actually landed on.
    • The CLI refuses to write, and exits 1, on: a network error, a non-200 response, a non-JSON body, a rate that isn't a positive number, or a missing rate for any currency a row uses.
  • tools/build.mjs (new, npm run build, which used to be a "not implemented" stub):
    • Recomputes every usd{} block on API offers and subscriptions from native values and data/fx.json.
    • blended_3to1 comes from the unrounded natives and is rounded once.
    • fx_rate_date is set only on non-USD rows.
    • Copies the fx block into pricing.json and rebuilds derived.cheapest_provider_per_model.
    • Refreshes the usd_* columns of the two CSV exports.
    • Validates fx.json before building.
  • tools/validate.mjs: three new checks:
    • A row whose currency has no rate fails with currency "XYZ" has no rate in fx (unsupported; see docs/fx.md).
    • Every usd field must equal what the build computes, otherwise is X, build computes Y …; run npm run build. This is what makes a hand-edited usd value fail CI.
    • The fx block in pricing.json must deep-equal data/fx.json.
  • docs/fx.md (new): the policy. It covers the source, one dated rate set per dataset, usd = native / rate, rounding to 6 significant digits applied once, unsupported currencies being refused, rows never mixing currencies, and usd blocks being build output only.
  • One-line pointers in docs/SCHEMA.md and loops/README.md. The api-daily loop runs npm run fx && npm run build; the loop prompt files themselves don't exist yet.
  • package.json: only the build line changed, plus a new fx line.

Data diff — please look at this

The seed usd values were rounded to 6 decimal places. The issue asks for 6 significant digits, so this PR re-rounds them once:

  • pricing.json: 117 changed lines, every one a usd number or a derived…blended_3to1_usd mirror. Examples: 6.737957 → 6.73796, 0.005966 → 0.00596605. If you strip the usd blocks and derived blends, the old and new documents are deep-equal. No cheapest_offer winner changed and no derived entry was added or removed.
  • CSVs: only usd_* cells changed.
  • One USD row changed: openrouter/mistral-small-3.2-24b-instruct-2506. Its blend 0.1328125 is an exact tie at the last digit, which the old values rounded down and roundSig now rounds up.

The build writes pricing.json byte-stably: it keeps 20.0-style number lexemes and the missing trailing newline. That avoids roughly 445 lines of formatting churn and makes re-runs idempotent.

Acceptance

  • Changing one rate in fx.json changes exactly the usd blocks of rows in that currency. This is a unit test on fixtures plus a test over the real data (bump CHF → exactly the 4 CHF rows change). The reviewer also checked it by hand: bumping CNY 6.7046 → 7.2 changed exactly the 56 CNY rows and only their CSV usd_* cells. The derived ranking was recomputed, and 6 winners correctly flipped from -intl to -cn rows.
  • A row in an unsupported currency fails validation, and the build refuses it too.
  • Unit tests for conversion and rounding: tools/fx.test.mjs and tools/build.test.mjs.

Verification

  • With Node 22.20: npm ci && npm run validate && npm test passes (ok data/pricing.json, 44/44 tests).
  • node tools/build.mjs on the committed data is idempotent: a second run leaves pricing.json and both CSVs byte-identical.
  • An independent review found one medium and two low issues, all fixed with tests:
    • an empty CSV cell was being treated as 0;
    • a fetch that throws (DNS, refused connection) escaped as an unhandled rejection;
    • fx.json rates were not checked before the build used them.

Size and overlap

Size. The PR is over the soft ceiling: about 1,060 lines in 6 new files plus +352/−182 across 8 modified files, 14 files in total.

  • About 460 of those lines are tests.
  • tools/build.mjs is 344 lines, spent roughly on:
    • CSV refresh: ~95
    • byte-stable serializer: ~81
    • derived ranking: ~57
    • CLI: ~56
    • usd math: ~40
  • The serializer and the CSV refresh exist so the data diff stays limited to numbers that actually changed. Without them, the build would reformat the whole dataset.

Overlap with sibling #14. package.json is edited by both PRs, on different script lines. I didn't touch tools/README.md (held by #14), so its build/fx rows still describe the old plan; a follow-up should add the fx row there.

Language. The tools are .mjs to match tools/validate.mjs; the issue named the file tools/fx.ts.

Plan. docs/plan.md §6 still lists "FX normalisation policy" as pending; docs/fx.md now covers it.

Verification notes (advisory; they did not change the verdict):

  • described tools/fx.ts is not on the branch

Note: this branch was 1 commit(s) behind main when this pull request was opened.


🤖 Generated by Colonizer in a microVM

Co-Authored-By: Colonizer Settlers 331648616+colonizer-settlers@users.noreply.github.com

…ery usd block (#12)

Refs #12

Co-Authored-By: Colonizer Settlers <331648616+colonizer-settlers@users.noreply.github.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

FX policy: ECB reference rates in data/fx.json, and the usd block recomputed by the build

1 participant