Repository navigation
feat(data): subscription token math — profiles, binding window, $/1M at full use (#13) - #65
Merged
Merged
Conversation
3 tasks
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Implements issue #13: one mechanical definition of "tokens per month at full use" for subscription plans, published as versioned assumption profiles, computed identically in the build language (TypeScript) and the Worker language (Rust), and pinned by one shared set of test vectors.
What changed
New files (9, ≈2,530 lines)
data/profiles/agentic-coding-v1.json(default),light-chat-v1.json,heavy-agentic-v1.json— one call = 20k input (18k cache-read + 2k uncached) + 1.5k output = 21,500 tokens for the default; 4 × 5-hour windows/day; 30-day month; reference pricing (Sonnet 5.5 list) for dollar/credit conversion. Each carries"policy": "immutable-once-published"— fixes mean publishing-v2, never editing.tests/vectors/token-math.json— 31 shared vectors: 20 seed plans (Google AI Pro/Free/Ultra/Standard, Z.AI lite/pro/max, GitHub Copilot pro/max, Warp build/max, Augment, JetBrains ultimate/free, Cerebras pro/max, StackBlitz, Alibaba Coding Plan Pro, BytePlus Lite, Synthetic), covering every unit type (requests, messages, prompts via the edge vectors, credits, usd, raw tokens) and every binding window (5h ×2, daily ×7, weekly ×5, monthly ×11), plus edge cases: no cap →"unknown"(anthropic/pro), relative multiples ignored (openai/pro-200 "20x"), credits with no published conversion →"unknown"(alibaba-cn lite), secondary-confidence caps never derived (perplexity/pro), weekly-only cap, off-peak noted but never applied (Z.AI), multiple caps in one window → minimum, and alt-profile checks (light-chat/heavy-agentic, incl. a tokens-pass-through invariance case).tools/math.ts— the library:estimateForPlan,profileCostPerCall,enrichSubscriptionRow,usdPriceMonthly. Pure, dependency-free ESM.tools/math.test.mjs— vitest suite (37 tests): the 31 shared vectors, a drift check over all 103 dataset rows, schema validation of an enriched copy, and unit tests for op-order/tie/price-resolution edges.crates/tokker-math— pure, wasm-safe Rust twin (Cargo.toml,src/lib.rs,tests/vectors.rs); serde-typed, no I/O in the library; the integration test runs the sametests/vectors/token-math.jsonviainclude_str!.Modified (5 files, +15/−2)
schema/pricing.v1.json— one additive optional property:binding_window(enum5h_rolling|daily|weekly|monthly|null) on subscription rows. Not added torequired; the untouched dataset still validates. Per docs/SCHEMA.md this is the minor path; the data file stays atschema_version 1.0.0because this PR does not rewrite data (see "Sibling coordination").tools/validate.mjs— one line:ESTIMATE_PROFILESnow also acceptslight-chat-v1/heavy-agentic-v1(legacylight-chat/heavy-agenticnames still accepted).docs/plan.md— one-sentence implementation note: the build useswindows_per_month = 4 × 30 = 120andweekly × 30/7(the dataset's STANDARD strings, which the committed estimates were computed with); the formula's4.33is shorthand.tools/README.md— one table row.Cargo.lock— tokker-math entry. RootCargo.tomluntouched (members = ["crates/*"]glob).The spec (identical in both languages, f64 op order fixed)
K = ((input−cached)·r_in + cached·r_cached + output·r_out)/1e6(default profile: $0.0226/call). A limit counts only if its amount is numeric, its window is 5h/daily/weekly/monthly, the row is not secondary-confident, and its unit is convertible. Monthly units: 5h ×120, daily ×30, weekly (a×30)/7, monthly ×1; token multiplier: rawtokenspass through unchanged, requests/prompts/messages ×21,500, credits via the plan's publishedcredits_per_call(Z.AI 8.04) orusd_per_credit(Copilot $0.01), usd via/K.est_tokens_per_month = min(candidates);binding_window= argmin (ties → coarser window);est_usd_per_mtok = round4(price_usd / tokens × 1e6). Off-peak multipliers are never applied. A vendor that publishes no cap yields"unknown"— never an invented number. Two errata vs the issue text, discovered against committed data and documented in the vectors'$comment: (1) token-denominated caps are vendor-quoted tokens, not calls, so they do not multiply by 21,500; (2)tokens_per_weekderives only from a weekly cap or a 5h cap (×28) — daily caps are never extrapolated.How verified
cargo test -p tokker-math11/11), and a scratch twin-harness ran the Rust lib over all 103 dataset rows and diffed againstenrichSubscriptionRow: 103/103 identical on est_tokens, est_usd and binding_window.estimate_assumptionstrings (Anthropic finopsllm secondary factor; OpenAI range midpoints/multiples; Zed markup; T3 dollar-window; relative multiples; 5 rows where committed"unknown"diverges from mechanically derivable caps — filed for owner review).binding_windowper row) and validates it against the schema with ajv.npm run validate,npm test(48+ tests),cargo fmt --check,cargo clippy --workspace --all-targets -- -D warnings,cargo test --workspace,cargo doc -D warnings,cargo build -p tokker-worker --target wasm32-unknown-unknown(tokker-math itself also builds for wasm32), no duplicate cratefield deps. Note: npm is absent in this sandbox; a bootstrap npm ran the identical commands, andpackage.json/package-lock.jsonare byte-identical to HEAD.Acceptance vs docs/report.md
Reproduced within rounding from published caps: Google AI Pro 0.0207≈0.021 (daily binds), Synthetic 0.0233≈0.023, Alibaba Coding Plan Pro 0.0258≈0.026, BytePlus Lite 0.0388≈0.039, Z.AI 0.1047–0.1571 inside 0.10–0.16 (weekly binds, as the table says), Copilot 0.5256–1.0512. Deliberate differences, explained: the Claude Pro/Max 5x (~0.065) and Max 20x (~0.077, weekly-binds) and ChatGPT Pro (~0.044) report rows come from secondary derivations (community measurements, published-range midpoints, relative multiples) that the mechanical math refuses to invent; vectors pin those plans
"unknown"with notes, and the committed rows keep their labelled secondary numbers (drift exceptions). Claude Max 20x's weekly-binding behaviour is exercised by the Z.AI vectors instead, since Claude publishes no numeric weekly cap.Size
≈2,545 changed lines across 14 files — over the 400/10 ceiling. The issue mandates the bulk: three profile files, a shared vector file with 15+ plans plus edge cases, and a complete second implementation in Rust with its own tests. The drift/twin checks (the strongest guarantee that build-math and Worker-math cannot diverge) account for most of the rest.
data/pricing.jsonitself is untouched.Sibling coordination
#10 (dataset split) owns the build: when its
tools/build.tslands, callingenrichSubscriptionRowper plan row (from thedata/plans/<vendor_id>.jsonfragments) writesest_*+binding_window, at which pointschema_versionshould bump to 1.1.0. #11 confirmed no overlap.tools/validate.mjswas touched only on the one ESTIMATE_PROFILES line. Verified byte-identical to HEAD in this branch:data/pricing.json,package.json,package-lock.json.Reviewer watch-list
"unknown"coexists with mechanically derivable caps — especiallygithub/free(numericmonthly/requests=2000vs assumption text "vendor publishes no absolute quota"). Filed separately for owner review.zai/pro,zai/max,byteplus/liteare flaggedconfidence:"secondary"while their caps reproduce exactly from official caps + published conversions — flag needs owner review (filed).debug_assert!in Rust (release builds skip them) — deliberate Rust idiom; the vectors pin well-formed profiles either way.Closes #13
🤖 Generated by Colonizer in a microVM
Co-Authored-By: Colonizer Settlers 331648616+colonizer-settlers@users.noreply.github.com