Commodity Price Tracker

Weekly tracker of US staple prices (eggs, milk, regular gasoline) pulled from two federal data APIs (USDA Market News and EIA Open Data), normalized onto a common weekly grid, and compared against a 2024 baseline.

Ran live on Vercel from February 2025 through early 2026 (now retired). The app still runs locally with zero configuration from the committed data snapshot:

npm install
npm run dev

What it shows

• Basket of Goods: a fixed weekly shop (1 dozen eggs + 1 gallon milk + 1 gallon regular gasoline) tracked against what the same basket cost on average in 2024.
• Per-commodity panels: latest price, week-over-week change, and delta vs. the 2024 annual average, in dollars and percent.

Data decisions

The frontend is the easy part. The real work was making two federal data sources with different shapes, cadences, and reporting days comparable.

1. Different sources, different shapes. USDA Market News reports agricultural prices as ranges (min/max) on ag-market weekdays; EIA reports energy prices as point estimates on a different schedule. 2024 egg prices predate the usable API window, so they come from a static USDA annual file while 2025 comes from the live API. Every series carries provenance metadata (dataSource: static-file | usda-api) so the blended history stays auditable.

2. Friday alignment. The series don't share dates, so nothing lines up week to week. Every observation is snapped to its week's Friday (adj_date), and multiple observations landing on the same Friday are averaged. All cross-commodity comparison happens on this grid, never on raw report dates.

3. A fixed 2024 baseline, not a moving average. Each commodity stores its 2024 annual mean (plus min/max) as metadata, and every panel reports the current price against that fixed reference. A frozen baseline gives a clean "vs. last year" comparison with no cherry-picked start date.

4. Raw / processed separation. data/prices.json holds raw observations with provenance; data/combined-prices.json holds the aligned grid, basket series, and chart-ready payloads. The pipeline (npm run process-prices) is the only thing that transforms data; the frontend makes one API call and only ever sees processed output.

5. Flagged missing data. A weekly basket entry is only marked isComplete when every commodity has a real price for that week. When one is missing, the latest known price is substituted and the entry is flagged (usedLatestPrices: true), not silently interpolated.

Every number on the dashboard traces back to a raw observation through this pipeline.

What I learned

Upstash KV was the wrong storage choice. It's a dict: get, set, keys, done. That was enough to serve one processed blob to the frontend. However, anything past "give me the blob" (backfill one commodity, inspect a date range, fix a bad week) meant pulling the whole payload down, editing it in a script, and writing it back. Price observations are rows with dates; they belong in SQL. KV is for caching, not for data you intend to query.

Architecture

USDA / EIA APIs ──> init & update scripts ──> raw JSON (prices.json)
                                                   │
                                     process-prices (align → average → baseline → basket)
                                                   │
                                     processed JSON (combined-prices.json)
                                                   │
                                     Next.js API route ──> Recharts dashboard

• Production (retired): data lived in Upstash KV on Vercel, refreshed by a cron API route.
• Local: storage falls back to the committed JSON files in data/. No keys, no KV, no setup.

Scripts

Script	What it does
npm run dev	Start the dashboard (works offline from the data snapshot)
npm run init-prices	Fetch and initialize all price data (requires USDA_API_KEY in .env.local)
npm run update-eggs / update-milk	Refresh one commodity and reprocess
npm run process-prices	Rebuild the aligned/basket data from raw

Notes & limitations

• The committed snapshot is frozen at late February 2025; the live deployment kept updating via cron until it was retired.
• Charts render the 2025 weekly series; 2024 appears as the annual-average baseline (the full 2024 weekly series is only partially retained in the snapshot).
• Egg prices in the snapshot capture the early-2025 avian-flu spike. The +166% vs. 2024 average reading is real, not a data bug.

Stack

Next.js 15 · React 19 · Recharts · shadcn/ui · Tailwind CSS · Upstash KV (production) · Node ingestion scripts