Skip to content

Automation: schedule the data-freshness jobs via GitHub Actions - #31

Open
aurph wants to merge 2 commits into
mainfrom
feat/freshness-automation
Open

aurph wants to merge 2 commits into
mainfrom
feat/freshness-automation

Conversation

@aurph

@aurph aurph commented Sep 3, 2026

Copy link
Copy Markdown
Owner

The root cause behind the 97-day-stale deals dataset (and the 62-day clusters) was never the data — it was that no refresh mechanism was ever scheduled, and the fix sat on a to-do list as "set up two cron-job.org jobs." This puts the schedule in the repo, where it can't be forgotten:

Job Cadence What it does
news-scan daily 7:17am ET POST /api/admin/scan-news-now — feeds the deals/queue approval pipeline
recorder-ping weekdays 11:17am ET GETs /api/gpu-prices/metrics + /api/kpis so the price/index history series accrue without organic traffic (after the index recorder's 10am ET gate)
deadman 2x daily GET /api/admin/freshness/check — a 503 names the stale datasets and fails the workflow, so GitHub's built-in failure email is the alert

Design rules carried over from ops/freshness-monitor.md: the watchdog lives off the infrastructure it watches (GitHub, not the Jetson or Replit), and it hits /freshness/check, never /health, so Replit can't mistake a stale-data 503 for an unhealthy instance. workflow_dispatch enabled for manual runs; permissions: contents: none.

Activation (one step): add repository secret GRIDTILT_ADMIN_KEY = the production ADMIN_API_KEY (Settings → Secrets and variables → Actions). Until it exists, the two admin jobs fail with a message saying exactly that; the public recorder-ping works immediately. Schedules run from the default branch, so this arms on merge. (GitHub pauses schedules after ~60 days of repo inactivity; any commit re-arms them.)

Verified: /api/kpis returns 200 live, route paths grep-confirmed against the code, YAML parse-validated. Carries the #30 lockfile fix so CI is green here.

🤖 Generated with Claude Code

https://claude.ai/code/session_01FY4ziPhfu8VBPFVy5cDMNy

Two advisories published upstream since the last merge (browserslist
unbounded memory growth, HIGH; postcss-selector-parser AST recursion,
low) trip CI's npm-audit gate on every new push, blocking the whole
open PR train. npm audit fix; 0 vulnerabilities after; tests/tsc/build
green. Lockfile-only.
The root cause of the 97-day-stale deals dataset was that no refresh
mechanism was ever scheduled - the fix sat on a to-do list as 'set up
two cron-job.org jobs'. This puts the schedule in the repo instead:
daily news scan (feeds the deals/queue approval pipeline), weekday
recorder pings (gpu-price + index history accrue without organic
traffic), and a twice-daily freshness deadman that fails the workflow
on a 503 so GitHub's built-in failure email is the alert. Watchdog
stays off the watched infrastructure (GitHub, not Jetson/Replit) and
hits /freshness/check, never /health, per ops/freshness-monitor.md.

Activation = one repository secret: GRIDTILT_ADMIN_KEY (the production
ADMIN_API_KEY). Until set, the two admin jobs fail with a message
saying exactly that. Schedules run from the default branch, so this
arms on merge. Registry mechanism note updated.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant