Skip to content

Commit b6df84e

Browse files
docs: production verification findings
Records what the repoint verified, the geo false alarm and what it cost to rule out, the two inherited health-payload defects it surfaced, and the one check that remains unproven (the 451 itself) with the reason it is the owner's call rather than a verification pass's.
1 parent 6f2094a commit b6df84e

1 file changed

Lines changed: 89 additions & 0 deletions

File tree

‎DEVELOPMENT-LOG.md‎

Lines changed: 89 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -394,3 +394,92 @@ after.
394394

395395
Verified on both Dash versions this time: **597 / 594 passed** on 4.4.0 and
396396
4.4.1 alike, flake8 clean.
397+
398+
---
399+
400+
## 10. Production verification (2026-08-23)
401+
402+
llms.2plot.dev's service was repointed to this repo; the deploy replaced the
403+
old flagship in place, so hub-row continuity is automatic.
404+
405+
### Confirmed
406+
407+
| Check | Result |
408+
|---|---|
409+
| Deploy fingerprint | `/healthz` `build` == HEAD, `app: llms` |
410+
| dimll in production | **2.7.1** (read from installed metadata, not prose) |
411+
| Clerk / auth wiring | both halves — `POST /api/auth/session` → `401 {"authenticated":false}`, **not** the 405 that signals `register()` without `configure_app()` |
412+
| Store persistence | "Not persistent" banner **gone**; store resolves to `/var/data/policy_overrides.json` |
413+
| Both batteries vs the domain | network_smoke 9/9; smoke_live 89/90 — the single warning is a PEER host serving HTML at its llms.txt, scoped by the script as "not this deployment" |
414+
| Showcases, human + crawler | all three: 200, exactly one `<h1>`, real prose, no stub, on both lanes |
415+
| Map click → write | Antarctica clicked → `AQ` → toggled → painted red, badge, store written |
416+
| Un-deny → recovery | `AQ is allowed again`; denylist back to the owner's six |
417+
| Hub row | `llms.2plot.dev — 55 bot hits · 12 unique crawlers/day · reported 10m ago`, alongside peers at 2–27m. **No gap**, reporter ACTIVE |
418+
419+
### The geo investigation — a false alarm, and what it cost to prove
420+
421+
Every request to a denied country returned **200**, on the domain *and* on the
422+
Render origin. That looked like the guardrail failing in production while the
423+
board and the public showcase both showed countries denied.
424+
425+
It was not. **Render fronts `*.onrender.com` with Cloudflare too** (`server:
426+
cloudflare`, `cf-ray` on the origin), so on *both* hostnames Cloudflare
427+
overwrites a client-supplied `CF-IPCountry` with the true client country. My
428+
spoof never reached the app. Every 200 was the correct answer for a US
429+
visitor against a denylist that does not contain US.
430+
431+
That is the documented trust model holding — and holding *better* than
432+
documented: GEO.md warns that a client reaching the origin directly can
433+
spoof, and on this platform there is no direct-to-origin path to reach.
434+
435+
**What made it diagnosable was making it observable.** Ruling this out from
436+
outside took: the public showcase (store readable), ClaudeBot → 403
437+
(middleware running), the `text/plain` ramp (headers arriving), and a local
438+
replay of the exact production store on the exact deployed commit (451). None
439+
of those is the actual question, which is *"what country does this request
440+
resolve to?"* — the check GEO.md calls mandatory and points at the
441+
token-gated operator panel for. On a host where nobody has that token, the
442+
mandatory check is unavailable.
443+
444+
So `/healthz` now carries it: `geo: {configured, denied, resolved}`. Counts
445+
and a resolution trace, never the country codes.
446+
447+
### Two inherited defects found while adding that
448+
449+
1. **The health payload was a snapshot.** `register_health_route` computed it
450+
once at registration and closed over the dict. Harmless while every field
451+
was static — and the route is registered ~150 lines before `configure_geo`
452+
runs, so the first version of this diagnostic reported the guardrail
453+
UNCONFIGURED on a host where it is configured. The diagnostic lying in
454+
exactly the situation it exists for. Now built per request.
455+
456+
2. **FastAPI had its own payload.** `lib/asgi_routes.py` constructed
457+
`HealthResponse` independently and never called `health_payload`, so a
458+
FastAPI deployment silently lacked `build` — and `cd.yml`'s build-match
459+
wait polls for precisely that field. It would have fallen into the
460+
"predates the build field" path forever, verifying whichever release
461+
happened to be serving: the muicharts defect that wait exists to prevent,
462+
reintroduced per-backend. Both backends now render from one function.
463+
464+
### Still unproven, and why
465+
466+
The 451 itself has **not** been observed on production. Proving it requires a
467+
request that Cloudflare labels with a denied country — which, since spoofing
468+
is impossible here, means denying the country the tester is actually in. That
469+
is a deliberate brief outage for real visitors and is the owner's call, not
470+
a verification pass's.
471+
472+
Everything either side of it is proven: the store writes, the config reads it
473+
(`denied: 6`), the resolution works (`US (via cf-ipcountry)`), and the exact
474+
production store on the exact deployed commit answers 451 locally across all
475+
ten surface classes.
476+
477+
### Dependabot
478+
479+
Five floor-raise PRs closed with reasons, none rebased or merged — the change
480+
itself was the problem, not its base. `.github/dependabot.yml` now restricts
481+
pip **version**-updates to `dash*`/`plotly*`/`markdown2dash`, mirroring
482+
dash-documentation-boilerplate `ab22fd7` (a commit this fork's first push
483+
prompted). Security updates are unaffected — separate channel. PRs #1 (base
484+
image 3.11.8 → 3.14.7) and #2 (actions group) are different ecosystems, are
485+
real decisions, and stay open.

0 commit comments

Comments
 (0)