infra(cdn): put Cloudflare in front of Cloud Run for AI Bot Activity + edge TTFB

[julianken]

Context

detached-node.dev serves directly from Google Cloud Run (us-west1) with no CDN. Two consequences:

1. No AI Bot Activity visibility in Microsoft Clarity. Clarity's Bot Activity dashboard (which tracks GPTBot, ClaudeBot, PerplexityBot, CCBot, etc., per-page and per-frequency) requires a connected CDN — Fastly, CloudFront, or Cloudflare. Currently invisible: we cannot see which AI crawlers are hitting which pages or how often. See https://learn.microsoft.com/en-us/clarity/ai-visibility/bot-activity-overview.
2. Global TTFB suboptimal. Cloud Run runs only in us-west1. EU/APAC visitors pay full transcontinental round-trip on every uncached request.

Plan and cost. The Clarity ↔ Cloudflare integration uses Cloudflare LogPush, which Microsoft's own cost-considerations doc notes is "typically available on paid plans and might have usage-based pricing" (https://learn.microsoft.com/en-us/clarity/ai-visibility/cost-considerations-bot-activity-integrations). LogPush to HTTPS destinations requires Cloudflare Pro plan (~$25/month) or higher. Free tier covers the proxy/TTFB/DDoS half, but not the AI Bot Activity unlock. This issue assumes Pro and budgets accordingly.

Acceptance criteria

• Cloudflare account with a zone for detached-node.dev on Pro plan (or higher if needed for LogPush destination)
• DNS migration to Cloudflare nameservers (required — Cloudflare must be authoritative for the zone; current Vercel nameservers cannot proxy through Cloudflare):
  • Mirror all existing Vercel DNS records at Cloudflare before flipping NS: A/AAAA for apex + any subdomains, MX records, all TXT records (SPF, DKIM, DMARC, domain-verification, sitemap-verification, any other vendor-verification TXTs)
  • Verify mirrored records with dig @<cloudflare-ns> <record> before NS change
  • Flip nameservers at the registrar to Cloudflare; confirm propagation
• Cloudflare proxy enabled (orange cloud) for the apex
• SSL/TLS mode set to Full (strict) to maintain end-to-end TLS to Cloud Run
• Cloud Run origin lockdown (in scope — protects the integrity of the Bot Activity numbers; the *.run.app hostname is discoverable via certificate transparency logs, so bots that find it can bypass Cloudflare entirely):
  • Restrict the Cloud Run service to Cloudflare IP ranges (https://www.cloudflare.com/ips/) via ingress controls or an in-app middleware check
  • Verify: curl -v https://<service>-<hash>-uw.a.run.app/ from a non-Cloudflare IP returns 403 or times out
  • Verify: curl -sI https://detached-node.dev/ (via Cloudflare) still returns 200
• CDN connected to Microsoft Clarity per https://learn.microsoft.com/en-us/clarity/ai-visibility/bot-activity-overview — verify Bot Activity dashboard populates within 48 hours of LogPush activation
• Budget AC: total monthly Cloudflare spend ≤ $30/mo (Pro is ~$25/mo; LogPush usage adds a small variable). If projected cost exceeds the ceiling, pause and re-decide.
• Production verification:
  • curl -sI https://detached-node.dev/ shows server: cloudflare and a cf-ray header
  • curl https://detached-node.dev/sitemap.xml returns the sitemap intact
  • curl https://detached-node.dev/robots.txt returns robots.txt intact
  • Multiple routes render correctly in a browser
• Document the Cloudflare account/zone ID, plan, LogPush configuration, and origin-lockdown method in docs/deployment.md

Out of scope

• Cloudflare Workers, R2, D1 — proxy + DNS + LogPush only
• Page Rules / Transform Rules tuning — baseline only; defer optimization
• WAF rules beyond Pro-tier defaults — defer
• Plan upgrades beyond Pro unless LogPush destination requires it (revisit if so)

Plain-English explanation

1. What: Put Cloudflare in front of the site so AI bot traffic becomes visible in Clarity's Bot Activity dashboard and global users get faster page loads. Also lock down the Cloud Run origin so bots can't bypass the CDN.
2. Why now: Bot Activity is the biggest blind spot in AI-discovery measurement — without a CDN, we cannot see which AI crawlers visit. Without origin lockdown, the numbers we do see would be incomplete by an unknown margin.
3. Cost: ~$25–30/mo on Cloudflare Pro for the LogPush feature that powers the Clarity integration. Free tier is insufficient per Microsoft's own docs.
4. Risk: Medium during DNS migration window (brief but visible) — mitigated by mirroring all records at Cloudflare before flipping NS. Cloudflare proxy adds <50ms first-byte for cache misses but improves p99 on geographic edges. Worst case: revert by disabling the orange cloud proxy and flipping NS back to Vercel.
5. Validation: Bot Activity dashboard populates within 48 hours of LogPush activation. cf-ray header present in production responses. Non-Cloudflare requests to the *.run.app origin return 403/timeout.

Note: This is primarily an ops task — DNS migration + dashboard setup + Clarity integration + origin lockdown — with minimal code changes. An implementer subagent may help with the docs/deployment.md update and any Cloud Run middleware for IP filtering; the DNS/dashboard steps are Julian-direct.

[julianken-bot]

Verdict: REQUEST CHANGES ⚠️

Reviewed as @julianken-bot (fresh context, opus tier) against the anti-slop spec-review rubric. Three findings, ordered by impact.

---

IMPORTANT — Free-tier premise contradicts Microsoft's own docs

The spec asserts "Cloudflare Free tier is sufficient for both unlocks." Microsoft Learn's official cost-considerations page for Bot Activity (verified 2026-05-18) says the Cloudflare integration uses Cloudflare LogPush and that "LogPush is typically available on paid plans and might have usage-based pricing." See https://learn.microsoft.com/en-us/clarity/ai-visibility/cost-considerations-bot-activity-integrations.

The TTFB / DDoS half of the AC is genuinely Free-tier. The AI Bot Activity half — the headline goal — is not, per Microsoft. Before starting work, either:

1. Confirm against Cloudflare's current LogPush availability matrix that Free-tier LogPush is now enabled for this destination (Cloudflare has been opening LogPush further over time), OR
2. Update the spec to acknowledge that the Clarity integration may require Pro/Business/Enterprise and that the cost is bounded but nonzero.

If (2), add a budget line to the AC ("acceptable monthly cost: $X").

IMPORTANT — DNS migration path is not viable as written

AC says "DNS for detached-node.dev (apex) migrated to Cloudflare nameservers (or existing DNS provider proxies through Cloudflare)."

Current nameservers (verified via dig +short NS detached-node.dev):

ns1.vercel-dns.com.
ns2.vercel-dns.com.

Vercel DNS does not proxy through Cloudflare — Cloudflare's orange-cloud proxy requires Cloudflare to be authoritative for the zone. The "or" branch is unreachable for this domain. The only viable path is a full nameserver migration to Cloudflare. The spec should drop the "or" clause and add an explicit step: "verify all existing Vercel DNS records (TXT, MX, etc.) are mirrored at Cloudflare before flipping NS." DNS migrations that lose records (email SPF/DKIM, domain-verification TXT) are how this kind of work breaks silently.

SUGGESTION — "Restrict Cloud Run to Cloudflare IPs" deserves a stronger framing

Spec frames Cloud Run origin lockdown as a "judgment call — defer if extra complexity for marginal value." This under-weights one specific consequence: AI bots that know the *.run.app hostname (it's discoverable via certificate transparency logs) can bypass Cloudflare entirely. That means the Bot Activity dashboard will be incomplete by an unknown margin — the very metric the project exists to unlock.

Recommend either (a) doing origin lockdown in this issue, or (b) adding a follow-up issue explicitly: "AC: verify Cloud Run origin returns 403 / network error for non-Cloudflare clients before declaring the Clarity numbers trustworthy."

---

Notes

• Acceptance criteria are testable and well-scoped — good shape.
• Out-of-scope list correctly excludes Workers/R2/D1.
• "Plain-English explanation" section is genuinely useful for an ops task.
• Risk framing ("revert by disabling the orange cloud") is honest.

Bottom line: the spec is the right shape and the right scope. The LogPush-plan question is the one that could turn this from a $0 task into a $20/month task or block it entirely, so it should be answered before work starts. The DNS clarification is a 2-line edit. Origin lockdown is debatable and the spec already debates it; just sharpen the trade-off.

---

Reviewed as @julianken-bot. Fresh context, no implementer narrative. Verification: dug DNS, fetched current Microsoft Learn cost doc, hit production headers.

[julianken]

Spec revised per bot review. Addressed:

• Free-tier premise: replaced with Cloudflare Pro (~$25/mo) per MS cost-considerations doc; added $30/mo budget AC.
• DNS migration: dropped the unreachable Vercel-proxy branch; added explicit step to mirror all existing records (A/AAAA, MX, SPF/DKIM/DMARC/verification TXTs) at Cloudflare before flipping NS.
• Origin lockdown: promoted from deferrable to in-scope AC with concrete verification (curl from non-Cloudflare IP returns 403/timeout).
• Stale URL fixed: /ai-bot-activity → /bot-activity-overview throughout.

Ready for re-review.

[julianken-bot]

Verdict: APPROVE ✅

Re-reviewed as @julianken-bot (fresh context, opus tier). All three prior findings resolved:

• Free-tier premise — now correctly states LogPush requires Cloudflare Pro (~$25/mo) per MS cost-considerations doc; budget AC of ≤$30/mo added. ✓
• DNS migration — unreachable "Vercel proxies through Cloudflare" branch removed; explicit mirror-before-flip checklist with dig @<cloudflare-ns> verification covers A/AAAA, MX, SPF/DKIM/DMARC and all vendor-verification TXTs. ✓
• Cloud Run origin lockdown — promoted from "judgment call" to in-scope AC with concrete curl verification from both a non-Cloudflare IP (expect 403/timeout) and via Cloudflare (expect 200). ✓
• Stale URL — /ai-bot-activity replaced with /bot-activity-overview throughout. ✓

R8 second pass turned up one minor implementation note worth flagging without blocking: Cloud Run native ingress (internal / internal-and-cloud-load-balancing / all) cannot restrict by external IP CIDRs on its own — Cloudflare-IP allowlisting typically needs Cloud Armor in front of an HTTPS LB, or the in-app middleware path the spec already mentions. The hedging language ("ingress controls or an in-app middleware check") leaves this open correctly, so it is informational, not a blocker.

Budget margin is tight ($25/mo plan + variable LogPush usage against a $30/mo ceiling) but the spec already has the right escape hatch ("pause and re-decide").

Spec is ready to dispatch.

---

Reviewed as @julianken-bot. Verification: fetched issue body + prior comment thread; cross-referenced MS Learn cost doc URL; sanity-checked Cloud Run ingress capabilities against my training prior.