[Metrics] Speed up new activity query

[nezaj]

I gave Claude the output of EXPLAIN on the current query and it identified that finding the earliest tx date for an app was the most expensive part (huge scan)

Instead we can do two things:

• Restrict transaction scan to only new txs since our last entry for our daily_app_transactions table
• Use active_date from daily_app_transactions table to get earliest date for existing apps

We'll still get all new transactions and look at a lot less data. Tested this for todays job and the query time wen from 6.5 min -> ~50 seconds

[coderabbitai]

No actionable comments were generated in the recent review. 🎉

ℹ️ Recent review info

⚙️ Run configuration

Configuration used: Repository UI

Review profile: CHILL

Plan: Pro

Run ID: 06c85070-58c5-41a9-a5ce-5ee32dd3c631

📥 Commits

Reviewing files that changed from the base of the PR and between e710176 and 4f57ad2.

📒 Files selected for processing (1)

• server/src/instant/scripts/daily_metrics.clj

---

📝 Walkthrough

Walkthrough

The insert-new-activity function's SQL backfilling logic for daily app transaction metrics was rewritten. The new CTE pipeline computes active dates per app using recorded dates or a 7-day default offset instead of per-app timestamps with explicit exclusion.

Changes

Daily Activity Metrics Backfill

Layer / File(s)	Summary
Active date computation in window backfill server/src/instant/scripts/daily_metrics.clj	The SQL CTE chain in insert-new-activity now uses window_txns, recorded_active_dates, and app_active_dates CTEs to compute per-app active dates from recorded rows or a 7-day offset default, then inserts aggregated rows with is_active based on created_at > active_date, replacing the earlier per-app earliest-timestamp cross-join and hardcoded-date approach.

Estimated code review effort

🎯 4 (Complex) | ⏱️ ~40 minutes

🚥 Pre-merge checks | ✅ 5

✅ Passed checks (5 passed)

Check name	Status	Explanation
Title check	✅ Passed	The title '[Metrics] Speed up new activity query' accurately summarizes the main change: optimizing query performance in the daily metrics calculation.
Description check	✅ Passed	The description is directly related to the changeset, explaining the optimization strategy and providing concrete performance metrics (6.5 min to ~50 seconds).
Docstring Coverage	✅ Passed	No functions found in the changed files to evaluate docstring coverage. Skipping docstring coverage check.
Linked Issues check	✅ Passed	Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check	✅ Passed	Check skipped because no linked issues were found for this pull request.

_{✏️ Tip: You can configure your own custom pre-merge checks in the settings.}

✨ Finishing Touches

🧪 Generate unit tests (beta)

• Create PR with unit tests
• Commit unit tests in branch faster-metrics-query

---

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

• X
• Mastodon
• Reddit
• LinkedIn

_{Comment @coderabbitai help to get the list of available commands and usage tips.}

[dwwoelfel]

Nice!