feat(database): optimize timescaledb queries with continuous aggregat… - #998
Merged
Mosas2000 merged 2 commits intoJul 29, 2026
Merged
Conversation
…es (StellaBridge#798) Add migration 048 to configure continuous aggregate views and refresh policies on prices, health_scores, and liquidity_snapshots hypertables. Refactor PriceModel, AnalyticsService, AggregationService, and LiquidityFragmentationService to query continuous aggregates when range >= 7 days.
…llaBridge#798) Add browser caching step for ~/.cache/ms-playwright and increase job timeout-minutes from 20 to 30 to prevent Playwright installation timeouts.
Contributor
|
Fix the failing E2E check |
Contributor
|
The implementation of TimescaleDB continuous aggregates is excellently structured, featuring robust fallback mechanisms that will significantly enhance long-range query performance. |
Contributor
|
The thorough test coverage effectively validates the newly optimized routing logic. Thank you, approving this for merge. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Description$\ge 7$ days, 30 days, 90 days) on the Analytics dashboard previously executed expensive raw table queries.
Over 90 days of operation, the prices, health_scores, and liquidity_snapshots hypertables accumulate millions of rows. Aggregating data over long ranges (
This PR introduces TimescaleDB continuous aggregate materialized views with automated refresh policies for 1-hour and 1-day windows across high-volume hypertables, and refactors backend services to route queries to continuous aggregates when requesting data ranges$\ge 7$ days.
Type of Change
Optimization (performance enhancement)
Database Migration
Refactoring (no logic breaking changes)
Key Changes
Created migration 048_continuous_aggregates_optimizations.ts defining TimescaleDB continuous aggregate materialized views:
prices_hourly & prices_daily
health_scores_hourly & health_scores_daily
liquidity_hourly & liquidity_daily
Configured automated refresh policies via add_continuous_aggregate_policy for 1-hour and 1-day bucket intervals.
Added performance indexes on (symbol, bucket DESC) for each aggregate view.
PriceModel (backend/src/database/models/price.model.ts): Refactored getTimeBucketed() to target prices_daily for lookbacks
AnalyticsService (backend/src/services/analytics.service.ts): Refactored getHistoricalComparison() for health_score, liquidity, and price metrics to query continuous aggregates when requested range is
AggregationService (backend/src/services/aggregation.service.ts): Refactored aggregatePrices(), aggregateHealthScores(), and aggregateVolume() to query continuous aggregate views for long time ranges (
LiquidityFragmentationService (backend/src/services/liquidityFragmentation.service.ts): Refactored getFragmentationTrend() for 7d and 30d ranges to query liquidity_hourly and liquidity_daily.
Added test file backend/tests/services/continuousAggregates.service.test.ts to verify query routing logic for
Confirmed all service unit tests pass cleanly:
Test Files 5 passed (5)
Tests 79 passed (79)
Impact & Compatibility
Breaking Change: No
Short-term Feeds: Unaffected (queries under 7 days continue to read directly from raw hypertables).
Closes #798