Repository navigation
fix(aws/recommendations): paginate CE recommendation + RDS engine-version describe (single-page truncation) #692
Description
Symptom
Three AWS describe/recommendation calls in CUDly read only the first page of their result set and silently discard everything after. Each output struct has a NextPageToken (or NextToken) that we never follow. For customers with large payer accounts (i.e. the entire reason CUDly exists), Cost Explorer can return recommendations across multiple pages; we are emitting only a fraction of them and the user has no signal that it is partial.
This is a correctness bug, not a Lambda-budget hazard like #691: the values we emit are wrong, not "potentially slow". A customer can act on a partial RI/SP purchase recommendation thinking it is the complete picture.
Affected call sites (verified on feat/multicloud-web-frontend + PR #690 tip)
High-impact -- silently truncates customer-facing recommendations
-
providers/aws/recommendations/client.go:93--GetReservationPurchaseRecommendationis called inside a rate-limiter retry loop but the loop only retries on retryable errors.result.NextPageTokenis never read. Every page-2+ RI recommendation is silently dropped. This is the path that produces every AWS RI recommendation the UI shows. -
providers/aws/recommendations/parser_sp.go:63--GetSavingsPlansPurchaseRecommendation-- same shape (rate-limiter retry around a single SDK call,result.NextPageTokenignored). Drops every page-2+ Savings Plans recommendation.
Lower-impact -- truncates RDS engine-version metadata
cmd/multi_service_engine_versions.go:198--DescribeDBMajorEngineVersionsis called once per engine in afor _, engine := range enginesloop butoutput.NextTokenis never read. If AWS spans the major-version list across pages for any engine, the RDS extended-support filter (which uses this output) silently misses versions, causing either over-recommendation or under-recommendation of RDS RIs.
Fix shape
For each call site, wrap the SDK call in a token-driven pagination loop:
var nextPageToken *string
for {
if err := ctx.Err(); err != nil {
return nil, err
}
input.NextPageToken = nextPageToken // or input.NextToken for the SDK calls that use it
result, err := <existing rate-limiter loop, but now per-page>
if err != nil { ... }
// accumulate result.<items>
if result.NextPageToken == nil || aws.ToString(result.NextPageToken) == "" {
break
}
nextPageToken = result.NextPageToken
}Same maxPages cap (e.g. 20) + ctx.Err() between pages from the PR #690 / #691 pattern; same nil-or-empty terminator from CR's category-A finding on PR #690 (see commit f2becf790).
Regression test per call site
- Mock the SDK to return N pages with
NextPageTokenset, plus a final page withNextPageToken == nil. Assert the parser sees ALL recommendations across all pages, not just page 1. - Add a separate mock that returns empty-string
NextPageToken. Assert the loop terminates without an extra call (parity with the PR fix(purchases): narrow Describe*Offerings + cap pagination (#688) #690 fix).
Acceptance criteria
- All 3 call sites paginate via
NextPageToken/NextToken - Each loop has
ctx.Err()between pages and amaxPagescap with diagnostic error - Each loop terminates on
nilOR empty-string token (PR fix(purchases): narrow Describe*Offerings + cap pagination (#688) #690 parity) - Per-call-site regression test asserting the multi-page accumulation
- No regression in existing RI/SP rec tests
- Manual verification against a large payer account:
cudly recommendations refreshfor a 100+ account org returns more recs than before the fix
Cross-references
- fix(purchases): narrow Describe*Offerings calls + cap pagination across 7 services (lookup root cause behind 60s budget overrun) #688 / PR fix(purchases): narrow Describe*Offerings + cap pagination (#688) #690 (sibling pattern in the purchase-path 7 services)
- fix(azure/gcp/aws): add ctx.Err() guards and page-cap to all pagination loops #691 (cross-cloud bounded-pagination hygiene -- different smell, separate fix)
- fix(purchases): narrow Describe*Offerings calls + cap pagination across 7 services (lookup root cause behind 60s budget overrun) #688 punch-list comments capturing the original survey findings
Activity
fix(aws): paginate CE recommendations + RDS engine versions (closes #692
cristim commentedon May 28, 2026
Verified shipped on feat/multicloud-web-frontend via PR #693 (merged 2026-05-22).
GetReservationPurchaseRecommendation,GetSavingsPlansPurchaseRecommendation, andDescribeDBMajorEngineVersionsnow loop over all pages using NextPageToken/NextToken. Large payer accounts no longer silently receive only page-1 results.fix(aws): correct SP coverage denominator and utilization dimension v…
fix(aws): paginate CE recommendations + RDS engine versions (closes #692