Repository navigation
fix(azure/gcp/aws): add ctx.Err() guards and page-cap to all pagination loops #691
Description
Summary
Pagination loops across the Azure and GCP provider clients are missing ctx.Err() checks between pages and have no maximum-page budget. This is the same failure shape fixed in PR #690 for the 7 AWS services -- discovered during the survey requested in #688.
All affected loops are unbounded and lack an early-exit on context cancellation. When running under a Lambda deadline these loops can eat the entire remaining budget before the SDK surfaces the cancellation error.
Affected locations
Azure -- for pager.More() { pager.NextPage(ctx) } without ctx.Err() check (smell #2)
providers/azure/services/compute/client.go:136-- GetRecommendationsproviders/azure/services/compute/client.go:171-- GetExistingCommitmentsproviders/azure/services/compute/client.go:362-- GetValidResourceTypesproviders/azure/services/cache/client.go:136-- GetRecommendationsproviders/azure/services/cache/client.go:170-- GetExistingCommitmentsproviders/azure/services/cache/client.go:359-- GetValidResourceTypesproviders/azure/services/database/client.go:138-- GetRecommendationsproviders/azure/services/database/client.go:172-- GetExistingCommitmentsproviders/azure/services/cosmosdb/client.go:139-- GetRecommendationsproviders/azure/services/cosmosdb/client.go:173-- GetExistingCommitmentsproviders/azure/services/cosmosdb/client.go:364-- GetValidResourceTypesproviders/azure/services/search/client.go:136-- GetRecommendationsproviders/azure/services/search/client.go:170-- GetExistingCommitmentsproviders/azure/services/search/client.go:358-- GetValidResourceTypes
GCP -- iterator loops without ctx.Err() check (smell #2)
providers/gcp/services/computeengine/client.go:205-- GetRecommendations iteratorproviders/gcp/services/computeengine/client.go:247-- GetExistingCommitments iteratorproviders/gcp/services/computeengine/client.go:441-- GetValidResourceTypes iteratorproviders/gcp/services/cloudsql/client.go:163-- GetRecommendations iteratorproviders/gcp/services/memorystore/client.go:163-- GetRecommendations iteratorproviders/gcp/services/memorystore/client.go:203-- GetExistingCommitments iterator
Minimum fix (small effort)
For each for pager.More() loop body, add at the top:
if err := ctx.Err(); err != nil {
return nil, fmt.Errorf("context cancelled during pagination: %w", err)
}For each GCP for { it.Next() } loop body, same guard.
Add a const maxPages = 50 (or suitable value) page budget per loop with a warning log when hit, matching the pattern from PR #690.
Context
- Parent issue: fix(purchases): narrow Describe*Offerings calls + cap pagination across 7 services (lookup root cause behind 60s budget overrun) #688
- Reference fix: PR fix(purchases): narrow Describe*Offerings + cap pagination (#688) #690 (applied the same pattern to 7 AWS services)
A separate issue covers the Azure Retail Prices API NextPageLink truncation (smell #5) in the same files -- that fix is larger and touches the HTTP-level pricing helpers.
Activity
cristim commentedon May 22, 2026
Scope extension: AWS rec-collection (same ctx-cancellation smell)
Folding in 2 AWS findings from the AWS-only follow-up survey (#688 punch-list). Same smell #2 (
ctx.Err()not treated as terminal), same fix shape -- belongs in this issue alongside the Azure/GCP loops.AWS -- ctx-cancellation accumulated as skippable failure instead of returning immediately
providers/aws/recommendations/client.go:119-120--GetAllRecommendationsloops over services callingGetRecommendationsForService(ctx, service). On error (includingcontext.Canceled/context.DeadlineExceeded) it logs and continues, returning partial results with nil error at line 126. Callers cannot distinguish a complete sweep from one that timed out halfway.providers/aws/recommendations/client.go:452-454--getSavingsPlansRecommendationshas the same pattern across SP types (line 463 returns partial results with nil error after a per-type error swallow).Fix shape (same as the Azure/GCP fix)
Inside each per-service / per-type loop body, before continuing:
Project memory entry:
feedback_ctx_cancel_terminal.md-- ctx cancellation is terminal in fan-out loops, never accumulated aslastErrand continued.Other AWS findings (separate issue needed)
Three other AWS findings from the survey are different smells and warrant their own issue, not this one:
recommendations/client.go:79--GetReservationPurchaseRecommendationsingle-page (smell Remove Hard SES dependency #5, correctness bug, silently misses RIs beyond page 1 on large payer accounts)recommendations/client.go:446--GetSavingsPlansPurchaseRecommendationsingle-page (same shape for SP)cmd/multi_service.go:1335--DescribeDBMajorEngineVersionssingle-pageA fourth finding (ctx-cancellation in a
DescribeDBInstancesfan-out) is low-risk side-path and can ride with the single-page ones.[-]fix(azure/gcp): add ctx.Err() guards and page-cap to all pagination loops[/-][+]fix(azure/gcp/aws): add ctx.Err() guards and page-cap to all pagination loops[/+]cristim commentedon May 28, 2026
Verification: cannot-verify-autopilot - Partially shipped. PR #693 (merged 2026-05-22) adds pagination to three AWS API calls (CE recommendations, RDS engine versions). However, the issue explicitly asks for ctx.Err() guards and page-cap in Azure and GCP pagination loops. No commit in the git history for
feat/multicloud-web-frontendaddresses Azure or GCP pagination loops with ctx.Err() guards.fix(azure/gcp/aws): ctx.Err guards + page cap on 22 pagination loops (c…