Skip to content

Commit 30e8912

Browse files
cristimcristim
authored
fix(purchases): tighten AWS SDK retry budget + bump Lambda timeout to 300s (closes #683) (#684)
* feat(providers/aws): add purchasecfg helper to bound purchase-path SDK retries Introduce providers/aws/internal/purchasecfg.NewConfig which returns a copy of the caller's aws.Config with RetryMaxAttempts=2 and http.Client.Timeout=15s. Worst-case wall clock: 2 * 15s = 30s, well inside the 300s Lambda budget. Recommendation-collection clients (recommendations.NewClient) are explicitly NOT touched; they call Cost Explorer and need the higher default retry budget. Adds five unit tests covering: retry cap, HTTP timeout, immutability of the base config, region preservation, and the wall-clock bound invariant. Closes #683. Refs #667 #668 #632 #681. * fix(purchases): wire purchasecfg into all 7 purchase-path SDK constructors Switch EC2, RDS, ElastiCache, MemoryDB, OpenSearch, Redshift, and SavingsPlans NewClient calls to use purchasecfg.NewConfig(cfg) so every purchase-path SDK client gets RetryMaxAttempts=2 and http.Client.Timeout=15s. OpenSearch and Redshift also pass the tightened config to their STS client (used for ARN construction / tagging on the purchase path). Recommendation-collection clients (recommendations.NewClient) are not touched; they use their own constructor and deliberately keep the SDK default retry budget for Cost Explorer calls. All 694 tests pass. go build ./... clean. Closes #683. Refs #667. * fix(terraform): bump Lambda timeout 60 -> 300s in all three env tfvars Raise lambda_timeout from 60s to 300s in github-dev, github-staging, and github-prod tfvars. The SDK retry tightening (purchasecfg, max 30s wall clock) combined with the increased Lambda budget means transient AWS API slowness either completes within 30s or surfaces a clear error to the user -- no more silent context deadline exceeded from the Lambda itself expiring mid-retry. Lambda Function URL has no upstream timeout ceiling, so 300s is safe. Closes #683. Refs #667 #681. * feat(purchase): cap per-rec execution at 30s via context.WithTimeout Each recommendation in processPurchaseRecommendations now runs under a 30s per-rec deadline derived from context.WithTimeout. This matches the purchasecfg hard limits (2 retries * 15s HTTP timeout) so a single hung rec (SDK retry storm, STS hang, or pagination hang) cannot exhaust the Lambda budget and starve the remaining recs in the same execution. On timeout, a purchase[<id>]: per-rec deadline 30s exceeded log line fires so CloudWatch pinpoints which rec hung and what the elapsed time was, without requiring a second failure to reproduce. Closes part of issue #683. * feat(purchase/api): add approve handler entry/exit diagnostic logging Adds purchase[<exec-id>]: tagged timing logs at every stage of the approval path so CloudWatch can pinpoint exactly where a future hang occurs: - approvePurchaseViaSession: entry (auth=session) + elapsed on exit - ApproveExecution: entry (auth=token) + elapsed on exit for both the preflight-reject path and the full execute path - ApproveAndExecute: entry + status-transition elapsed + execute elapsed Actor emails are masked to ***@Domain before logging (PII rule). Part of issue #683 diagnostic logging extension. * feat(purchase): add timing logs to resolveAccountProvider and sub-resolvers Adds purchase[resolveAccountProvider/resolveAWSProvider/resolveAzureProvider/ resolveGCPProvider]: entry + elapsed-on-exit logs so CloudWatch can show whether a hang is in credential resolution (STS AssumeRole, OIDC token exchange) vs the cloud API call itself. Each log includes provider, account ID, auth mode, and elapsed time. Error paths also log before returning so a failed resolution is visible in the trace even if the caller swallows the error in an aggregator. Part of issue #683 diagnostic logging extension. * feat(providers): add STS GetCallerIdentity and Azure subscriptions timing logs Instruments the two network calls that ValidateCredentials makes during provider construction so CloudWatch distinguishes a credential-validation hang from the subsequent offering-search or purchase hang: - providers/aws/provider.go: purchase[STS]: GetCallerIdentity starting / returned in <elapsed> / failed after <elapsed> - providers/azure/provider.go: purchase[Azure]: subscriptions.NewListPager/ NextPage starting / returned in <elapsed> / failed after <elapsed> Both use the purchase[...]: prefix for grep-ability. Profile/subscription context is logged but no tokens, key material, or full identities. Part of issue #683 diagnostic logging extension. * feat(providers/aws/services): add per-page timing logs to findOfferingID in all 7 AWS service clients Adds purchase[<exec-id>]: tagged per-pagination-page timing logs to the findOfferingID loop in each of the 7 AWS purchase-path service clients (EC2, RDS, ElastiCache, MemoryDB, OpenSearch, Redshift, SavingsPlans). Each log line records: - Entry: service, instance/node type, term, payment option - Per page: page number, offering count returned, page elapsed - Hit: page number where match was found, total elapsed - Exhausted: total pages scanned, total elapsed (no-match path) The exec-id is threaded from PurchaseCommitment(opts.ExecutionID) into findOfferingID so all lines share the same purchase[<uuid>]: prefix and a single CloudWatch Logs Insights query can reconstruct the full pagination trace for any hanging execution. ValidateOffering and GetOfferingDetails callers pass "" (rendered as "no-exec") since they run outside the purchase flow. Part of issue #683 diagnostic logging extension. * feat(execution): add per-goroutine entry/exit timing in FanOutWithConcurrency Adds purchase[fan-out]: entry + elapsed-on-exit log lines to each goroutine launched by FanOutWithConcurrency. Both the account fan-out (multi-account execution) and the per-rec fan-out (processPurchaseRec- ommendations) go through FanOutWithConcurrency, so one addition covers both call sites. Logs: - purchase[fan-out]: goroutine starting for account=<id> - purchase[fan-out]: goroutine for account=<id> completed in <elapsed> - purchase[fan-out]: goroutine for account=<id> failed after <elapsed>: <err> Together with the per-rec ctx.WithTimeout and findOfferingID page logs, a CloudWatch Logs Insights query on purchase[ now reconstructs the full timing tree for a stuck execution: fan-out goroutine start -> credential resolution -> STS -> service client -> findOfferingID pagination -> PurchaseCommitment. Part of issue #683 diagnostic logging extension. * fix(purchase/tests): fix per-rec context.WithTimeout breaking mock matchers Move context.WithTimeout from the fan-out closure (processPurchaseRecommendations) into executeSinglePurchase where it belongs -- the budget governs the cloud API call chain, not the fan-out bookkeeping. Update all GetServiceClient and PurchaseCommitment mock.On() setups in execution_test.go, coverage_extra_test.go, and manager_test.go to match on deadline-carrying contexts via: mock.MatchedBy(func(c context.Context) bool { _, ok := c.Deadline(); return ok }) This positively asserts the 30s per-rec timeout is propagated to every cloud SDK call rather than silently accepting any context (mock.Anything). Also fix the savingsplans/client_test.go build error: findOfferingID gained an execID string parameter in commit 5 but the direct test call was not updated (only the PurchaseCommitment path was updated). All 180 purchase package tests and 308 AWS service tests now pass. Part of issue #683 diagnostic logging extension. * fix(purchase): address CodeRabbit review on PR #684 Three changes per CR pass-1 triage: 1. Actionable (execution.go): distinguish context.DeadlineExceeded from context.Canceled in the post-PurchaseCommitment error log. The previous branch logged "per-rec deadline 30s exceeded" for any non-nil recCtx.Err(), including parent cancellations -- mislabeling Canceled as a timeout makes diagnostic triage harder. Logic extracted into logRecCtxErr helper to keep executeSinglePurchase cyclomatic complexity within the gocyclo limit. 2. Nitpick (execution_test.go): replace mock.Anything with a deadline-aware mock.MatchedBy on the first arg of all 12 CreateAndValidateProvider expectations. The 30s per-rec budget is set before the factory call, so all three mocked stages (factory, GetServiceClient, PurchaseCommitment) now assert deadline propagation end-to-end. 3. Nitpick (purchasecfg/config.go + config_test.go): preserve the caller's *http.Client when one is provided in base config. NewConfig now shallow- copies the client and only overrides Timeout, keeping any custom Transport, Jar, or CheckRedirect. Falls back to a fresh client when base.HTTPClient is nil or a non-*http.Client implementation. Adds TestNewConfig_PreservesCustomTransport to pin the contract. * test(purchase): tighten ctx matcher to assert per-rec 30s budget (CR #684) CodeRabbit on PR #684 (review 4348606803) noted that the 53 mock matchers using `_, ok := c.Deadline(); return ok` only assert *some* deadline is set -- they would still pass if `executeSinglePurchase` silently reused the parent's longer deadline instead of its own `context.WithTimeout(ctx, 30*time.Second)` wrap. Tighten the contract: introduce `hasPerRecDeadline(max time.Duration)` in `execution_test.go` and replace all 53 sites across `execution_test.go`, `coverage_extra_test.go`, and `manager_test.go` with `mock.MatchedBy(hasPerRecDeadline(30*time.Second))`. The matcher now positively asserts the remaining deadline is in (0, 30s], so removing or loosening the wrap fails the test. * refactor(aws/services): extract findOfferingID helpers to fit gocyclo budget memorydb: extract scanMemoryDBOfferingPage (per-page offering scan with mismatch guard) and isLastMemoryDBPage (terminal-page check) from findOfferingID, mirroring the helpers already in the ec2 package. ec2: extract buildEC2QueryFromRec (details type-assertion + payment-option conversion + query assembly) from findOfferingID into a standalone method. Both findOfferingID functions now sit at cyclomatic complexity ≤10. No behaviour change -- pure extraction verified by 72 passing unit tests.
1 parent fd33b50 commit 30e8912

26 files changed

Lines changed: 676 additions & 185 deletions

File tree

‎internal/api/handler_purchases.go‎

Lines changed: 8 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -360,6 +360,9 @@ func (h *Handler) approvePurchase(ctx context.Context, req *events.LambdaFunctio
360360
// invocation drives its own executeAndFinalize, which already fans out
361361
// per-account in parallel via executeMultiAccount.
362362
func (h *Handler) approvePurchaseViaSession(ctx context.Context, req *events.LambdaFunctionURLRequest, execution *config.PurchaseExecution) (any, error) {
363+
t0 := time.Now()
364+
logging.Infof("purchase[%s]: approvePurchaseViaSession entry (auth=session)", execution.ExecutionID)
365+
363366
session, err := h.requireSession(ctx, req)
364367
if err != nil {
365368
return nil, err
@@ -376,12 +379,16 @@ func (h *Handler) approvePurchaseViaSession(ctx context.Context, req *events.Lam
376379
if err := h.purchase.ApproveAndExecute(ctx, execution.ExecutionID, session.Email); err != nil {
377380
// ApproveAndExecute returns either a transition error (the row
378381
// drifted out of pending/notified between our check and the UPDATE
379-
// — race with cancel/expire) or an execution error (AWS API failed,
382+
// -- race with cancel/expire) or an execution error (AWS API failed,
380383
// status is now "failed" on disk). Both surface as 409 to the
381384
// caller; the History view shows the resulting row state.
385+
logging.Errorf("purchase[%s]: approvePurchaseViaSession failed after %s: %v",
386+
execution.ExecutionID, time.Since(t0), err)
382387
return nil, NewClientError(409, fmt.Sprintf("execution %s could not be approved: %v", execution.ExecutionID, err))
383388
}
384389

390+
logging.Infof("purchase[%s]: approvePurchaseViaSession completed in %s (auth=session)",
391+
execution.ExecutionID, time.Since(t0))
385392
return map[string]string{"status": "completed"}, nil
386393
}
387394

‎internal/execution/fanout.go‎

Lines changed: 11 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -8,6 +8,7 @@ import (
88
"runtime"
99
"strconv"
1010
"sync"
11+
"time"
1112

1213
"github.com/LeanerCloud/CUDly/pkg/logging"
1314
)
@@ -92,7 +93,17 @@ func FanOutWithConcurrency[T any](
9293
}
9394
}
9495
}()
96+
t0 := time.Now()
97+
logging.Infof("purchase[fan-out]: goroutine starting for account=%s", accountID)
9598
val, err := fn(ctx, accountID)
99+
elapsed := time.Since(t0)
100+
if err != nil {
101+
logging.Errorf("purchase[fan-out]: goroutine for account=%s failed after %s: %v",
102+
accountID, elapsed, err)
103+
} else {
104+
logging.Infof("purchase[fan-out]: goroutine for account=%s completed in %s",
105+
accountID, elapsed)
106+
}
96107
results[idx] = Result[T]{AccountID: accountID, Value: val, Err: err}
97108
}(i, id)
98109
}

‎internal/purchase/approvals.go‎

Lines changed: 50 additions & 5 deletions
Original file line numberDiff line numberDiff line change
@@ -22,7 +22,8 @@ import (
2222
// NULL — the column is nullable TEXT and we don't want to claim "approved
2323
// by nobody".
2424
func (m *Manager) ApproveExecution(ctx context.Context, executionID, token, actor string) error {
25-
logging.Infof("Approving execution: %s", executionID)
25+
t0 := time.Now()
26+
logging.Infof("purchase[%s]: ApproveExecution entry (auth=token actor=%q)", executionID, maskActor(actor))
2627

2728
execution, err := m.config.GetExecutionByID(ctx, executionID)
2829
if err != nil {
@@ -50,10 +51,42 @@ func (m *Manager) ApproveExecution(ctx context.Context, executionID, token, acto
5051
// Preflight guard (issue #609): reject non-AWS orphan executions before
5152
// the cloud SDK is reached. See OrphanExecutionError for the full rationale.
5253
if err := OrphanExecutionError(execution); err != nil {
54+
logging.Errorf("purchase[%s]: ApproveExecution preflight rejected after %s: %v",
55+
executionID, time.Since(t0), err)
5356
return err
5457
}
5558

56-
return m.ApproveAndExecute(ctx, executionID, actor)
59+
err = m.ApproveAndExecute(ctx, executionID, actor)
60+
if err != nil {
61+
logging.Errorf("purchase[%s]: ApproveExecution (token path) failed after %s: %v",
62+
executionID, time.Since(t0), err)
63+
} else {
64+
logging.Infof("purchase[%s]: ApproveExecution (token path) completed in %s",
65+
executionID, time.Since(t0))
66+
}
67+
return err
68+
}
69+
70+
// maskActor masks an actor email/username for safe log emission.
71+
// Full email addresses are PII; log only the domain part.
72+
// Empty actor is logged as "<anon>" so the distinction between
73+
// "no actor provided" and "actor was empty string" is visible.
74+
func maskActor(actor string) string {
75+
if actor == "" {
76+
return "<anon>"
77+
}
78+
if at := len(actor) - 1; at >= 0 {
79+
for i, c := range actor {
80+
if c == '@' {
81+
return "***" + actor[i:]
82+
}
83+
}
84+
}
85+
// Not an email address -- log only the last 4 chars (e.g. a username).
86+
if len(actor) > 4 {
87+
return "..." + actor[len(actor)-4:]
88+
}
89+
return "****"
5790
}
5891

5992
// OrphanExecutionError returns a descriptive error when the execution has no
@@ -108,24 +141,36 @@ func OrphanExecutionError(execution *config.PurchaseExecution) error {
108141
// approval drives its own executeAndFinalize, which already fans out
109142
// per-account in parallel via executeMultiAccount.
110143
func (m *Manager) ApproveAndExecute(ctx context.Context, executionID, actor string) error {
144+
t0 := time.Now()
145+
logging.Infof("purchase[%s]: ApproveAndExecute starting (actor=%q)", executionID, maskActor(actor))
146+
111147
updated, err := m.config.TransitionExecutionStatus(ctx, executionID, []string{"pending", "notified"}, "approved")
112148
if err != nil {
149+
logging.Errorf("purchase[%s]: ApproveAndExecute status transition failed after %s: %v",
150+
executionID, time.Since(t0), err)
113151
return fmt.Errorf("approve: %w", err)
114152
}
153+
logging.Infof("purchase[%s]: status transitioned to approved in %s", executionID, time.Since(t0))
115154

116155
if actor != "" {
117156
a := actor
118157
updated.ApprovedBy = &a
119158
if saveErr := m.config.SavePurchaseExecution(ctx, updated); saveErr != nil {
120-
// Attribution is best-effort once the atomic flip has landed —
159+
// Attribution is best-effort once the atomic flip has landed --
121160
// dropping ApprovedBy must not stop the purchase from firing.
122161
// Log loudly so the audit gap is visible.
123162
logging.Errorf("AUDIT GAP: failed to stamp approved_by on %s: %v", executionID, saveErr)
124163
}
125164
}
126165

127-
logging.Infof("Execution %s approved, executing synchronously", executionID)
128-
return m.executeAndFinalize(ctx, updated)
166+
logging.Infof("purchase[%s]: executing purchase synchronously", executionID)
167+
execErr := m.executeAndFinalize(ctx, updated)
168+
if execErr != nil {
169+
logging.Errorf("purchase[%s]: ApproveAndExecute failed after %s: %v", executionID, time.Since(t0), execErr)
170+
} else {
171+
logging.Infof("purchase[%s]: ApproveAndExecute completed in %s", executionID, time.Since(t0))
172+
}
173+
return execErr
129174
}
130175

131176
// CancelExecution cancels a pending execution. actor carries the email of

‎internal/purchase/coverage_extra_test.go‎

Lines changed: 23 additions & 22 deletions
Original file line numberDiff line numberDiff line change
@@ -4,6 +4,7 @@ import (
44
"context"
55
"errors"
66
"testing"
7+
"time"
78

89
"github.com/LeanerCloud/CUDly/internal/config"
910
"github.com/LeanerCloud/CUDly/pkg/common"
@@ -285,7 +286,7 @@ func TestHandleExecutePurchase_SaveError(t *testing.T) {
285286
mockSTS.On("GetCallerIdentity", ctx, mock.Anything).Return(nil, errors.New("sts error"))
286287

287288
// Provider factory returns error → purchase fails
288-
mockFactory.On("CreateAndValidateProvider", ctx, "aws", mock.Anything).Return(nil, errors.New("provider error"))
289+
mockFactory.On("CreateAndValidateProvider", mock.Anything, "aws", mock.Anything).Return(nil, errors.New("provider error"))
289290

290291
manager := &Manager{
291292
config: mockStore,
@@ -582,7 +583,7 @@ func TestManager_ExecuteSinglePurchase_ProviderError(t *testing.T) {
582583
}
583584

584585
mockStore.On("GetPurchasePlan", ctx, "plan-prov-err").Return(plan, nil)
585-
mockFactory.On("CreateAndValidateProvider", ctx, "aws", mock.Anything).Return(nil, errors.New("provider unavailable"))
586+
mockFactory.On("CreateAndValidateProvider", mock.Anything, "aws", mock.Anything).Return(nil, errors.New("provider unavailable"))
586587
mockSTS.On("GetCallerIdentity", ctx, mock.Anything).Return(nil, errors.New("sts error"))
587588

588589
manager := &Manager{
@@ -630,8 +631,8 @@ func TestManager_ExecuteSinglePurchase_ServiceClientError(t *testing.T) {
630631
}
631632

632633
mockStore.On("GetPurchasePlan", ctx, "plan-svc-err").Return(plan, nil)
633-
mockFactory.On("CreateAndValidateProvider", ctx, "aws", mock.Anything).Return(mockProvider, nil)
634-
mockProvider.On("GetServiceClient", ctx, common.ServiceRDS, "eu-west-1").Return(nil, errors.New("service client error"))
634+
mockFactory.On("CreateAndValidateProvider", mock.Anything, "aws", mock.Anything).Return(mockProvider, nil)
635+
mockProvider.On("GetServiceClient", mock.MatchedBy(hasPerRecDeadline(30*time.Second)), common.ServiceRDS, "eu-west-1").Return(nil, errors.New("service client error"))
635636
mockSTS.On("GetCallerIdentity", ctx, mock.Anything).Return(nil, errors.New("sts error"))
636637

637638
manager := &Manager{
@@ -679,9 +680,9 @@ func TestManager_ExecuteSinglePurchase_PurchaseNotSuccessful(t *testing.T) {
679680
}
680681

681682
mockStore.On("GetPurchasePlan", ctx, "plan-not-success").Return(plan, nil)
682-
mockFactory.On("CreateAndValidateProvider", ctx, "aws", mock.Anything).Return(mockProvider, nil)
683-
mockProvider.On("GetServiceClient", ctx, common.ServiceElastiCache, "ap-southeast-1").Return(mockServiceClient, nil)
684-
mockServiceClient.On("PurchaseCommitment", ctx, mock.AnythingOfType("common.Recommendation"), mock.AnythingOfType("common.PurchaseOptions")).Return(
683+
mockFactory.On("CreateAndValidateProvider", mock.Anything, "aws", mock.Anything).Return(mockProvider, nil)
684+
mockProvider.On("GetServiceClient", mock.MatchedBy(hasPerRecDeadline(30*time.Second)), common.ServiceElastiCache, "ap-southeast-1").Return(mockServiceClient, nil)
685+
mockServiceClient.On("PurchaseCommitment", mock.MatchedBy(hasPerRecDeadline(30*time.Second)), mock.AnythingOfType("common.Recommendation"), mock.AnythingOfType("common.PurchaseOptions")).Return(
685686
common.PurchaseResult{Success: false}, nil,
686687
)
687688
mockSTS.On("GetCallerIdentity", ctx, mock.Anything).Return(nil, errors.New("sts error"))
@@ -732,9 +733,9 @@ func TestManager_ExecuteSinglePurchase_PurchaseNotSuccessful_WithError(t *testin
732733

733734
specificErr := errors.New("capacity limit exceeded")
734735
mockStore.On("GetPurchasePlan", ctx, "plan-err-result").Return(plan, nil)
735-
mockFactory.On("CreateAndValidateProvider", ctx, "aws", mock.Anything).Return(mockProvider, nil)
736-
mockProvider.On("GetServiceClient", ctx, common.ServiceOpenSearch, "us-west-2").Return(mockServiceClient, nil)
737-
mockServiceClient.On("PurchaseCommitment", ctx, mock.AnythingOfType("common.Recommendation"), mock.AnythingOfType("common.PurchaseOptions")).Return(
736+
mockFactory.On("CreateAndValidateProvider", mock.Anything, "aws", mock.Anything).Return(mockProvider, nil)
737+
mockProvider.On("GetServiceClient", mock.MatchedBy(hasPerRecDeadline(30*time.Second)), common.ServiceOpenSearch, "us-west-2").Return(mockServiceClient, nil)
738+
mockServiceClient.On("PurchaseCommitment", mock.MatchedBy(hasPerRecDeadline(30*time.Second)), mock.AnythingOfType("common.Recommendation"), mock.AnythingOfType("common.PurchaseOptions")).Return(
738739
common.PurchaseResult{Success: false, Error: specificErr}, nil,
739740
)
740741
mockSTS.On("GetCallerIdentity", ctx, mock.Anything).Return(nil, errors.New("sts error"))
@@ -788,9 +789,9 @@ func TestManager_ExecuteSinglePurchase_WithEngine(t *testing.T) {
788789
mockStore.On("GetPurchasePlan", ctx, "plan-engine").Return(plan, nil)
789790
mockStore.On("SavePurchaseHistory", ctx, mock.AnythingOfType("*config.PurchaseHistoryRecord")).Return(nil)
790791
mockEmail.On("SendPurchaseConfirmation", ctx, mock.AnythingOfType("email.NotificationData")).Return(nil)
791-
mockFactory.On("CreateAndValidateProvider", ctx, "aws", mock.Anything).Return(mockProvider, nil)
792-
mockProvider.On("GetServiceClient", ctx, common.ServiceRDS, "us-east-1").Return(mockServiceClient, nil)
793-
mockServiceClient.On("PurchaseCommitment", ctx, mock.AnythingOfType("common.Recommendation"), mock.AnythingOfType("common.PurchaseOptions")).Return(
792+
mockFactory.On("CreateAndValidateProvider", mock.Anything, "aws", mock.Anything).Return(mockProvider, nil)
793+
mockProvider.On("GetServiceClient", mock.MatchedBy(hasPerRecDeadline(30*time.Second)), common.ServiceRDS, "us-east-1").Return(mockServiceClient, nil)
794+
mockServiceClient.On("PurchaseCommitment", mock.MatchedBy(hasPerRecDeadline(30*time.Second)), mock.AnythingOfType("common.Recommendation"), mock.AnythingOfType("common.PurchaseOptions")).Return(
794795
common.PurchaseResult{Success: true, CommitmentID: "ri-engine-001"}, nil,
795796
)
796797
mockSTS.On("GetCallerIdentity", ctx, mock.Anything).Return(nil, errors.New("sts error"))
@@ -845,9 +846,9 @@ func TestManager_SavePurchaseHistory_Error(t *testing.T) {
845846
// SavePurchaseHistory returns error — should be logged but not fail executePurchase
846847
mockStore.On("SavePurchaseHistory", ctx, mock.AnythingOfType("*config.PurchaseHistoryRecord")).Return(errors.New("history write error"))
847848
mockEmail.On("SendPurchaseConfirmation", ctx, mock.AnythingOfType("email.NotificationData")).Return(nil)
848-
mockFactory.On("CreateAndValidateProvider", ctx, "aws", mock.Anything).Return(mockProvider, nil)
849-
mockProvider.On("GetServiceClient", ctx, common.ServiceEC2, "us-east-1").Return(mockServiceClient, nil)
850-
mockServiceClient.On("PurchaseCommitment", ctx, mock.AnythingOfType("common.Recommendation"), mock.AnythingOfType("common.PurchaseOptions")).Return(
849+
mockFactory.On("CreateAndValidateProvider", mock.Anything, "aws", mock.Anything).Return(mockProvider, nil)
850+
mockProvider.On("GetServiceClient", mock.MatchedBy(hasPerRecDeadline(30*time.Second)), common.ServiceEC2, "us-east-1").Return(mockServiceClient, nil)
851+
mockServiceClient.On("PurchaseCommitment", mock.MatchedBy(hasPerRecDeadline(30*time.Second)), mock.AnythingOfType("common.Recommendation"), mock.AnythingOfType("common.PurchaseOptions")).Return(
851852
common.PurchaseResult{Success: true, CommitmentID: "ri-hist-001"}, nil,
852853
)
853854
mockSTS.On("GetCallerIdentity", ctx, mock.Anything).Return(nil, errors.New("sts error"))
@@ -1083,11 +1084,11 @@ func TestManager_ExecuteSinglePurchase_DetailsByService(t *testing.T) {
10831084
mockStore.On("GetPurchasePlan", ctx, plan.ID).Return(plan, nil)
10841085
mockStore.On("SavePurchaseHistory", ctx, mock.AnythingOfType("*config.PurchaseHistoryRecord")).Return(nil)
10851086
mockEmail.On("SendPurchaseConfirmation", ctx, mock.AnythingOfType("email.NotificationData")).Return(nil)
1086-
mockFactory.On("CreateAndValidateProvider", ctx, "aws", mock.Anything).Return(mockProvider, nil)
1087-
mockProvider.On("GetServiceClient", ctx, tc.serviceType, tc.region).Return(mockServiceClient, nil)
1087+
mockFactory.On("CreateAndValidateProvider", mock.Anything, "aws", mock.Anything).Return(mockProvider, nil)
1088+
mockProvider.On("GetServiceClient", mock.MatchedBy(hasPerRecDeadline(30*time.Second)), tc.serviceType, tc.region).Return(mockServiceClient, nil)
10881089
mockServiceClient.On(
10891090
"PurchaseCommitment",
1090-
ctx,
1091+
mock.MatchedBy(hasPerRecDeadline(30*time.Second)),
10911092
mock.AnythingOfType("common.Recommendation"),
10921093
mock.AnythingOfType("common.PurchaseOptions"),
10931094
).Run(func(args mock.Arguments) {
@@ -1221,11 +1222,11 @@ func TestManager_ExecuteSinglePurchase_LegacyEmptyDetails(t *testing.T) {
12211222
mockStore.On("GetPurchasePlan", ctx, plan.ID).Return(plan, nil)
12221223
mockStore.On("SavePurchaseHistory", ctx, mock.AnythingOfType("*config.PurchaseHistoryRecord")).Return(nil)
12231224
mockEmail.On("SendPurchaseConfirmation", ctx, mock.AnythingOfType("email.NotificationData")).Return(nil)
1224-
mockFactory.On("CreateAndValidateProvider", ctx, "aws", mock.Anything).Return(mockProvider, nil)
1225-
mockProvider.On("GetServiceClient", ctx, tc.serviceType, tc.region).Return(mockServiceClient, nil)
1225+
mockFactory.On("CreateAndValidateProvider", mock.Anything, "aws", mock.Anything).Return(mockProvider, nil)
1226+
mockProvider.On("GetServiceClient", mock.MatchedBy(hasPerRecDeadline(30*time.Second)), tc.serviceType, tc.region).Return(mockServiceClient, nil)
12261227
mockServiceClient.On(
12271228
"PurchaseCommitment",
1228-
ctx,
1229+
mock.MatchedBy(hasPerRecDeadline(30*time.Second)),
12291230
mock.AnythingOfType("common.Recommendation"),
12301231
mock.AnythingOfType("common.PurchaseOptions"),
12311232
).Run(func(args mock.Arguments) {

0 commit comments

Comments
 (0)