Repository navigation
chore: use ForceRollback to find failed resources when using ContinueUpdateRollback with rollback --force - #2078
ShadowCat567 wants to merge 9 commits into
Conversation
ForceRollback to find failed resources to orphan when using ContinueUpdateRollbackForceRollback to find failed resource when using ContinueUpdateRollback with --force
ForceRollback to find failed resource when using ContinueUpdateRollback with --forceForceRollback to find failed resources when using ContinueUpdateRollback with --force
ForceRollback to find failed resources when using ContinueUpdateRollback with --forceForceRollback to find failed resources when using ContinueUpdateRollback with rollback --force
Dependency Review✅ No vulnerabilities or license issues or OpenSSF Scorecard issues found. |
Codecov Report✅ All modified and coverable lines are covered by tests. Additional details and impacted files@@ Coverage Diff @@
## main #2078 +/- ##
=======================================
Coverage 92.10% 92.10%
=======================================
Files 82 82
Lines 13189 13189
Branches 1880 1880
=======================================
Hits 12148 12148
Misses 1003 1003
Partials 38 38
Flags with carried forward coverage won't be shown. Click here to find out more. ☔ View full report in Codecov by Harness. 🚀 New features to boost your workflow:
|
Signed-off-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
There was a problem hiding this comment.
See the review summary comment for the overview; the notes below are inline.
| } | ||
|
|
||
| // Log which resources are skipped by `ContinueUpdateRollback`. | ||
| if (options.orphanFailedResources && monitor.skippedResources.length > 0) { | ||
| await this.ioHelper.defaults.warn( | ||
| `Rolled back stack ${deployName} (orphaning: ${skippedResourceIds(monitor.skippedResources).join(', ')})`, |
There was a problem hiding this comment.
🟡 Recommended — The Rolled back stack <name> (orphaning: …) warning is emitted inside the rollback loop right after monitor.stop(), before the success/failure outcome is known and before the possible continue (loop again) or throw. On the multi-iteration --force path an iteration that observed skipped resources but has not finished rolling back will announce a completed rollback and then loop again, possibly more than once; and if the rollback ultimately fails and the code throws, the user has already been told the stack rolled back. The past-tense wording asserts a completed, successful rollback that may not have happened — unlike the pre-call Continuing rollback of stack … message, which stays honestly present-progressive.
Suggested change: Emit the orphan-list message only once the rollback has actually succeeded — move it into the if (finalStackState.stackStatus.isRollbackSuccess || !stackErrorMessage) success branch so it prints with the real outcome and not on the continue/throw paths, or reword it to present-progressive to match the pre-call message.
| resourcesToSkip.length > 0 ? ` (orphaning: ${resourcesToSkip.join(', ')})` : ''}`); | ||
| await cfn.continueUpdateRollback({ | ||
| StackName: stackArn, | ||
| ClientRequestToken: randomUUID(), | ||
| RoleARN: executionRoleArn, | ||
| ResourcesToSkip: resourcesToSkip, | ||
| ForceRollback: options.orphanFailedResources, |
There was a problem hiding this comment.
🟡 Recommended — The continueUpdateRollback request now sends ForceRollback: options.orphanFailedResources (true on the --force path) together with ResourcesToSkip: resourcesToSkip, which is an empty array on that path. CloudFormation documents ForceRollback and ResourcesToSkip as mutually exclusive — specifying both in the same request returns a validation error (ContinueUpdateRollback). Since --force is now the only path that sets ForceRollback, if CloudFormation treats a present-but-empty ResourcesToSkip as specifying both, the entire cdk rollback --force flow breaks. In practice Query-protocol serialization of an empty list likely emits no members on the wire, so it most likely does not trip the error — but the unit test asserts ResourcesToSkip: [] against a client mock that does not enforce the service-side rule, so a green test proves nothing here.
Suggested change: Omit ResourcesToSkip entirely when forcing so the two inputs stay on provably separate paths, e.g.:
await cfn.continueUpdateRollback({
StackName: stackArn,
ClientRequestToken: randomUUID(),
RoleARN: executionRoleArn,
...(options.orphanFailedResources
? { ForceRollback: true }
: { ResourcesToSkip: resourcesToSkip }),
});…to auto-skip-on-rollback
There was a problem hiding this comment.
See the review summary comment for the overview; the notes below are inline.
| RoleARN: executionRoleArn, | ||
| ResourcesToSkip: resourcesToSkip, | ||
| ...(options.orphanFailedResources ? | ||
| { ForceRollback: true } : |
There was a problem hiding this comment.
🟡 Recommended — The removed code derived the skip list from the event poller and filtered it to top-level resources only (parentStackLogicalIds.length === 0). Sending ForceRollback: true instead delegates selection to CloudFormation, which skips every resource in UPDATE_FAILED, including those in nested stacks, and sets them to UPDATE_COMPLETE so they no longer match the stack template (ContinueUpdateRollback). So cdk rollback --force can now orphan nested-stack resources it never touched before — an observable behavior change under an existing command, landing under a chore: prefix.
Suggested change: Confirm the widened orphaning scope is intentional and call it out in the PR description and the user-facing rollback --force docs so the semantic change is not silent. --force is already an explicit destructive opt-in, so a feature flag is likely unnecessary, but the broadened behavior should be documented.
| */ | ||
| function skippedResourceIds(skipped: readonly SkippedResource[]) { | ||
| return skipped.map((r) => [...r.parentStackLogicalIds, r.logicalResourceId].join('.')); | ||
| } |
There was a problem hiding this comment.
⚪ Optional — skippedResourceIds() joins parentStackLogicalIds with the logical ID to produce Parent.Child qualification — the one piece of reporting logic specific to the nested-stack orphaning this PR enables. Both new tests exercise only top-level resources with parentStackLogicalIds: [], so the qualifier join is never asserted and a regression in the Parent.Child format would go uncaught.
Suggested change: Add one case with a non-empty parentStackLogicalIds to lock in the Parent.Child output format of skippedResourceIds().
Signed-off-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
CloudFormation recently released a new flag on
ContinueUpdateRollback,ForceRollback, which specifies whether CloudFormation should force a rollback to continue by skipping resources that are currently in the stateUpdateFailed. This is effectively the same behavior we have withcdk rollback --force. The initial implementation ofcdk rollback --forcewould use a poller to look for all the resources that failed to update while a deployment was happening and manually add them to the orphaned resources list. This flag onContinueUpdateRollbackallows us to offload this work to CloudFormation.Added some new monitoring behavior to the stack-event-poller so we can still print out which resources are going to be skipped for the user to see.
Had to update
@aws-sdkversion in yarn.lock to get a more recent version of the CloudFormation SDK for the new feature.By submitting this pull request, I confirm that my contribution is made under the terms of the Apache-2.0 license