[iankhou]

Reason for this change

An integ test's snapshot records the synthesized template, but not what was actually deployed: parameter and SSM values, conditions, Fn::If, references and attributes are only known after deployment. To check deployed resources against their intended configuration (for example in an AWS service emulator), we need that ground truth stored with each integ test.

Stack (review the earlier ones first):

1. fix(toolkit-lib): stricter, opt-in evaluation modes for EvaluateCloudFormationTemplate #2074: evaluator changes
2. feat(toolkit-lib): best-effort evaluation of a whole template (internal) #2075: best-effort evaluation of a whole template
3. feat(toolkit-lib): unstable evaluate action to resolve a stack's template against its deployed stack #2066: the unstable toolkit-lib evaluate action (now also returns each stack's id and physical resource ids)
4. This PR: integ-runner writes resolved snapshots

Description of changes

New unstable feature --unstable=resolved-snapshots. When enabled, after a test case's stacks are deployed and before they are destroyed, integ-runner evaluates the test stacks with toolkit-lib's evaluate action and writes <stackName>.resolved.json into the snapshot directory, next to <stackName>.template.json. The assertion stack is not evaluated.

The file holds the evaluated Resources and Outputs, the expressions that could not be evaluated, resources with account-wide settings (AWS::ApiGateway::Account), and nested stacks. It is normalized so it is the same for every deployment of the test, in any account or region:

• generated physical ids become ${LogicalId}; the name at the end of a URL or ARN physical id becomes ${LogicalId.Name} (e.g. a queue's name), and the generated parts of |-separated ids become ${LogicalId[n]}
• the stack id, account, region and partition become ${AWS::StackId}, ${AWS::AccountId}, ${AWS::Region} and ${AWS::Partition} (taken from the stack id, no extra calls)
• names the template chooses (a physical id that appears in the resource's own properties, such as StageName: prod) are kept

Example (a Lambda function and an alarm on an SQS queue):

"Handler886CB40B": { "Type": "AWS::Lambda::Function", "Properties": {
  "Environment": { "Variables": { "QUEUE_URL": "${JobsDF1CC2D4}", "STAGE": "prod" } },
  "Role": "arn:${AWS::Partition}:iam::${AWS::AccountId}:role/${HandlerServiceRoleFCDC14AE}", ... } },
"QueueDepthDC6C83BE": { "Type": "AWS::CloudWatch::Alarm", "Properties": {
  "Dimensions": [{ "Name": "QueueName", "Value": "${JobsDF1CC2D4.Name}" }], ... } }

The files are written after the snapshot is synthesized, which replaces the snapshot directory; a snapshot updated without deploying therefore drops them rather than leaving stale ones. Snapshot verification compares only the stack templates, so the files don't affect it.

Other changes: ICdk gets an evaluate method, implemented by the toolkit-lib engine (which enables toolkit-lib's unstable evaluate feature), and CdkTestApp.evaluate.

Known limitations (follow-ups)

• Values generated by a service that aren't any resource's physical id, such as an API Gateway RootResourceId used as a ParentId, are not normalized yet. The fix is for evaluate to report which expression produced each value, so the normalizer can put the expression back.
• Account-wide settings are listed, not normalized: their deployed value depends on what else is deployed in the account.

Describe any new or updated permissions being added

None beyond what evaluate reads: cloudformation:DescribeStacks, cloudformation:ListStackResources, cloudformation:GetTemplate, and Cloud Control GetResource for attributes it cannot derive.

Description of how you validated changes

• Unit tests for the normalizer, including two deployments of the same app in different accounts and regions normalizing to identical output, and runner tests (evaluated after deploy and before destroy, written to the snapshot directory; not evaluated without the flag).
• Ran a small integ test (SQS queue, Lambda function and role, S3 bucket, CloudWatch alarm, one assertion) twice against a test account with the flag: both runs succeeded and produced byte-identical .resolved.json files.
• Normalized the resolved templates of two real apps (about 2,000 values each), each deployed, torn down and redeployed: 2 and 7 values still differ, the API Gateway root resource ids above and one app's own non-deterministic synthesis (a random schedule).

Checklist

• This change contains a major version upgrade for a dependency and I confirm all breaking changes are addressed
  • Release notes for the new version:

---

By submitting this pull request, I confirm that my contribution is made under the terms of the Apache-2.0 license

[github-actions]

Dependency Review

✅ No vulnerabilities or license issues or OpenSSF Scorecard issues found.

Scanned Files

None

[codecov-commenter]

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 92.07%. Comparing base (536a6ae) to head (ad2314c).

Additional details and impacted files

@@                         Coverage Diff                          @@
##           feat/toolkit-lib-evaluate-action    #2087      +/-   ##
====================================================================
- Coverage                             92.11%   92.07%   -0.04%     
====================================================================
  Files                                    82       82              
  Lines                                 13194    13194              
  Branches                               1879     1878       -1     
====================================================================
- Hits                                  12153    12149       -4     
- Misses                                 1003     1007       +4     
  Partials                                 38       38              

Flag	Coverage Δ
suite.unit	92.07% <ø> (-0.04%)	⬇️

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:

• ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
• 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.