feat: support multiple SSM parameters for large runner matcher configs (#4790)

[edersonbrilhante]

PR Description

Implements support for splitting the runner matcher configuration across multiple SSM parameters.
PARAMETER_RUNNER_MATCHER_CONFIG_PATH can now accept multiple parameter paths separated by a colon (:).

This avoids SSM size limits for large configurations and improves scalability for environments with many runner types and labels.

Closes #4790

[npalm]

@edersonbrilhante nice contribution. I just tried the parameter split with the multi-runner example in this repo. By adding a few config and adding more labes. Result is I got this error, related to the added foreach

│
│   on ../../modules/webhook/webhook.tf line 27, in resource "aws_ssm_parameter" "runner_matcher_config":
│   27:   for_each = { for idx, val in local.matcher_chunks : idx => val }
│     ├────────────────
│     │ local.matcher_chunks will be known only after apply
│
│ The "for_each" map includes keys derived from resource attributes that cannot be determined until apply, and so Terraform cannot determine the full set of keys that will
│ identify the instances of this resource.
│
│ When working with unknown values in for_each, it's better to define the map keys statically in your configuration and place apply-time results only in the map values.
│
│ Alternatively, you could use the -target planning option to first apply only the resources that the for_each value depends on, and then apply a second time to fully
│ converge.

Wondering you you tested the change? I was not able to get it working right now.

[edersonbrilhante]

I did test it, but I forgot to include a large configuration and only tested the normal case. My bad. I found the same bug now.

[edersonbrilhante]

@npalm I just pushed my fix. I tested a normal config and a tested a really big one with 7 ssm resources and it worked fine now.

[edersonbrilhante]

@npalm Also, during this fix I found 1 bug created the last release.
Check this PR #4806

[npalm]

Thx for the updates, will not be able to able to get the PR merged in the next weeks. I will catch up end of the month. Sorry for tthe delay.

[edersonbrilhante]

No problem. :)

[Copilot]

Pull Request Overview

Implements support for splitting the runner matcher configuration across multiple SSM parameters to avoid size limits for large configurations. The primary change allows PARAMETER_RUNNER_MATCHER_CONFIG_PATH to accept multiple parameter paths separated by a colon.

• Added logic to split large runner matcher configurations into chunks based on SSM parameter tier limits
• Updated infrastructure to create multiple SSM parameters when needed and pass them as a list
• Modified configuration loader to handle multiple parameter paths and combine them at runtime

Reviewed Changes

Copilot reviewed 10 out of 10 changed files in this pull request and generated 2 comments.

Show a summary per file

File	Description
modules/webhook/webhook.tf	Implements chunking logic and creates multiple SSM parameters for large configs
modules/webhook/eventbridge/webhook.tf	Updates environment variable to join multiple parameter paths with colons
modules/webhook/eventbridge/variables.tf	Changes ssm_parameter_runner_matcher_config from object to list of objects
modules/webhook/eventbridge/dispatcher.tf	Updates IAM policy and environment variables to handle multiple parameters
modules/webhook/direct/webhook.tf	Updates environment variables to join multiple parameter paths with colons
modules/webhook/direct/variables.tf	Changes ssm_parameter_runner_matcher_config from object to list of objects
lambdas/functions/webhook/src/ConfigLoader.ts	Implements logic to load and combine configuration from multiple SSM parameters
lambdas/functions/webhook/src/ConfigLoader.test.ts	Adds test coverage for multi-parameter configuration loading

---

_{Tip: Customize your code reviews with copilot-instructions.md. Create the file or learn how to get started.}

[npalm]

@edersonbrilhante took a while but tested and reviewed the PR.

• I left some small comments, one permission needs to be dropped
• can you have a look on the copilot suggestions as well
• I have tested incrasing the matching resulting in a split. and down size back to a single matcher variable. All works like a hcarm.

[npalm]

Expect to have time later this week to re-review the PR. Thanks for the updates

[stuartp44]

Small feedback from me

change requested on generated files.

[npalm]

@edersonbrilhante would you have time to have a look at this issue #4901. It seems this changes is breaking the module in some caes.