feat(secrets): auto-extract API key from JSON-structured Secrets Manager secrets

[litianningdatadog]

Summary

• When DD_API_KEY_SECRET_ARN is set and the fetched secret is a JSON object, automatically extract the dd_api_key field as the API key
• Falls back to the raw secret string if the value is not valid JSON or the dd_api_key field is absent — preserving existing behavior for plain-string secrets
• No new environment variable introduced; the JSON key name is hardcoded as dd_api_key

Test Plan

• Unit tests cover JSON extraction (dd_api_key present), fallback to raw (key absent), and plain string secrets
• Verify with a real Secrets Manager secret in JSON format: {"dd_api_key": "<your-key>"}
• Verify plain-string secrets continue to work unchanged
• Integration test using test function, secret in Json and secret in plain text. The positive and negative tests all performed as expected.

[Copilot]

Pull request overview

This PR adds support for JSON-structured Secrets Manager secrets by introducing a new DD_API_KEY_SECRET_JSON_KEY environment variable. When set alongside DD_API_KEY_SECRET_ARN, the extension parses the secret value as JSON and extracts the named field as the API key, addressing the need for teams that store secrets in key/value format (e.g., {"apiKey": "<key>"}).

Changes:

• Added JSON extraction logic to decrypt_aws_sm() function that parses secrets as JSON when json_key is provided
• Added api_key_secret_json_key field to all three config layers (Config, EnvConfig, YamlConfig) with proper merge logic
• Added three unit tests covering successful extraction, missing key error handling, and backward compatibility with plain secrets
• Updated configuration examples in YAML and environment config tests

Reviewed changes

Copilot reviewed 4 out of 4 changed files in this pull request and generated 1 comment.

File	Description
bottlecap/src/secrets/decrypt.rs	Added JSON extraction logic with new json_key parameter; includes three new unit tests
bottlecap/src/config/mod.rs	Added api_key_secret_json_key field to Config struct and its Default impl
bottlecap/src/config/env.rs	Added api_key_secret_json_key field to EnvConfig with documentation; updated merge logic and test fixtures
bottlecap/src/config/yaml.rs	Added api_key_secret_json_key field to YamlConfig; updated merge logic, example YAML config, and test fixtures

[duncanista]

Honestly, not super happy about having a new env var just for this, we should talk as a team if this is something we want to actually support

[litianningdatadog]

Honestly, not super happy about having a new env var just for this, we should talk as a team if this is something we want to actually support

I think the user makes a valid point for below reasons:

• The AWS UI makes key-value support intuitive.
• Supporting key-value pairs would make secrets more versatile, rather than limiting them to a single purpose.

Without introducing a new key, I do not think there is a better candidate for this.