TT-17624: Add GCP provider

[vladzabolotnyi]

Description

Ticket: https://tyktech.atlassian.net/browse/TT-17624

Related Issue

Motivation and Context

Test Coverage For This Change

Screenshots (if appropriate)

Types of changes

• Bug fix (non-breaking change which fixes an issue)
• New feature (non-breaking change which adds functionality)
• Breaking change (fix or feature that would cause existing functionality to change)
• Refactoring or add test (improvements in base code or adds test coverage to functionality)
• Documentation updates or improvements.

Checklist

• I have reviewed the guidelines for contributing to this repository.
• Make sure you are requesting to pull a topic/feature/bugfix branch (right side). If PRing from your fork, don't come from your master!
• Make sure you are making a pull request against our master branch (left side). Also, it would be best if you started your change off our latest master.
• My change requires a change to the documentation.
  • I have manually updated the README(s)/documentation accordingly.
  • If you've changed APIs, describe what needs to be updated in the documentation.
• I have updated the documentation accordingly.
• Modules and vendor dependencies have been updated; run go mod tidy && go mod vendor
• When updating library version must provide reason/explanation for this update.
• I have added tests to cover my changes.
• All new and existing tests passed.
• Check your code additions will not fail linting checks:
  • gofmt -s -w .
  • go vet ./...

[github-actions]

CLA Assistant Lite bot:
Thank you for your submission, we really appreciate it. Like many open-source projects, we ask that you sign our Contributor License Agreement before we can accept your contribution. You can sign the CLA by just posting a Pull Request Comment same as the below format.

---

I have read the CLA Document and I hereby sign the CLA

---

Vlad Zabolotnyi seems not to be a GitHub user. You need a GitHub account to be able to sign the CLA. If you have already a GitHub account, please add the email address used for this commit to your account.
_{You can retrigger this bot by commenting recheck in this Pull Request}

[probelabs]

This PR introduces a new provider for Google Cloud Platform (GCP) Secret Manager, enabling its use as a key-value store. The implementation is comprehensive, adding both read (Get) and write (Set) capabilities. It supports various authentication methods, including Application Default Credentials (ADC), explicit credentials (file/JSON), and service account impersonation. The provider also includes robust configuration validation, data integrity checks via CRC32C checksums, and detailed error mapping from gRPC status codes to the application's kv error types.

A key highlight of this PR is the extensive and hermetic test suite, which features a fake in-memory gRPC server to simulate the Secret Manager API. This allows for reliable testing of the provider's logic without actual network calls or GCP infrastructure.

Files Changed Analysis

• go.mod, go.sum: Updated to include new dependencies for the GCP client libraries (cloud.google.com/go/secretmanager, google.golang.org/api, etc.).
• kv/providers/gcp/gcp.go: The core implementation of the gcpProvider. It defines the Config struct, the NewFactory for instantiation and validation, and implements the kv.Provider, kv.Setter, kv.Initializer, kv.Closer, and kv.Timeouter interfaces.
• kv/providers/gcp/fake_test.go: A significant addition that implements a fake gRPC server for Secret Manager. This enables isolated, fast, and reliable testing of the provider's interactions with the GCP API.
• kv/providers/gcp/*_test.go: A comprehensive suite of new test files covering configuration parsing (gcp_test.go), client lifecycle (client_internal_test.go), Get method functionality (get_internal_test.go), Set method logic (set_internal_test.go), and internal helpers (gcp_internal_test.go).

Architecture & Impact Assessment

• What this PR accomplishes: It integrates GCP Secret Manager as a new, fully-featured backend for the kv storage abstraction, providing users with more deployment options for managing secrets and configuration.
• Key technical changes introduced:
  • A new gcpProvider with full read/write functionality.
  • The Set method correctly handles the creation of new secrets and the addition of new versions to existing ones, including tolerance for race conditions.
  • A fake gRPC server for hermetic testing, improving the overall quality and reliability of the test suite.
• Affected system components: This is an additive change encapsulated within the kv/providers/gcp package. It expands the capabilities of any system component that uses the kv.Provider interface by making a new storage backend available. Existing providers are unaffected.

graph TD
    subgraph Application
        Service[Service using KV Store]
    end

    subgraph "KV Abstraction"
        ProviderFactory[kv.ProviderFactory]
        ProviderInterface[kv.Provider]
    end

    subgraph "KV Providers"
        style GCPProvider fill:#d2e5ff,stroke:#333,stroke-width:2px
        GCPProvider[gcp.gcpProvider]
        ExistingProviders[...]
    end

    subgraph "External Services"
        GCPSecretManager[GCP Secret Manager API]
    end

    Service --> ProviderFactory
    ProviderFactory -- Creates --> ProviderInterface
    ProviderInterface -- Implemented by --> GCPProvider
    ProviderInterface -- Implemented by --> ExistingProviders
    GCPProvider -- Interacts with --> GCPSecretManager

Loading

Scope Discovery & Context Expansion

• The scope of this PR is to deliver a complete, production-ready GCP Secret Manager provider with both read and write capabilities. The implementation is not a partial stub but a fully functional component.
• The provider is designed with a focus on security and robustness, evidenced by the detailed configuration validation, support for multiple authentication patterns (including impersonation), and data integrity checks.
• The inclusion of a fake gRPC server in fake_test.go is a notable architectural decision. It allows for thorough testing of complex logic, such as the Set method's create-then-add-version flow and error handling, in a completely isolated environment.

Metadata

• Review Effort: 4 / 5
• Primary Label: feature

---

Powered by Visor from Probelabs

Last updated: 2026-07-20T14:26:10.604Z | Triggered by: pr_updated | Commit: a02cecd

💡 TIP: You can chat with Visor using /visor ask <your question>

[probelabs]

Security Issues (2)

Severity	Location	Issue
🔴 Critical	kv/providers/gcp/gcp.go:208-212	Service account impersonation is not correctly configured. The `Scopes` field in `impersonate.CredentialsConfig` is required to create a valid token source for impersonation, but it is missing. The code includes a `FIX` comment acknowledging this. Without scopes, the call to `impersonate.CredentialsTokenSource` will fail, rendering the impersonation feature non-functional and preventing a key security mechanism from being used. 💡 Suggestion Provide the necessary scopes for the Secret Manager API. The `https://www.googleapis.com/auth/cloud-platform` scope is a common choice, or the more specific `https://www.googleapis.com/auth/secretmanager` can be used to follow the principle of least privilege. 🔧 Suggested Fix cs := impersonate.CredentialsConfig{ TargetPrincipal: gp.cfg.ImpersonateServiceAccount, Delegates: gp.cfg.ImpersonateDelegates, Scopes: []string{"https://www.googleapis.com/auth/cloud-platform"}, }
🟡 Warning	kv/providers/gcp/gcp.go:174-176	The `validateExternalAccount` function is a stub and performs no validation. When using `external_account` credentials for Workload Identity Federation, the credential configuration contains fields that should be validated to prevent misconfiguration, as recommended by Google Cloud documentation. Accepting this credential type without validation increases the risk of runtime errors and difficult-to-diagnose authentication failures. 💡 Suggestion Implement validation for external account credentials according to Google Cloud's best practices. This should involve checking for the presence and format of key fields within the credential JSON to ensure it is a valid configuration for workload identity federation.

Security Issues (2)

Severity	Location	Issue
🔴 Critical	kv/providers/gcp/gcp.go:208-212	Service account impersonation is not correctly configured. The `Scopes` field in `impersonate.CredentialsConfig` is required to create a valid token source for impersonation, but it is missing. The code includes a `FIX` comment acknowledging this. Without scopes, the call to `impersonate.CredentialsTokenSource` will fail, rendering the impersonation feature non-functional and preventing a key security mechanism from being used. 💡 Suggestion Provide the necessary scopes for the Secret Manager API. The `https://www.googleapis.com/auth/cloud-platform` scope is a common choice, or the more specific `https://www.googleapis.com/auth/secretmanager` can be used to follow the principle of least privilege. 🔧 Suggested Fix cs := impersonate.CredentialsConfig{ TargetPrincipal: gp.cfg.ImpersonateServiceAccount, Delegates: gp.cfg.ImpersonateDelegates, Scopes: []string{"https://www.googleapis.com/auth/cloud-platform"}, }
🟡 Warning	kv/providers/gcp/gcp.go:174-176	The `validateExternalAccount` function is a stub and performs no validation. When using `external_account` credentials for Workload Identity Federation, the credential configuration contains fields that should be validated to prevent misconfiguration, as recommended by Google Cloud documentation. Accepting this credential type without validation increases the risk of runtime errors and difficult-to-diagnose authentication failures. 💡 Suggestion Implement validation for external account credentials according to Google Cloud's best practices. This should involve checking for the presence and format of key fields within the credential JSON to ensure it is a valid configuration for workload identity federation.

\n\n ### Architecture Issues (2)

Severity	Location	Issue
🟠 Error	kv/providers/gcp/gcp.go:218-222	Service account impersonation is not correctly configured. The `Scopes` field in `impersonate.CredentialsConfig` is required but is missing, which will cause any attempt to use the `impersonate_service_account` feature to fail at runtime. The developer's own comment `// FIX: Scopes are required. What are they?` highlights this. 💡 Suggestion Populate the `Scopes` field with the necessary scope for GCP Secret Manager, which is `https://www.googleapis.com/auth/cloud-platform`.
🟡 Warning	kv/providers/gcp/gcp.go:175-177	The validation for `external_account` credentials is not implemented, as indicated by the `TODO` comment. The `validateExternalAccount` function is a stub that always returns `nil`. This allows for a configuration path (`credentials_type: "external_account"`) that is advertised as supported but lacks validation, potentially leading to hard-to-diagnose runtime errors from misconfigured credentials. 💡 Suggestion Implement the validation logic for external account credentials as outlined in the GCP documentation referenced in the TODO. If the feature is not ready, consider disallowing the `external_account` credentials type until validation is in place to avoid user confusion.

✅ Performance Check Passed

No performance issues found – changes LGTM.

---

Powered by Visor from Probelabs

Last updated: 2026-07-20T14:25:08.304Z | Triggered by: pr_updated | Commit: a02cecd

💡 TIP: You can chat with Visor using /visor ask <your question>

[github-actions]

🚨 Jira Linter Failed

Commit: a02cecd
Failed at: 2026-07-20 14:24:10 UTC

The Jira linter failed to validate your PR. Please check the error details below:

🔍 Click to view error details

configuration error: Jira user email is required for API authentication

Next Steps

• Ensure your branch name contains a valid Jira ticket ID (e.g., ABC-123)
• Verify your PR title matches the branch's Jira ticket ID
• Check that the Jira ticket exists and is accessible

---

This comment will be automatically deleted once the linter passes.