Skip to content

fix(gocd): deploy rust consumers and api on snuba ST - #8425

Merged
pbhandari merged 7 commits into
masterfrom
prajjwalbhandari/eap-738-st-snuba-deploys
Sep 2, 2026
Merged

fix(gocd): deploy rust consumers and api on snuba ST#8425
pbhandari merged 7 commits into
masterfrom
prajjwalbhandari/eap-738-st-snuba-deploys

Conversation

@pbhandari

@pbhandari pbhandari commented Sep 2, 2026

Copy link
Copy Markdown
Contributor

ST snuba primary deploys were not rolling rust consumers (or most python ones / snuba-api) because:

  • deploy-snuba-rs-st is a no-op (deploy-st-rs.sh) due to a poor decision we made
  • deploy-snuba-py-st uses deploy-st-py.sh, which only updates snuba + snuba-admin containers

That meant sentry-options schema changes never left SaaS, which is why ST (e.g. goldmansachs) crashlooped on an outdated allocation-policy schema.

This makes ST deploy-primary use the same scripts as SaaS (deploy-py.sh / deploy-rs.sh).

Fixes EAP-738

@pbhandari
pbhandari requested a review from a team as a code owner September 2, 2026 19:44
@linear-code

linear-code Bot commented Sep 2, 2026

Copy link
Copy Markdown

EAP-738

@phacops

phacops commented Sep 2, 2026

Copy link
Copy Markdown
Contributor

deploy-snuba-rs-st is a no-op by design (deploy-st-rs.sh)

You mispelled "bug".

deploy-snuba-py-st uses deploy-st-py.sh, which only updates snuba + snuba-admin containers

Because of exactly this.

Comment thread gocd/templates/pipelines/snuba-py.libsonnet
Comment thread gocd/templates/pipelines/snuba-py.libsonnet
@MeredithAnya

Copy link
Copy Markdown
Member

@pbhandari can we confirm the container names ? I think the --container-name="snuba" is in the st definitions but not in the other, I think the naming conventions might not be equal here

gocdtasks.script(importstr '../bash/deploy-rs.sh'),
gocdtasks.script(importstr '../bash/deploy-rs.sh'),
],
},

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Bug: The deploy-rs.sh script, now used for ST regions, attempts to update hardcoded container names that may not exist in ST deployments, likely causing deployment failures.
Severity: HIGH

Suggested Fix

Modify the deploy-rs.sh script to handle cases where a container does not exist in the target deployment. One approach is to check for a container's existence before attempting to update its image. Alternatively, if the underlying k8s-deploy tool supports it, use a flag to ignore missing containers gracefully.

Prompt for AI Agent
Review the code at the location below. A potential bug has been identified by an AI
agent. Verify if this is a real issue. If it is, propose a fix; if not, explain why it's
not valid.

Location: gocd/templates/pipelines/snuba-rs.libsonnet#L206

Potential issue: The pull request makes `deploy-rs.sh` the standard deployment script
for all regions, including single-tenant (ST) environments. This script contains a
hardcoded list of container names, some of which are specific to the SaaS environment
and do not exist in ST deployments. The script uses `k8s-deploy` to update container
images. If this tool behaves like standard Kubernetes commands (`kubectl`), it will fail
when it tries to update a container that is not present in the target deployment
manifest. This would cause deployments to ST regions to fail, blocking updates for those
customers.

Also affects:

  • gocd/templates/pipelines/snuba-py.libsonnet:226~226

Comment thread gocd/templates/bash/deploy-py.sh Outdated
Comment thread gocd/templates/bash/deploy-py.sh Outdated

@cursor cursor Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Cursor Bugbot has reviewed your changes and found 1 potential issue.

Fix All in Cursor

❌ Bugbot Autofix is OFF. To automatically fix reported issues with cloud agents, enable autofix in the Cursor dashboard.

Reviewed by Cursor Bugbot for commit 30f8033. Configure here.

Comment thread gocd/templates/bash/deploy-py.sh
Comment thread gocd/templates/bash/deploy-py.sh Outdated
@pbhandari
pbhandari merged commit ba69009 into master Sep 2, 2026
65 checks passed
@pbhandari
pbhandari deleted the prajjwalbhandari/eap-738-st-snuba-deploys branch September 2, 2026 20:41
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants