Skip to content

feat(seer): Show out-of-credits failures - #129566

Open
billyvg wants to merge 8 commits into
masterfrom
billyvong/cw-2328-sentry-ui-show-out-of-credits-failures
Open

billyvg wants to merge 8 commits into
masterfrom
billyvong/cw-2328-sentry-ui-show-out-of-credits-failures

Conversation

@billyvg

@billyvg billyvg commented Nov 2, 2026 •

Copy link
Copy Markdown
Member

When an org runs out of Seer credits, Seer ends the run with failure_reason=out_of_credits (getsentry/seer#9242) or records pr_creation_error_code=out_of_credits on the repo's PR state (getsentry/seer#9268). Until now the UI showed those as generic failures ("This one is on us") next to a retry that fails the same way. This PR reads both fields, says the step stopped for credits, and makes its retry follow the credit check. It builds on #129545 (useSeerCreditCheck), #129551 (out-of-credits CTA), #129563 (agent chat gate), #129565 (autofix entry point gates) and #130463 (chat recheck), all merged.

This PR renders no out-of-credits CTA. The credit gates already show one on each surface: the alert at the top of the Autofix tab and drawer (its autofix-continue copy covers a run that stopped part way), the alert above the chat composer, and the muted alert in the issue sidebar. A stopped step or failed PR only gets a one-line status here.

A step that stopped for credits (root cause, plan, code changes, PR iteration):

  • The Seer drawer cards and the chat autofix card keep what the step wrote before it stopped: its reasoning, the root cause's evidence and, for code changes, the partial diff. Seer's own error block is left out. The drawer cards draw that work the way their running state does, like the Seer Agent chat. Below that, one line says which step stopped. It's muted in the drawer and danger in the chat card, which matches each card's existing failure text. Partial code changes can't be copied as Markdown or turned into a PR.
  • The issue sidebar previews say the step stopped for credits. A PR-iteration feedback item that stopped is marked failed, with out-of-credits copy.
  • In agent chat, the run's last response replaces Seer's plain-text error block with a muted "Seer stopped because your organization is out of credits." Earlier reasoning and tool calls stay in the thinking block. feat(seer): Show the credits notice when a chat runs out mid-conversation #130463 rechecks credits when a run stops for them, so the alert above the composer comes up as well.

A PR create or push that failed for credits:

  • The drawer's Pull Requests card keeps "Retry PR" for that repo and adds a line under the PRs saying it failed for credits. The chat PR card says the same instead of "This one is on us".
  • The chat failure toast and the PR links bar tooltip use out-of-credits copy instead of the raw backend string.

A step or PR the backend refuses for credits. The gates' credit checks fail open and can be out of date, so the Autofix endpoint has the final say: it answers 402 with code: seer_out_of_credits. startStep and createPR then recheck every Seer credit check, so the gates bring up their CTA and disable their buttons. They toast out-of-credits copy for the step, or for the repo's PR, instead of the backend's detail. A refused kickoff no longer paints a failed run that says "This one is on us", since the start card offers the CTA.

Before After
Start Analysis, refused for credits Before: the drawer goes blank, with no message and no CTA After: the toast says Seer couldn't start on the root cause, and the start CTA comes up with Start Analysis disabled
Yes, draft a PR, refused for credits Before: the toast shows the backend's message, and the buttons stay enabled with no CTA After: the toast says Seer couldn't open a pull request, and the continue CTA comes up with the buttons disabled

These use the real drawer and useExplorerAutofix, with the Autofix endpoint stubbed in the browser: it refuses with a 402 and code: seer_out_of_credits, and the credit check says the organization can pay until that refusal. Before is this branch without the commit that handles the refusal.

Retries follow the credit check. Each retry stays on screen. While the credit check says the org can't afford the operation, the retry is disabled, and its tooltip reads "Your organization does not have enough Seer credits for this." The check refetches on mount and window focus, so the retry turns back on once the org tops up. Until this mount's check answers, the retry stays disabled, even with an earlier answer cached, so it never flashes on. Only retries on screen for a credit failure run the check. Each retry checks its autofix action through #129565's useAutofixCreditCheck, so both gates read one action-to-operation map.

Retry Where Operation
Re-run, header Re-run step Root cause card autofix_root_cause
Re-run, header Re-run step Plan card autofix_solution
Re-run, header Re-run step Code changes card autofix_code_changes
Add context & retry, header Re-run step Code changes card, iteration stopped autofix_pr_iteration
Retry PR Pull Requests card, inbox preview autofix_open_pr
Try again Chat autofix card the step that stopped
Draft a pull request Chat code changes card, after a PR failed for credits autofix_open_pr

Decisions the spec doesn't cover, for review:

  • The spec has no failure copy. The status lines keep the earlier headlines, one sentence each, and the tooltip is new.
  • Without seer-credit-checks the hook reports enough credits, so a stopped step shows its retry enabled and the backend decides.
  • Only retries of something that stopped for credits are gated here. Retries after other failures, and re-runs of finished steps, don't call the credit check. The chat card's "Draft a pull request" is the exception: feat(seer): Gate autofix entry points on Seer credits #129565 already holds it on autofix_open_pr for every run, so this PR only adds the tooltip when the earlier create failed for credits.
  • On a drawer card that stopped for credits, the header Re-run step does what it does on a finished card: it opens the context prompt, or Seer chat where the card offers chat. On cards that failed for other reasons, it still shows nothing.
  • A stopped PR iteration is retried through the PR feedback form. Since ref(seer): Stop gating PR iteration UI on its feature flags #129036 dropped the autofix-pr-iteration-manual gate, that retry is always there once the run has a PR. hideErroredPrIteration still hides CI-driven iterations that fail, whether or not they ran out of credits.
  • The chat card's "Try again" sends a chat message, so the composer's agent_chat_message gate also applies to it. On a code changes card whose iteration stopped, it still asks Seer to retry the Code Changes step, as it does for other failures, but it waits for the autofix_pr_iteration check.
  • The PR links bar has no retry and no status line. Its tooltip is the only status, and the composer's alert carries the CTA.
Location Screenshot
Seer drawer › Code Changes, stopped for credits Seer drawer › Code Changes, stopped for credits
Seer drawer › Code Changes, stopped for credits, after a top-up Seer drawer › Code Changes, after a top-up
Seer drawer › Code Changes, PR iteration stopped for credits Seer drawer › Code Changes, PR iteration stopped for credits
Seer drawer › Pull Requests, failed for credits Seer drawer › Pull Requests, failed for credits
Seer Explorer › autofix card › Code Changes, stopped for credits Chat autofix card, stopped for credits
Seer Explorer › autofix card › Pull Request, failed for credits Chat autofix card, PR failed for credits
Seer Explorer › conversation, run stopped for credits Seer Explorer › conversation, run stopped for credits
Seer Explorer › PR links bar, failed for credits Seer Explorer › PR links bar, failed for credits

The screenshots come from the stories. Members see "Contact Admin" only when their org can self-serve, so the two captures with a CTA used a self-serve subscription.

Fixes CW-2328

🤖 Generated with Claude Code

@linear-code

linear-code Bot commented Nov 2, 2026

Copy link
Copy Markdown

CW-2328

@github-actions github-actions Bot added the Scope: Frontend Automatically applied to PRs that change frontend components label Nov 2, 2026
@github-actions

github-actions Bot commented Nov 2, 2026 •

Copy link
Copy Markdown
Contributor

📊 Type Coverage Diff

Metric Before After Delta
Coverage 96.03% 96.03% ±0%
Typed 142,321 142,430 🟢 +109
Untyped 5,890 5,893 🔴 +3
🔍 3 new type safety issues introduced

Non-null assertions (!) (1 new)

File Line Detail
static/app/components/seer/outOfCredits/messages.tsx 88 states[0]!

Type assertions (as) (2 new)

File Line Detail
static/app/components/events/autofix/useExplorerAutofix.tsx 343 `as
| null
| undefined` — `e as | {responseJSON?: {code?: unknown}; status?: number} | null | undefined` |

| static/app/components/seer/outOfCredits/__stories__/seerCreditCheckStub.tsx | 27 | as never — queryFn as never |

This is informational only and does not block the PR.

@billyvg
billyvg force-pushed the billyvong/cw-2321-sentry-ui-out-of-credits-cta-with-escalation branch from b2159ac to 53361ad Compare November 5, 2026 20:08
@billyvg
billyvg force-pushed the billyvong/cw-2328-sentry-ui-show-out-of-credits-failures branch from 33c3969 to 2c09c7d Compare November 5, 2026 20:14
@billyvg
billyvg force-pushed the billyvong/cw-2328-sentry-ui-show-out-of-credits-failures branch from 2c09c7d to 7d6e086 Compare November 5, 2026 20:37
@github-actions

github-actions Bot commented Nov 5, 2026 •

Copy link
Copy Markdown
Contributor

decision-model-linter

voice

static/app/components/events/autofix/v3/rootCauseCard.tsx

  • warn miscapitalizesProductName = TRUE (0.64): Use the canonical spelling for this named Sentry product or feature. Leave generic descriptions lowercase and preserve package names and identifiers.

static/app/components/events/autofix/v3/solutionCard.tsx

  • warn miscapitalizesProductName = TRUE (0.73): Use the canonical spelling for this named Sentry product or feature. Leave generic descriptions lowercase and preserve package names and identifiers.

decision-model-linter: 209 calls on 38 files, 545,564 input tokens, $0.023, model typesafe/jev-1.13-20260917

@sentry

sentry Bot commented Nov 5, 2026 •

Copy link
Copy Markdown
Contributor

Sentry Snapshot Testing

Name Added Removed Changed Renamed Unchanged Skipped Status
sentry-frontend
sentry-frontend
0 0 0 0 934 0 ✅ Unchanged

⚙️ sentry-frontend Snapshot Settings

@billyvg
billyvg force-pushed the billyvong/cw-2321-sentry-ui-out-of-credits-cta-with-escalation branch from d009d44 to dba69c6 Compare November 5, 2026 21:38
@billyvg
billyvg force-pushed the billyvong/cw-2328-sentry-ui-show-out-of-credits-failures branch from 34dc2ea to fa441db Compare November 5, 2026 21:50
@billyvg
billyvg force-pushed the billyvong/cw-2321-sentry-ui-out-of-credits-cta-with-escalation branch from dba69c6 to a43272f Compare November 5, 2026 22:03
@billyvg
billyvg force-pushed the billyvong/cw-2328-sentry-ui-show-out-of-credits-failures branch from fa441db to ef3a3c0 Compare November 5, 2026 22:08
Base automatically changed from billyvong/cw-2321-sentry-ui-out-of-credits-cta-with-escalation to master November 6, 2026 17:41
@billyvg
billyvg force-pushed the billyvong/cw-2328-sentry-ui-show-out-of-credits-failures branch from ef3a3c0 to b8e81cf Compare November 6, 2026 18:06
@billyvg
billyvg force-pushed the billyvong/cw-2328-sentry-ui-show-out-of-credits-failures branch from a1cfa65 to 942d7f5 Compare November 10, 2026 21:53

@srest2021 srest2021 left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can we add out-of-credits error handling to startStep and createPR in useExplorerAutofix.tsx as well (doesn't have to be this PR)? These still show the generic toast.

@billyvg
billyvg force-pushed the billyvong/cw-2328-sentry-ui-show-out-of-credits-failures branch from 942d7f5 to f7c5e66 Compare November 13, 2026 18:08
billyvg and others added 3 commits November 13, 2026 13:16
Runs and PR creates that stopped because the org ran out of Seer credits looked like generic errors and offered a retry that fails the same way. Autofix now reads failure_reason=out_of_credits and pr_creation_error_code=out_of_credits and shows the out-of-credits CTA instead.

The drawer cards and the chat autofix card keep what the step wrote before it stopped, including a partial diff, and drop their re-run and Try again buttons. The PR card and chat card stop offering a PR retry, and the inbox disables it with an explanation. The PR failure toast and tooltips use out-of-credits copy, and the chat swaps Seer's plain-text error block for the CTA.

Fixes CW-2328

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Steps and PRs that stopped because the org ran out of Seer credits no longer render the out-of-credits CTA. The credit gates own it on each surface: the alert at the top of the Autofix tab and drawer, the alert above the chat composer and the muted alert in the issue sidebar. A stopped step or failed PR now shows one muted (drawer) or danger (chat card) line saying why it stopped, and the chat transcript swaps Seer's error block for that line.

Their retries stay on screen and follow useSeerCreditCheck for the operation they would spend: disabled with an out-of-credits tooltip while the org can't afford it, enabled again once it can. That covers Re-run, Add context & retry and the card header's Re-run step in the drawer, Retry PR in the drawer and inbox, and Try again and Draft a pull request in the chat card.

Refs CW-2328

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Both cards gained an out-of-credits state with no story. Each story file shows the card stopped
for credits with Re-run disabled, the same run after a top-up, and a run that stopped before Seer
wrote anything.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
billyvg and others added 5 commits November 13, 2026 13:16
Credit checks also need token-billed-seer-enabled as of #129545, so the tests, stories and
SeerCreditCheckStub that turn on seer-credit-checks turn it on too.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The backend's SeerOperation calls the PR step autofix_open_pr and rejects autofix_create_pr, so
the PR retry gates always failed open.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The autofix embed rendered a stopped step's text with ArtifactPartialDetails, whose SeerMarkdown
imports the embeds, so knip --cycles failed. The embed now renders that text with its own
markdown, like the rest of its steps.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
#129565 landed `AUTOFIX_CREDIT_OPERATIONS` and `useAutofixCreditCheck`, which
map each autofix action to the operation the credit check prices. The
retries here kept their own copy of that map. `useOutOfCreditsRetry` now
takes the autofix action and checks it through `useAutofixCreditCheck`, so
both gates read one map.

The retry also waits on `hasCheckedSinceMount` instead of `isPending`, as the
#129565 gates do. A cached answer from an earlier mount no longer enables a
retry before this mount's recheck answers.

SeerExplorerContent reuses `isRunOutOfCredits`, which #130463 added, for the
transcript's out-of-credits line. The chat window story for a run that
stopped for credits now shows the composer's out-of-credits alert, which
that recheck brings up.

The chat window stories answered the credit usage meter's wallet request
with an empty list when they had no wallet fixture, so the meter crashed
and took every story in the file with it. Those requests now fail, as
they do for an org without token-billed Seer, and a story without a
wallet is viewed by a member without billing.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
When the organization has run out of Seer credits, the Autofix endpoint
refuses a step or PR with a 402 and `code: seer_out_of_credits`.
`startStep` and `createPR` treated that like any other failure. They
toasted the backend's detail, and a refused kickoff with no run yet
painted a failed run that said "This one is on us".

Both now recheck every Seer credit check, so the credit gates bring up the
out-of-credits CTA and disable their buttons, and toast out-of-credits
copy for the step, or for the repo's PR. A refused kickoff no longer
paints a failed run, since the start card offers the CTA instead.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@billyvg
billyvg force-pushed the billyvong/cw-2328-sentry-ui-show-out-of-credits-failures branch from f7c5e66 to 3b945d0 Compare November 13, 2026 18:17
@billyvg
billyvg marked this pull request as ready for review November 13, 2026 18:31
@billyvg
billyvg requested review from a team as code owners November 13, 2026 18:31

This branch was successfully deployed

1 active deployment
Preview — 3b945d04 Deployed Nov 13, 2026 by vercel[bot]
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Scope: Frontend Automatically applied to PRs that change frontend components

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants