Skip to content

feat(seer): Report an unconnected SCM account or missing repo on lookups - #130316

Open
joseph-sentry wants to merge 1 commit into
joey--seer-report-no-scm-integrationfrom
joey--seer-report-unconnected-scm-account
Open

joseph-sentry wants to merge 1 commit into
joey--seer-report-no-scm-integrationfrom
joey--seer-report-unconnected-scm-account

Conversation

@joseph-sentry

@joseph-sentry joseph-sentry commented Nov 6, 2026 •

Copy link
Copy Markdown
Contributor

we're adding more types of failures on repo lookups:

  • missing repo
  • unconnected account (there's no integration that matches the provided SCM org)

this is so when the seer agent calls these tools with these misconfigurations we can offer accurate information to the user:

  • missing repo: are you sure your integration has access to this repo, or is the agent using the wrong slug
  • unconnected account: please make sure to install the app on this org, or again correct the agent's call arg

@github-actions github-actions Bot added the Scope: Backend Automatically applied to PRs that change backend components label Nov 6, 2026
@joseph-sentry joseph-sentry changed the title joey seer report unconnected scm account feat(seer): Report an unconnected SCM account or missing repo on lookups Nov 6, 2026
@joseph-sentry
joseph-sentry force-pushed the joey--seer-report-unconnected-scm-account branch from 407ea53 to b34660e Compare November 9, 2026 18:13
@joseph-sentry
joseph-sentry force-pushed the joey--seer-report-unconnected-scm-account branch from b34660e to f948379 Compare November 9, 2026 19:40
we're adding more types of failures on repo lookups:
- missing repo
- unconnected account (there's no integration that matches the provided SCM org)

this is so when the seer agent calls these tools with these misconfigurations we can offer accurate information to the user:
- missing repo: are you sure your integration has access to this repo, or is the agent using the wrong slug
- unconnected account: please make sure to install the app on this org, or again correct the agent's call arg
@joseph-sentry
joseph-sentry force-pushed the joey--seer-report-unconnected-scm-account branch from f948379 to 84b7d4b Compare November 12, 2026 20:04
@joseph-sentry
joseph-sentry removed this pull request from stack #130317 November 12, 2026 20:44
@joseph-sentry
joseph-sentry marked this pull request as ready for review November 12, 2026 20:45
@joseph-sentry
joseph-sentry requested a review from a team as a code owner November 12, 2026 20:45
Comment on lines +984 to +993
repo_segments = _name_segments(repo_full_name)
# A name with no owner can't be under any account, so there's nothing to say about it.
if len(repo_segments) < 2:
return None
under_connected_account = any(
_is_under_account(repo_segments, integration) for integration in integrations
)
return RepositoryDefinitionFailure(
failure_reason=(
"repo_not_in_sentry" if under_connected_account else "scm_account_not_connected"

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🤖 Classifier normalizes names more loosely than the lookup, so misses get wrong reasons

_name_segments strips whitespace around each / and casefolds. _find_repository does neither: it matches name exactly and compares a url suffix. So a lookup that misses only because of how the name is written still gets a confident failure reason, and the reason is wrong:

  • GitLab display name without the spaces. Take My Group/Sub/App, where Sentry stores My Group / Sub / App with URL path my-group/sub/app. _find_repository misses it, because Q(name=...) is exact and the URL doesn't end with that string. Here, ["my group", "sub", "app"] starts with the integration name My Group / Sub, so the response is repo_not_in_sentry, but the repo is in Sentry. This input is realistic. Seer's own _normalize_repo_full_name (seer/automation/explorer/tools/utils.py) says the LLM passes the stripped form, and that Seer only recovers from it when it already has an external_id hint.
  • Host-qualified name. Take github.com/acme-labs/app. url__iendswith matches the repo as a candidate, but the Python check rejects it. Then ["github.com", "acme-labs", "app"] doesn't start with acme-labs, so the response is scm_account_not_connected. The agent would then ask the user to install the app on an account where it's already installed, for a repo Sentry already has. That is the misleading advice this PR is meant to prevent.

Fix: normalize the requested name once and use the result for both the lookup and the classification.

  • Whitespace: _find_repository could also try Q(name__iexact=" / ".join(segments)), since GitLab's name_with_namespace always joins with /, and compare segment-normalized names in its Python check.
  • Hosts: strip a leading scheme, and a first segment equal to a connected integration's host, before the lookup.
  • Smaller fix: if that's out of scope here, make _repository_lookup_failure return None when the first segment is a connected integration's host. In that case the reason isn't actually known, and None matches the function's "when it's known" contract.

Written by an AI agent and included in this review by its author.

failure_reason=(
"repo_not_in_sentry" if under_connected_account else "scm_account_not_connected"
),
connected_accounts=[integration.name for integration in integrations],

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🤖 connected_accounts gives GitLab display names, duplicates and no provider

The description says this list should help the agent correct its call argument. As written, it doesn't help much:

  • GitLab names: for GitLab, integration.name is the group's display name (My Group / Sub). It isn't the path (my-group/sub), which is what get_repository_definition returns as owner on success and what a retry should pass. An agent that builds My Group / Sub/app from this list still misses in _find_repository.
  • Duplicates: an account appears once per integration. The new test at test_tools.py:3228 compares a set because the list holds acme-labs three times. GitHub and Cursor Origin connected to the same account is a plausible real setup.
  • No provider: with the provider dropped, GitHub's acme-labs and Cursor Origin's acme-labs look the same.

Fix: report the prefix the agent should pass: the URL path for GitLab, and integration.name elsewhere. Don't derive it from domain_name for Cursor Origin, whose path is codebase/<slug>. Then deduplicate the list, add the provider if you want it, and make the test assert the exact list rather than a set.

Written by an AI agent and included in this review by its author.

failure_reason=(
"repo_not_in_sentry" if under_connected_account else "scm_account_not_connected"
),
connected_accounts=[integration.name for integration in integrations],

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🤖 GitLab connected_accounts gives the agent a display name it can't reuse

connected_accounts lists integration.name. For GitLab that is the group's full_name, a display name like My Group / Sub (see gitlab/integration.py:799). The point of this field is to let the agent fix its slug and retry. The form it can retry with is the URL path (my-group/sub): RepositoryDefinitionResponse builds owner/name from get_repo_url_path, and _find_repository matches display names only by exact, case-sensitive name=. So My Group / Sub/app and my group / sub / app both still miss.

For GitHub, GHE and Cursor Origin, name already equals the path segment, so only GitLab is affected. You can't simply switch every provider to the path taken from domain_name, though. Cursor Origin's domain_name is https://cursor.com/codebase/<slug>, so the generic parse on L944 yields codebase/<slug> there. That also means the URL-path branch of _is_under_account never matches for Cursor Origin, which is classified through its name alone. Options:

  • report the group path for GitLab (metadata["domain_name"].partition("/")[2], the same parse GitlabIntegration.source_url_matches uses), or
  • report both forms, e.g. connected_accounts=[{"name": ..., "path": ...}], so the user-facing message can show the display name and the agent can retry with the path.

Seer doesn't read this field yet (no report_failure_reason callers on seer main), so the shape is still cheap to change.

Written by an AI agent and included in this review by its author.

Comment on lines +3211 to +3213
for repo_name in ("my-group/other/app", "my-group/app"):
result = self._lookup(self.organization, repo_name)
assert result.failure_reason == "scm_account_not_connected"

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🤖 Not-connected GitLab cases repeat test_failure_for_each_provider

These two cases don't catch any mutant that test_failure_for_each_provider misses. Its GitLab subtest already checks my-group/sub-2/app → scm_account_not_connected. That fails on the same wrong implementations these would catch: comparing only the first segment, dropping the last account segment, or a plain startswith on the joined strings. The two repo_not_in_sentry cases above them do add coverage (the display-name form tests .strip(), and nested/app tests deeper repos), so keep those and drop this loop.

The same applies to the final other-org/app assertion in test_failure_checks_every_connected_account (L3230–3231). That test earns its place with the first assertion, which fails if only one integration is checked.

Written by an AI agent and included in this review by its author.

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Scope: Backend Automatically applied to PRs that change backend components

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants