[sentry-junior]

This re-lands the network guard from #127944 on current master, in a mode that can't break anything. #127945, #127946, #127947, getsentry/getsentry#22583 and getsentry/getsentry#22585 fixed what it found last time. Without a guard, real network calls in tests will creep back in.

Merge order: this PR first, then getsentry/getsentry#23239, which turns the guard on for getsentry.

What it does

sentry.testutils.pytest.network_guard is the same plugin as in #127944. It patches the Python socket connect and resolver entry points and records each attempt to reach a host outside the runner, per test, with the target and a short origin stack. xdist workers send their records to the controller. The allowlist (loopback, the unspecified address, the local hostname, the live server bridge IP, the reporting DSN hosts), the low-overhead design, and the native-client gap (libpq, librdkafka, grpc core) are unchanged. #127944 has the measurements.

Changes from #127944:

• Off by default, also in CI. backend.yml sets report on master pushes and manual runs only. Pull requests run with the guard off and pay nothing.
• Nothing fails. The JSON report is uploaded as network-guard-<run>-<shard>, and the upload step is continue-on-error.
• New offenders show up in the job summary. Each shard appends a table of the tests that reached the network and aren't known offenders, with the attempt and its origin frame.
• Known offenders file. tests/network_guard_known_offenders.txt (relative to the pytest rootdir) takes test files, classes, or test nodeids. Their attempts are still recorded but listed separately, and strict mode doesn't block them, so strict can be turned on with the remaining debt exempted per test. The summary also lists entries that ran clean, so the file shrinks as tests get fixed. It starts empty.

Offenders on master today

20 tests in 9 files reach 8 hostnames. None of them are known offenders yet.

• slack.com: 9 test_on_agent_update_* tests in tests/sentry/seer/entrypoints/slack/test_slack.py. These make real HTTPS connections through slack_sdk.
• api.github.com and controlserver: 6 tests in tests/sentry/tasks/test_post_process.py (4), tests/sentry/tasks/test_commit_context.py and tests/sentry/issues/auto_source_code_config/test_process_event.py. These are lookups only, from safe_create_connection and get_control_silo_ip_address.
• example.com, new.com and other.com: 3 tests in tests/sentry/sentry_apps/api/endpoints/test_project_legacy_webhooks.py. is_safe_hostname looks these up while validating webhook URLs.
• abc: 1 test in tests/sentry/integrations/perforce/test_integration.py, a lookup from is_safe_hostname.
• testserver: 1 test in tests/sentry/middleware/test_notification_tracking.py, a lookup from proxy_cell_request.

Open decisions

• Fix these 20 tests, or list them as known offenders first. The Slack tests come first, because they make real connections.
• When to switch master to strict. Proposal: after a week or two of master runs with no new offenders. Also whether pull requests should run it then, since the guard has no measurable cost.
• Whether new offenders on master should notify someone, or only appear in the job summary.

via @joshuarli.

--

View Session [Sentry]

[linear-code]

DI-2858