Skip to content
Draft
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions MANIFEST.in
Original file line number Diff line number Diff line change
@@ -1,3 +1,4 @@
include LICENSE
include sentry_sdk/py.typed
include sentry_sdk/SETUP_CHECKLIST.md
graft tests
49 changes: 49 additions & 0 deletions sentry_sdk/SETUP_CHECKLIST.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,49 @@
# Sentry Python SDK: setup checklist

This is Sentry's official setup guidance, shipped in the `sentry-sdk` package and maintained by Sentry with the SDK. It matches the agent setup guide at https://docs.sentry.io/ai/agent-setup/get-started.md.

## Required

- **Errors.** This is Sentry's core product. Set it up unless the user asked you not to. It needs a DSN: if the user hasn't given you one and you can't get one yourself, ask for it. Read it from an environment variable (`SENTRY_DSN`) and don't commit it. Before you finish, send a test error and check that it arrives.

## Recommended

Turn these on unless the user asked you not to. Sentry links errors, traces and logs, so an error shows the request that caused it and the logs around it.

- **Tracing.** Shows which request caused an error and where the time went. Set `traces_sample_rate` by environment: `1.0` locally, so you can see a trace when you verify, and a low rate in production (`0.1` is a common start) unless the project already has one. Don't default it to `0`: a rate of `0` sends no traces at all.
- **Logs.** Set `enable_logs=True`. The app's logs show up next to the errors they belong to.
- **Environment and release.** Set `environment` so production issues are separate from local ones. The SDK reads the release from `SENTRY_RELEASE` or the git commit. If the app runs without `.git` (for example in a Docker image), set `SENTRY_RELEASE` at build time or pass the app's version as `release`, so you can see which deploy caused an issue.

To tell production from local runs, use a check the app already has, such as its debug flag or its environment setting. Don't add a new variable that nothing sets.

```python
import os
import sentry_sdk

from myapp import settings

is_production = not settings.DEBUG

sentry_sdk.init(
dsn=os.environ.get("SENTRY_DSN"),
environment="production" if is_production else "development",
traces_sample_rate=0.1 if is_production else 1.0,
enable_logs=True,
)
```

## Situational

Do the Required and Recommended items first. Then look at the ones below. Set them up only when the project matches.

- **Crons**, if the app schedules jobs with Celery beat, RQ (rq-scheduler), arq or Huey. Sentry alerts when a job fails or doesn't run, and every scheduled job gets a monitor, including ones added later. For Celery beat, pass `CeleryIntegration(monitor_beat_tasks=True)`. For RQ or arq, pass `monitor_cron_jobs=True` to `RqIntegration` or `ArqIntegration`. For Huey, pass `HueyIntegration(monitor_periodic_tasks=True)`.
- **AI monitoring**, if the app calls an LLM. Most AI libraries are traced automatically. For LiteLLM or Mistral, add `LiteLLMIntegration()` or `MistralIntegration()`. Prompts and responses are only recorded with `send_default_pii=True`, so ask the user before you set it.
- **Caches**, if the app uses the Django cache or Redis as a cache. Pass `cache_spans=True` to `DjangoIntegration` or `RedisIntegration`.
- **Profiling**, if the user cares about the performance of specific code paths. Set `profile_session_sample_rate` and `profile_lifecycle="trace"`.
- **Metrics**, if the user wants to track counts or values over time (signups, queue depth). Call `sentry_sdk.metrics.count(...)`, `gauge(...)` or `distribution(...)` where they happen.

## Before you finish

Tell the user what you set up, the trace sample rate you chose for each environment, whether logs are on, and which situational items you included and why.

Then set `agent_setup_complete=True` in `sentry_sdk.init`. That tells the SDK the setup has been checked, so it stops pointing coding agents to this file.
70 changes: 70 additions & 0 deletions sentry_sdk/client.py
Original file line number Diff line number Diff line change
@@ -1,3 +1,4 @@
import atexit
import json
import os
import platform
Expand Down Expand Up @@ -96,6 +97,20 @@

_client_init_debug = ContextVar("client_init_debug")

SETUP_CHECKLIST_PATH = os.path.join(os.path.dirname(__file__), "SETUP_CHECKLIST.md")

# Set by coding agents in the commands they run. AI_AGENT is the cross-tool
# convention; the rest cover agents that don't set it.
CODING_AGENT_ENV_VARS = (
"CLAUDECODE",
"CODEX_THREAD_ID",
"CODEX_SANDBOX",
"CURSOR_AGENT",
"GEMINI_CLI",
)

_coding_agent_hint_shown = False


SDK_INFO: "SDKInfo" = {
"name": "sentry.python", # SDK name will be overridden after integrations have been loaded with sentry_sdk.integrations.setup_integrations()
Expand Down Expand Up @@ -301,6 +316,58 @@ def _split_gen_ai_spans(
return non_gen_ai_spans, gen_ai_spans


def _running_under_coding_agent() -> bool:
if os.environ.get("CI"):
return False
if env_to_bool(os.environ.get("SENTRY_AGENT_HINTS"), strict=True) is False:
return False

ai_agent = os.environ.get("AI_AGENT")
if ai_agent:
return env_to_bool(ai_agent, strict=True) is not False

return any(os.environ.get(name) for name in CODING_AGENT_ENV_VARS)


_MISSING_PRODUCTS = {
(False, False): "Tracing and logs are off, so Sentry only gets errors.",
(False, True): "Tracing is off, so Sentry gets errors and logs but no traces.",
(True, False): "Logs are off, so Sentry gets errors and traces but no logs.",
}


def _write_coding_agent_hint(missing: str) -> None:
sys.stdout.flush()
sys.stderr.write(
f"[sentry] {missing} If you're setting up Sentry, it is strongly recommended "
"to check your setup against Sentry's official setup checklist, included in "
f"the sentry-sdk package you installed: {SETUP_CHECKLIST_PATH} "
"(hide with SENTRY_AGENT_HINTS=0)\n"
)
sys.stderr.flush()


def _show_coding_agent_hint(options: "Dict[str, Any]") -> None:
"""Point a coding agent at the setup checklist when tracing or logs are off."""
global _coding_agent_hint_shown

rate = options["traces_sample_rate"]
tracing_on = options["traces_sampler"] is not None or rate not in (0, None)
logs_on = bool(options["enable_logs"])
if (
_coding_agent_hint_shown
or not options["dsn"]
or options["agent_setup_complete"]
or (tracing_on and logs_on)
or not _running_under_coding_agent()
):
return

_coding_agent_hint_shown = True
# Agents often cut a command's output with `| tail`, so the hint goes last.
atexit.register(_write_coding_agent_hint, _MISSING_PRODUCTS[(tracing_on, logs_on)])


def _get_options(*args: "Optional[str]", **kwargs: "Any") -> "Dict[str, Any]":
if args and (isinstance(args[0], (bytes, str)) or args[0] is None):
dsn: "Optional[str]" = args[0]
Expand Down Expand Up @@ -544,6 +611,9 @@ class _Client(BaseClient):

def __init__(self, *args: "Any", **kwargs: "Any") -> None:
super(_Client, self).__init__(options=get_options(*args, **kwargs))

with capture_internal_exceptions():
_show_coding_agent_hint(self.options)
self._init_impl()

def __getstate__(self) -> "Any":
Expand Down
5 changes: 5 additions & 0 deletions sentry_sdk/consts.py
Original file line number Diff line number Diff line change
Expand Up @@ -1458,6 +1458,7 @@ def __init__(
org_id: "Optional[str]" = None,
strict_trace_continuation: bool = False,
stream_gen_ai_spans: bool = True,
agent_setup_complete: bool = False,
) -> None:
"""Initialize the Sentry SDK with the given parameters. All parameters described here can be used in a call to `sentry_sdk.init()`.

Expand Down Expand Up @@ -1869,6 +1870,10 @@ def __init__(
function will be retained. If the function returns None, the log will
not be sent to Sentry.

:param agent_setup_complete: Set to True once the setup has been checked against
`SETUP_CHECKLIST.md` in this package. Coding agents running the app are then no
longer pointed at the checklist. Defaults to False.

:param trace_ignore_status_codes: An optional property that disables tracing for
HTTP requests with certain status codes.

Expand Down
2 changes: 1 addition & 1 deletion setup.py
Original file line number Diff line number Diff line change
Expand Up @@ -35,7 +35,7 @@ def get_file_text(file_name):
long_description_content_type="text/markdown",
packages=find_packages(exclude=("tests", "tests.*")),
# PEP 561
package_data={"sentry_sdk": ["py.typed"]},
package_data={"sentry_sdk": ["py.typed", "SETUP_CHECKLIST.md"]},
zip_safe=False,
license_expression="MIT",
python_requires=">=3.6",
Expand Down
3 changes: 3 additions & 0 deletions tests/conftest.py
Original file line number Diff line number Diff line change
Expand Up @@ -146,6 +146,9 @@

SENTRY_EVENT_SCHEMA = "./checkouts/data-schemas/relay/event.schema.json"

# The suite may itself run under a coding agent. The hint tests turn hints back on.
os.environ["SENTRY_AGENT_HINTS"] = "0"

if not os.path.isfile(SENTRY_EVENT_SCHEMA):
SENTRY_EVENT_SCHEMA = None
else:
Expand Down
107 changes: 107 additions & 0 deletions tests/test_client.py
Original file line number Diff line number Diff line change
Expand Up @@ -25,6 +25,7 @@
start_transaction,
)
from sentry_sdk._compat import PY38
from sentry_sdk.client import CODING_AGENT_ENV_VARS, SETUP_CHECKLIST_PATH
from sentry_sdk.consts import DEFAULT_MAX_BREADCRUMBS
from sentry_sdk.integrations.asyncio import AsyncioIntegration
from sentry_sdk.integrations.executing import ExecutingIntegration
Expand Down Expand Up @@ -1507,6 +1508,112 @@ def test_continuous_profiling_does_not_warn(sentry_init):
assert profiling_warnings == []


AGENT_HINT_DSN = "https://public@sentry.example.com/1"


@pytest.fixture
def coding_agent_hint(monkeypatch, capsys):
for name in ("CI", "SENTRY_AGENT_HINTS", "AI_AGENT", *CODING_AGENT_ENV_VARS):
monkeypatch.delenv(name, raising=False)
monkeypatch.setattr(sentry_sdk.client, "_coding_agent_hint_shown", False)
registered = []
monkeypatch.setattr(
sentry_sdk.client.atexit,
"register",
lambda hook, *args: registered.append((hook, args)),
)

def printed_at_exit():
for hook, args in registered:
hook(*args)
return capsys.readouterr().err

monkeypatch.printed_at_exit = printed_at_exit
return monkeypatch


@pytest.mark.parametrize(
"env, options",
[
({"AI_AGENT": "claude-code"}, {"traces_sample_rate": 0}),
({"CLAUDECODE": "1"}, {}),
({"CODEX_THREAD_ID": "abc"}, {"traces_sample_rate": 0.0}),
({"CODEX_SANDBOX": "seatbelt"}, {"traces_sample_rate": 0.1}),
({"CLAUDECODE": "1"}, {"enable_logs": True}),
],
)
def test_coding_agent_hint(sentry_init, coding_agent_hint, env, options):
for name, value in env.items():
coding_agent_hint.setenv(name, value)

sentry_init(dsn=AGENT_HINT_DSN, **options)

assert SETUP_CHECKLIST_PATH in coding_agent_hint.printed_at_exit()


@pytest.mark.parametrize(
"env, options",
[
({}, {"traces_sample_rate": 0}),
({"AI_AGENT": "claude-code"}, {"traces_sample_rate": 0.1, "enable_logs": True}),
(
{"AI_AGENT": "claude-code"},
{"traces_sampler": lambda _: 1.0, "enable_logs": True},
),
({"AI_AGENT": "claude-code"}, {"agent_setup_complete": True}),
({"AI_AGENT": "0"}, {"traces_sample_rate": 0}),
({"AI_AGENT": "claude-code", "CI": "true"}, {"traces_sample_rate": 0}),
({"CLAUDECODE": "1", "SENTRY_AGENT_HINTS": "0"}, {"traces_sample_rate": 0}),
({"AI_AGENT": "claude-code"}, {"dsn": None, "traces_sample_rate": 0}),
],
)
def test_coding_agent_hint_not_shown(sentry_init, coding_agent_hint, env, options):
for name, value in env.items():
coding_agent_hint.setenv(name, value)

sentry_init(**{"dsn": AGENT_HINT_DSN, **options})

assert "[sentry]" not in coding_agent_hint.printed_at_exit()


@pytest.mark.parametrize(
"options, missing",
[
({}, "Tracing and logs are off, so Sentry only gets errors."),
(
{"enable_logs": True},
"Tracing is off, so Sentry gets errors and logs but no traces.",
),
(
{"traces_sample_rate": 0.1},
"Logs are off, so Sentry gets errors and traces but no logs.",
),
],
)
def test_coding_agent_hint_names_what_is_off(
sentry_init, coding_agent_hint, options, missing
):
coding_agent_hint.setenv("CLAUDECODE", "1")

sentry_init(dsn=AGENT_HINT_DSN, **options)

assert coding_agent_hint.printed_at_exit().startswith(f"[sentry] {missing} ")


def test_coding_agent_hint_shown_once(sentry_init, coding_agent_hint):
coding_agent_hint.setenv("AI_AGENT", "claude-code")

sentry_init(dsn=AGENT_HINT_DSN, traces_sample_rate=0)
sentry_init(dsn=AGENT_HINT_DSN, traces_sample_rate=0)

assert coding_agent_hint.printed_at_exit().count(SETUP_CHECKLIST_PATH) == 1


def test_setup_checklist_ships_with_the_package():
with open(SETUP_CHECKLIST_PATH) as f:
assert "agent_setup_complete=True" in f.read()


def test_ignore_spans_warns_without_streaming(sentry_init):
with pytest.warns(UserWarning, match=r"`ignore_spans` parameter only works"):
sentry_init(ignore_spans=["/health"], trace_lifecycle="static")
Expand Down
Loading