Skip to content

feat: fail when global telemetry limits cannot be enforced#2

Merged
brandonwise merged 1 commit into
mainfrom
auto/pm-presence-20260324
Mar 25, 2026
Merged

feat: fail when global telemetry limits cannot be enforced#2
brandonwise merged 1 commit into
mainfrom
auto/pm-presence-20260324

Conversation

@brandonwise

Copy link
Copy Markdown
Owner

Summary

This PR tightens release safety: if global average telemetry limits are configured (max_avg_latency_ms / max_avg_cost_usd) but telemetry is missing, the gate now fails instead of silently passing.

Why

Teams are using agent quality gates to stop regressions in CI, but missing telemetry created a blind spot where configured global limits were never enforced.

What changed

  • Updated evaluator logic to fail when:
    • max_avg_latency_ms is set and no latency telemetry is present
    • max_avg_cost_usd is set and no cost telemetry is present
  • Added focused tests for both new fail-fast paths
  • Updated README to document telemetry-required behavior for global limits

Validation evidence

All checks run locally and passed.

  1. Repo-level/full test suite
  • Command: python3 -m pytest
  • Result: ✅ pass (7 passed in 0.04s)
  1. Targeted tests for changed module
  • Command: python3 -m pytest tests/test_evaluator.py -k 'global_latency_limit_requires_telemetry_when_configured or global_cost_limit_requires_telemetry_when_configured'
  • Result: ✅ pass (2 passed, 4 deselected in 0.01s)
  1. Lint/type/build checks
  • Command: python3 -m ruff check src tests
  • Result: ✅ pass (All checks passed!)
  • Command: python3 -m build
  • Result: ✅ pass (Successfully built ... .tar.gz and ... .whl)
  1. Smoke/integration check
  • Command: PYTHONPATH=src python3 -m agent_release_gate.cli evaluate --spec examples/spec.yaml --results examples/results.json
  • Result: ✅ pass (gate_passed=True, pass_rate=1.0)

Risk

Low. Change is scoped to gate decision logic for already-configured global limits.

@brandonwise brandonwise merged commit ea73ee5 into main Mar 25, 2026
2 checks passed
@brandonwise brandonwise deleted the auto/pm-presence-20260324 branch March 25, 2026 01:12
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant