Skip to content

test(live-smoke): cover the exempt_ips skip-list end to end - #145

Merged
rennf93 merged 1 commit into
masterfrom
feat/exempt-ips-live-smoke
Sep 26, 2026
Merged

rennf93 merged 1 commit into
masterfrom
feat/exempt-ips-live-smoke

Conversation

@rennf93

@rennf93 rennf93 commented Sep 26, 2026 •

Copy link
Copy Markdown
Owner

Summary

Adds the exempt_ips live-smoke scenario required before the next fastapi-guard release tracks the guard-core release that ships the field (guard-core PR #118). The single-client harness (nginx pins the resolved client IP to 192.168.50.50 for every request) expresses the maintainer's suggested exempt/non-exempt contrast through paired scenario configs:

  • exempt_ip_exceeds_the_rate_limit_without_being_throttled: with exempt_ips covering the client IP and rate_limit 2, four requests stay 200 across the crossing, the pipeline writes no rate-limit bucket for the exempt IP at all (smoke:rate_limit:rate:<ip> absent in Redis, pinning that the exempt skip sits ahead of the limiter), and a penetration payload from the exempt IP still gets 400 (exemption is not immunity)
  • the_same_limit_throttles_a_non_exempt_client: the identical limit without exempt_ips throttles at 429 and leaves exactly the bucket the exempt run never wrote
  • blacklist_beats_exemption: an IP that is both exempt and blacklisted gets 403

Compatibility note

The scenarios are red against PyPI guard-core below 4.1.0 by design: the older engine's pydantic model ignores the exempt_ips override key, so the no-throttle assertions fail until the guard-core release ships. This rides the documented lockstep hold pattern (v8.0.0 was held until guard-core 4.0.0 shipped), and the live smoke CI on this PR is expected red until then.

Test plan

  • verified live against a locally built guard-core 4.1.0 wheel: LIVE_SMOKE_ONLY_MODULES=exempt_ips pytest tests/live_smoke -m live_smoke green (3/3 scenarios, stack up/down clean)
  • full-module completeness check green with the new module loaded
  • ruff format/check and mypy clean on the new file

Summary by CodeRabbit

  • Tests
    • Added live checks confirming exempt IPs can exceed rate limits without throttling, while penetration detection still blocks suspicious requests.
    • Verified non-exempt clients are throttled at the configured limit and blacklisting still blocks exempt IPs.
    • Documented the smoke test results for guard-core 4.1.0 and earlier versions.

…the guard-core exempt_ips release

Three scenarios against the live stack: an exempt IP (the stack's fixed
client IP from nginx's X-Real-IP) exceeding rate_limit 2 keeps getting 200
across the crossing while the pipeline writes no rate-limit bucket for it
(smoke:rate_limit:rate:<ip> stays absent in Redis, pinning that the exempt
skip sits ahead of the limiter), a penetration payload from the exempt IP
still gets 400 (exemption is not immunity); the same limit without
exempt_ips throttles the identical sequence at 429 and leaves exactly the
bucket the exempt run never wrote (the paired configs express the
exempt/non-exempt contrast the single-client harness allows); and an IP
that is both exempt and blacklisted gets 403 (blacklist beats exemption).

Red against PyPI guard-core below 4.1.0 (the override key is ignored by
the older pydantic model, so the no-throttle assertions fail) by design:
this is release prep riding the documented lockstep hold. Verified live
against a locally built guard-core 4.1.0 wheel: LIVE_SMOKE_ONLY_MODULES=
exempt_ips green (3/3), full-module completeness check green.
@rennf93 rennf93 added the no-issue Chore or dependency PR that does not need an issue label Sep 26, 2026
@coderabbitai

coderabbitai Bot commented Sep 26, 2026 •

Copy link
Copy Markdown

Review in Change Stack →

Navigate logical layers of code changes, visualize relationships, and explore their blast radius.

No actionable comments were generated in the recent review. 🎉

ℹ️ Recent review info
⚙️ Run configuration

Configuration used: Organization UI

Review profile: ASSERTIVE

Plan: Advanced

Run ID: de8065d2-3a7e-457a-b595-0b61019caf02

📥 Commits

Reviewing files that changed from the base of the PR and between 4beb62e and 1c05fa7.

📒 Files selected for processing (2)
  • CHANGELOG.md
  • tests/live_smoke/scenarios/exempt_ips.py

Included review availability: This review used your included allowance. Your plan provides up to 4 included reviews per hour; 3 remain after this review.


📝 Walkthrough

Walkthrough

The change adds live smoke scenarios for exempt IP behavior, non-exempt throttling, penetration detection, and blacklist precedence. The changelog records the expected outcomes and the guard-core version used for the successful run.

Changes

Exempt IP smoke scenarios

Layer / File(s) Summary
Rate-limit exemption scenarios
tests/live_smoke/scenarios/exempt_ips.py, CHANGELOG.md
The scenarios check exempt and non-exempt rate-limit responses, Redis bucket state, penetration detection, and blacklist precedence. The changelog documents these checks and the reported successful run against a local guard-core 4.1.0 wheel.

Priority: ⬇️ Low

Estimated code review effort: 2 (Simple) | ~10 minutes

Change: Other

Merge Risk: ⚪ Minimal · up to 1c05f

The new smoke scenarios cover the stated exemption behavior. No merge-blocking issue is established; run the normal checks with guard-core 4.1.0.

Security Architecture Review

Security architecture risk: 🔵 Low · up to 1c05f

The scenarios do not add an application endpoint or change the rate-limit implementation. The separate response-header change has limited apparent security impact, but compatibility with the pending engine release remains unverified here.

Retained concerns
No architecture-level concerns identified.

Security review details

Security Blast Radius

  • inferred — The new scenario entrypoints are reachable through the test runner, not through a new application route. The separate adapter edit affects responses created by the guard response factory, but does not change its enforcement decisions.

Trust Boundaries and Controls

  • observed — The tests pin a proxy-supplied identity and assert that rate-limit exemption does not suppress detection or blacklist responses. They do not establish how a production deployment resolves client-IP headers.

Resilience and Maintainability Implications

  • observed — The existing harness clears smoke-prefixed Redis state on configuration-changing restarts and tears down the stack at session end. Same-configuration restarts do not clear that state, and isolation between concurrent smoke invocations is not established; no production-state effect from the new scenarios is shown.
🚥 Pre-merge checks | ✅ 4 | ❌ 1

❌ Failed checks (1 warning)

Check name Status Explanation Resolution
Docstring Coverage ⚠️ Warning Docstring coverage is 0.00% which is insufficient. The required threshold is 80.00%. Docstring coverage is scoped to functions touched by this diff. Analyzed 3 functions across 1 files. (1 skipped: 1 … Write docstrings for the functions missing them to satisfy the coverage threshold.
✅ Passed checks (4 passed)
Check name Status Explanation
Description Check ✅ Passed Check skipped - CodeRabbit’s high-level summary is enabled.
Title check ✅ Passed The title clearly and concisely describes the main change: adding end-to-end live-smoke coverage for the exempt_ips skip-list behavior.
Linked Issues check ✅ Passed Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check ✅ Passed Check skipped because no linked issues were found for this pull request.
Full details: Docstring Coverage

Explanation

Docstring coverage is 0.00% which is insufficient. The required threshold is 80.00%. Docstring coverage is scoped to functions touched by this diff. Analyzed 3 functions across 1 files. (1 skipped: 1 unsupported.)

  • Fix all pre-merge checks with AI
✨ Finishing Touches 💡 2
📝 Generate docstrings 💡
  • Commit to this branch
  • Create a new PR
🛠️ Fix failing CI checks 💡
  • Commit to this branch
  • Create a new PR

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands.

@github-actions github-actions Bot added documentation Docs, README, CHANGELOG, governance files tests Test suite changes labels Sep 26, 2026
@rennf93
rennf93 merged commit 8a9ce52 into master Sep 26, 2026
15 of 16 checks passed
@rennf93
rennf93 deleted the feat/exempt-ips-live-smoke branch September 27, 2026 18:26
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

documentation Docs, README, CHANGELOG, governance files no-issue Chore or dependency PR that does not need an issue tests Test suite changes

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant