Overview
The gateway currently applies rate limits at the principal level only. This means all backend routes are treated as part of the same quota, even though different routes may have different cost profiles and should be rate limited independently.
For example, a lightweight health/read endpoint and an expensive AI/RAG endpoint should not necessarily consume from the same rate-limit bucket.
Current behavior
Rate limiting is keyed by the authenticated principal ID:
rateLimitKey := principal.ID
As a result, requests from the same API key share one quota regardless of the requested path, route, or upstream backend service.
Expected behavior
Rate limits should be configurable per route, target backend, or endpoint group.
Example behavior:
/v1/ask may use a stricter policy because it triggers expensive backend work.
/health, /readyz, or lightweight read endpoints may be exempt or use a higher limit.
Requests to different upstream services should be able to consume separate quota buckets.
Rate-limit headers should reflect the policy applied to the matched route/backend.
Proposed implementation
Introduce route-aware or upstream-aware rate-limit keys, such as:
rateLimitKey := principal.ID + ":" + route.ID
or:
rateLimitKey := principal.ID + ":" + backend.Name
Add a routing/policy configuration layer that maps path patterns to policies:
routes:
path_prefix: /v1/ask
upstream: rag-api
rate_limit_policy: expensive-ai
path_prefix: /health
upstream: rag-api
rate_limit_policy: public-health
Acceptance criteria
Gateway can select a rate-limit policy based on path, route, or backend target.
Requests to different configured route groups can consume separate quota buckets.
Existing per-principal rate limiting remains supported as the default behavior.
Unit tests cover route-specific limits.
Integration tests verify that exhausting one route quota does not incorrectly exhaust another route quota.
Documentation explains how route-scoped policies are configured.
Overview
The gateway currently applies rate limits at the principal level only. This means all backend routes are treated as part of the same quota, even though different routes may have different cost profiles and should be rate limited independently.
For example, a lightweight health/read endpoint and an expensive AI/RAG endpoint should not necessarily consume from the same rate-limit bucket.
Current behavior
Rate limiting is keyed by the authenticated principal ID:
As a result, requests from the same API key share one quota regardless of the requested path, route, or upstream backend service.
Expected behavior
Rate limits should be configurable per route, target backend, or endpoint group.
Example behavior:
/v1/ask may use a stricter policy because it triggers expensive backend work.
/health, /readyz, or lightweight read endpoints may be exempt or use a higher limit.
Requests to different upstream services should be able to consume separate quota buckets.
Rate-limit headers should reflect the policy applied to the matched route/backend.
Proposed implementation
Introduce route-aware or upstream-aware rate-limit keys, such as:
rateLimitKey := principal.ID + ":" + route.IDor:
rateLimitKey := principal.ID + ":" + backend.NameAdd a routing/policy configuration layer that maps path patterns to policies:
Acceptance criteria
Gateway can select a rate-limit policy based on path, route, or backend target.
Requests to different configured route groups can consume separate quota buckets.
Existing per-principal rate limiting remains supported as the default behavior.
Unit tests cover route-specific limits.
Integration tests verify that exhausting one route quota does not incorrectly exhaust another route quota.
Documentation explains how route-scoped policies are configured.