Skip to content

fix(copilot): set TRUST_PROXY=1 so rate limits can't be bypassed with X-Forwarded-For - #1096

Open
diecoscai wants to merge 1 commit into
stagingfrom
fix/prediction-trust-proxy
Open

diecoscai wants to merge 1 commit into
stagingfrom
fix/prediction-trust-proxy

Conversation

@diecoscai

Copy link
Copy Markdown

Why

Anyone can bypass the per-chatflow rate limit on /api/v1/prediction by sending a different X-Forwarded-For on each request.

The limiter counts by req.ip. Since the Flowise 3.0.11 upgrade (merge 7a0d0a7), the server sets trust proxy from TRUST_PROXY, and when that variable is unset it uses true (packages/server/src/index.ts:181-193, from upstream FlowiseAI#4454). With true, Express takes the leftmost X-Forwarded-For entry as the client IP, and the client writes that entry. The AWS ALB appends the real IP at the right, it does not replace the header.

Before the upgrade, AAI set trust proxy from NUMBER_OF_PROXIES. The copilot manifest still sets NUMBER_OF_PROXIES: '1', but the code no longer reads it, and TRUST_PROXY is not set. express-rate-limit logs ERR_ERL_PERMISSIVE_TRUST_PROXY for exactly this.

Checked on 2026-11-11, one read-only request per host: GET /api/v1/ip with X-Forwarded-For: 6.6.6.6 returns {"ip":"6.6.6.6"} on api.staging.ias.theanswer.ai, api.ias.theanswer.ai and prod.studio.theanswer.ai.

What changes

One variable in copilot/flowise/manifest.yml: TRUST_PROXY: '1', so Express trusts exactly one proxy (the ALB) and takes the IP the ALB appended. No code change. TRUST_PROXY is the upstream setting (FlowiseAI#5226), so future upstream merges don't conflict.

Verification (local, real server)

Real server build, Postgres 16 + Redis 7, a local stand-in for the ALB in "append" mode, and a chatflow with apiConfig.rateLimit 2/60s. 500 = the limiter let the request through (the test flow is empty), 429 = limited.

Case TRUST_PROXY unset (today) TRUST_PROXY=1
/api/v1/ip with X-Forwarded-For: 6.6.6.6 6.6.6.6 simulated client IP
One client, rotating X-Forwarded-For, 4 calls 500 500 500 500 500 500 429 429
Control: one client, no rotation, 3 calls 500 500 429 500 500 429
Client 1 ×3, then client 2 500 500 429 500 500 500 429 500

Mutation testing (Stryker 8.7.1, local only) on index.ts:181-193, where each mutant booted the real server with TRUST_PROXY=1 and ran the checks above: 28 mutants, 10 killed, 18 survived. Every surviving mutant only changes the undefined / blank / 'true' / 'false' branches or widens the numeric check, which '1' never takes. Every mutant that changes what '1' resolves to was killed.

Conditions this depends on

  1. The ALB must not append the client port (routing.http.xff_client_port.enabled). With ip:port in the header the limit is bypassed again (reproduced locally). Today /api/v1/ip on IAS staging and prod returns the IP with no port.
  2. No public path may reach the container without the ALB. In the repo's IaC nothing opens the container port: the addon security group has no ingress, and the web frontend returns 404 on /api/v1/*. The live security groups still need a check with AWS access.

Not covered

AnswerAI prod (prod.studio.theanswer.ai) runs on Render behind Cloudflare, not on copilot, and has the same bypass. The hop count there is not documented and the Render staging service is suspended, so this PR does not touch render.yaml. That needs its own decision.

After merge

Deploy to staging.ias, then call /api/v1/ip with X-Forwarded-For: 6.6.6.6 three times: it must return your real IP with no port. Then deploy IAS prod and repeat on api.ias.theanswer.ai.

🤖 Generated with Claude Code

Since the Flowise 3.0.11 upgrade the server reads TRUST_PROXY, not
NUMBER_OF_PROXIES. Unset means trust proxy=true, which takes the leftmost
X-Forwarded-For entry and lets callers bypass per-chatflow rate limits.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@vercel

vercel Bot commented Nov 11, 2026

Copy link
Copy Markdown

The latest updates on your projects. Learn more about Vercel for GitHub.

Project Deployment Actions Updated
answerai-docs Building Building Preview Nov 11, 2026 4:32pm UTC
the-answerai Error Error Nov 11, 2026 4:32pm UTC

Request Review

This branch had an error being deployed

1 failed and 1 active deployments
Preview – answerai-docs — 19c960e7 Deployed Nov 11, 2026 by vercel[bot]
Preview – the-answerai — 19c960e7 Deployed Nov 11, 2026 by vercel[bot]
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant