Remove x-forwarded-client-cert header from request headers in proxy configurations by bharath-k1999 · Pull Request #23544 · hashicorp/consul

bharath-k1999 · 2026-05-11T05:03:11Z

Description

Consul configures inbound public HTTP listeners to parse the x-forwarded-client-cert header for internal service mesh metadata. Before this change, those listeners could forward the same header to the local service instance on the last hop.

This change keeps the header available for Envoy metadata processing, then strips x-forwarded-client-cert from the request before forwarding it to the local application. That prevents downstream workloads from receiving client certificate details that Consul only needs for proxy-side processing.

The change also updates the affected xDS listener snapshots and adds a focused unit test for the new header stripping behavior.

Testing & Reproduction steps

Reproduced the issue with mesh sanitizeXForwardedClientCert disabled and an unpatched Consul server image. Requests sent through the gateway reached the backend workload with an x-forwarded-client-cert header containing SPIFFE and certificate-chain details.

Verified the workaround path by enabling mesh sanitizeXForwardedClientCert and confirming the backend workload no longer received the header.

Verified this fix independently of the workaround by running a patched Consul server image with mesh sanitizeXForwardedClientCert=false, sending traffic through the gateway, and capturing the backend workload request headers. The captured request included expected proxy headers such as x-forwarded-proto and x-request-id, and did not include x-forwarded-client-cert.

Ran the following tests:

go test ./agent/xds -run Test_makeHTTPFilter_stripForwardClientCertHeader -count=1
go test ./agent/xds -run Test_makeHTTPFilter_maxRequestHeadersKb -count=1
go test ./agent/xds -run TestAllResourcesFromSnapshot -update -count=1
go test ./agent/xds -run TestAllResourcesFromSnapshot -count=1

Links

PR Checklist

updated test coverage
external facing docs updated
appropriate backport labels added
not a security concern

PCI review checklist

I have documented a clear reason for, and description of, the change I am making.
If applicable, I've documented a plan to revert these changes if they require more than reverting the pull request.
If applicable, I've documented the impact of any changes to security controls.

Examples of changes to security controls include using new access control methods, adding or removing logging pipelines, etc.

…onfigurations

…removal

Copilot

Pull request overview

This PR prevents inbound HTTP listeners from forwarding the x-forwarded-client-cert (XFCC) header to the local application by keeping it available for Envoy-side metadata processing and then stripping it before routing to local_app.

Changes:

Enable requestHeadersToRemove: ["x-forwarded-client-cert"] on generated inbound HTTP routes when XFCC parsing/forwarding is enabled.
Update xDS listener golden snapshots to reflect the new header removal behavior.
Add unit + integration test coverage to verify XFCC is not observable by the upstream echo workload.

Reviewed changes

Copilot reviewed 42 out of 42 changed files in this pull request and generated 2 comments.

Show a summary per file

File	Description
test/integration/connect/envoy/helpers.bash	Improves extraction of JSON from `curl -v` output used by integration tests.
test/integration/connect/envoy/case-l7-intentions-request-normalization/verify.bats	Adds integration assertion that XFCC is stripped before reaching the echo backend.
test/integration/connect/envoy/case-l7-intentions-request-normalization-disabled/verify.bats	Adds the same XFCC stripping assertion with request normalization disabled.
agent/xds/listeners.go	Adds a new listener filter option and applies `requestHeadersToRemove` on the generated route.
agent/xds/listeners_test.go	Adds a focused unit test ensuring the route contains `x-forwarded-client-cert` in `RequestHeadersToRemove` when enabled.
agent/xds/testdata/listeners/http2-public-listener.latest.golden	Updates snapshot to include `requestHeadersToRemove` for XFCC.
agent/xds/testdata/listeners/http-public-listener.latest.golden	Updates snapshot to include `requestHeadersToRemove` for XFCC.
agent/xds/testdata/listeners/http-listener-with-timeouts.latest.golden	Updates snapshot to include `requestHeadersToRemove` for XFCC.
agent/xds/testdata/listeners/custom-trace-listener.latest.golden	Updates snapshot to include `requestHeadersToRemove` for XFCC.
agent/xds/testdata/listeners/connect-proxy-with-mesh-config-request-normalization-all-envoy-options-enabled.latest.golden	Updates snapshot to include `requestHeadersToRemove` for XFCC.
agent/xds/testdata/listeners/connect-proxy-with-mesh-config-request-normalization-all-consul-options.latest.golden	Updates snapshot to include `requestHeadersToRemove` for XFCC.
agent/xds/testdata/listeners/connect-proxy-with-http-max-request-headers.latest.golden	Updates snapshot to include `requestHeadersToRemove` for XFCC.
agent/xds/testdata/builtin_extension/listeners/wasm-http-remote-file.latest.golden	Updates snapshot to include `requestHeadersToRemove` for XFCC.
agent/xds/testdata/builtin_extension/listeners/wasm-http-local-file.latest.golden	Updates snapshot to include `requestHeadersToRemove` for XFCC.
agent/xds/testdata/builtin_extension/listeners/tproxy-and-permissive-mtls-and-envoy-extension.latest.golden	Updates snapshot to include `requestHeadersToRemove` for XFCC.
agent/xds/testdata/builtin_extension/listeners/propertyoverride-remove-outlier-detection.latest.golden	Updates snapshot to include `requestHeadersToRemove` for XFCC.
agent/xds/testdata/builtin_extension/listeners/propertyoverride-patch-specific-upstream-service-splitter.latest.golden	Updates snapshot to include `requestHeadersToRemove` for XFCC.
agent/xds/testdata/builtin_extension/listeners/propertyoverride-patch-specific-upstream-service-failover.latest.golden	Updates snapshot to include `requestHeadersToRemove` for XFCC.
agent/xds/testdata/builtin_extension/listeners/propertyoverride-outbound-doesnt-apply-to-inbound.latest.golden	Updates snapshot to include `requestHeadersToRemove` for XFCC.
agent/xds/testdata/builtin_extension/listeners/propertyoverride-listener-outbound-add.latest.golden	Updates snapshot to include `requestHeadersToRemove` for XFCC.
agent/xds/testdata/builtin_extension/listeners/propertyoverride-listener-inbound-add.latest.golden	Updates snapshot to include `requestHeadersToRemove` for XFCC.
agent/xds/testdata/builtin_extension/listeners/propertyoverride-inbound-doesnt-apply-to-outbound.latest.golden	Updates snapshot to include `requestHeadersToRemove` for XFCC.
agent/xds/testdata/builtin_extension/listeners/propertyoverride-cluster-load-assignment-outbound-add.latest.golden	Updates snapshot to include `requestHeadersToRemove` for XFCC.
agent/xds/testdata/builtin_extension/listeners/propertyoverride-cluster-load-assignment-inbound-add.latest.golden	Updates snapshot to include `requestHeadersToRemove` for XFCC.
agent/xds/testdata/builtin_extension/listeners/propertyoverride-add-round-robin-lb-config.latest.golden	Updates snapshot to include `requestHeadersToRemove` for XFCC.
agent/xds/testdata/builtin_extension/listeners/propertyoverride-add-outlier-detection.latest.golden	Updates snapshot to include `requestHeadersToRemove` for XFCC.
agent/xds/testdata/builtin_extension/listeners/propertyoverride-add-outlier-detection-multiple.latest.golden	Updates snapshot to include `requestHeadersToRemove` for XFCC.
agent/xds/testdata/builtin_extension/listeners/propertyoverride-add-keepalive.latest.golden	Updates snapshot to include `requestHeadersToRemove` for XFCC.
agent/xds/testdata/builtin_extension/listeners/otel-access-logging-http.latest.golden	Updates snapshot to include `requestHeadersToRemove` for XFCC.
agent/xds/testdata/builtin_extension/listeners/lua-outbound-doesnt-apply-to-local-upstreams-with-envoy-constraint-violation.latest.golden	Updates snapshot to include `requestHeadersToRemove` for XFCC.
agent/xds/testdata/builtin_extension/listeners/lua-outbound-doesnt-apply-to-local-upstreams-with-consul-constraint-violation.latest.golden	Updates snapshot to include `requestHeadersToRemove` for XFCC.
agent/xds/testdata/builtin_extension/listeners/lua-outbound-doesnt-apply-to-inbound.latest.golden	Updates snapshot to include `requestHeadersToRemove` for XFCC.
agent/xds/testdata/builtin_extension/listeners/lua-outbound-applies-to-local-upstreams.latest.golden	Updates snapshot to include `requestHeadersToRemove` for XFCC.
agent/xds/testdata/builtin_extension/listeners/lua-outbound-applies-to-local-upstreams-tproxy.latest.golden	Updates snapshot to include `requestHeadersToRemove` for XFCC.
agent/xds/testdata/builtin_extension/listeners/lua-inbound-doesnt-apply-to-local-upstreams.latest.golden	Updates snapshot to include `requestHeadersToRemove` for XFCC.
agent/xds/testdata/builtin_extension/listeners/lua-inbound-applies-to-inbound.latest.golden	Updates snapshot to include `requestHeadersToRemove` for XFCC.
agent/xds/testdata/builtin_extension/listeners/lambda-and-lua-connect-proxy.latest.golden	Updates snapshot to include `requestHeadersToRemove` for XFCC.
agent/xds/testdata/builtin_extension/listeners/ext-authz-http-upstream-http-service.latest.golden	Updates snapshot to include `requestHeadersToRemove` for XFCC.
agent/xds/testdata/builtin_extension/listeners/ext-authz-http-upstream-grpc-service.latest.golden	Updates snapshot to include `requestHeadersToRemove` for XFCC.
agent/xds/testdata/builtin_extension/listeners/ext-authz-http-local-http-service.latest.golden	Updates snapshot to include `requestHeadersToRemove` for XFCC.
agent/xds/testdata/builtin_extension/listeners/ext-authz-http-local-grpc-service.latest.golden	Updates snapshot to include `requestHeadersToRemove` for XFCC.
.changelog/23544.txt	Adds a release note describing the inbound XFCC stripping behavior change.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

+  # The first sed drops curl verbose lines that start with '*'.
+  # The second sed converts a trailing '}* <some text...>' fragment to just '}'.
+  local json=$(echo "$output" | sed '/^\*/d' | sed 's/}\*.*/}/' | sed -n -e '/^{$/,/^}$/{ p; }')
  echo $json | jq -r '.' || echo "Output did not contain valid JSON: $output" >&3


+		if opts.stripForwardClientCertHeader {
+			route.RequestHeadersToRemove = append(route.RequestHeadersToRemove, "x-forwarded-client-cert")
+		}


Remove x-forwarded-client-cert header from request headers in proxy c…

e322e5d

…onfigurations

bharath-k1999 requested review from a team as code owners May 11, 2026 05:03

bharath-k1999 added backport/2.0 Changes are backported to 2.0 backport/ent/2.0 Changes are backported to 2.0 ent labels May 11, 2026

bharath-k1999 added 9 commits May 11, 2026 11:06

Merge branch 'main' into bharath/fix-Last-hop-proxy-leaks

79f74a0

Add golden files for inbound listener configurations

4b84de1

Merge branch 'main' into bharath/fix-Last-hop-proxy-leaks

893ef32

fix: remove x-forwarded-client-cert header from proxy configurations

0f592ce

Merge branch 'main' into bharath/fix-Last-hop-proxy-leaks

dca2797

fix: enhance output parsing and add test for x-forwarded-client-cert …

c850f60

…removal

Merge branch 'main' into bharath/fix-Last-hop-proxy-leaks

59ee364

Merge branch 'main' into bharath/fix-Last-hop-proxy-leaks

f504fef

add changelog

18f3cd0

bharath-k1999 added backport/ent/1.21 changes are backported to 1.21 ent backport/ent/1.22 Changes are backported to 1.22 ent labels May 25, 2026

nitin-sachdev-29 requested a review from Copilot May 26, 2026 10:27

Copilot started reviewing on behalf of nitin-sachdev-29 May 26, 2026 10:27 View session

Copilot AI reviewed May 26, 2026

View reviewed changes

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Remove x-forwarded-client-cert header from request headers in proxy configurations#23544

Remove x-forwarded-client-cert header from request headers in proxy configurations#23544
bharath-k1999 wants to merge 10 commits into
mainfrom
bharath/fix-Last-hop-proxy-leaks

bharath-k1999 commented May 11, 2026 •

edited

Loading

Uh oh!

Copilot AI left a comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

Conversation

bharath-k1999 commented May 11, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Description

Testing & Reproduction steps

Links

PR Checklist

PCI review checklist

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Reviewed changes

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

bharath-k1999 commented May 11, 2026 •

edited

Loading