Add rate limits initial documentation by nicu-da · Pull Request #1450 · canton-network/cf-docs

nicu-da · 2026-08-24T08:19:06Z

https://github.com/DACH-NY/canton-network-internal/issues/6475

Signed-off-by: Nicu Reut <nicu.reut@digitalasset.com>

mintlify · 2026-08-24T08:19:09Z

Preview deployment for your docs. Learn more about Mintlify Previews.

Project	Status	Preview	Updated (UTC)
cantonfoundation	🟢 Ready	View Preview	Aug 24, 2026, 8:22 AM

💡 Tip: Enable Workflows to automatically generate PRs for you.

martinflorian-da

Thanks a lot! I do expect we'll need to tweak this more going forward but this is super helpful to maintain an overview.

My main nudge at this time would be to remove references to SV app...

martinflorian-da · 2026-08-24T13:50:02Z

+</Warning>
+
+```hocon
+canton.scan-apps.scan-app.parameters.rate-limiting.client-ip-headers = ["x-envoy-external-address"]


This seems to miss a sentence about "what is this snippet here telling me".

It will also confuse people not used to thinking about the low-level HOCON configs; WDYT of making these two name: ADDITIONAL_CONFIG_... style snippets instead?

...and possibly we also want to expose these via Helm values at some point? (So might need a hook here to replace with Helm value overrides if we end up enabling these.)

martinflorian-da · 2026-08-24T13:57:59Z

+
+Both limits must be enforced over a **fixed window of 60 seconds**. A client is allowed at most `limit` requests per 60s window; once the budget for the current window is exhausted, all further requests from that client (or, for the global limit, to that service) are rejected with `429 Too Many Requests` until the window rolls over.
+
+For the sequencer, limits must be applied to gRPC traffic requests.


How confident are you that you can throw "gRPC traffic request" at operators and they'll end up configuring the right thing? Can this be mapped to something on a lower layer (HTTP2?)?

I expect that most proxies have native support for gRPC. Will just switch to gRPC (HTTP2)

nicu-da · 2026-08-24T14:57:16Z

Thanks a lot! I do expect we'll need to tweak this more going forward but this is super helpful to maintain an overview.

My main nudge at this time would be to remove references to SV app...

I definitely expect we keep evolving this in the short to medium term, but wanted to front run this so that SVs can also get a head start on basic rate limits.

Co-authored-by: Martin Florian <martin.florian@digitalasset.com> Co-authored-by: Nicu Reut <nicu.reut@digitalasset.com>

Signed-off-by: Nicu Reut <nicu.reut@digitalasset.com>

nicu-da · 2026-08-26T07:43:38Z

@stas-sbi asking for a review from you as well to make sure we're aligned on the initial version/requirements for infra rate limiting

giner

Looks good to me! Left a few minor comments

giner · 2026-08-26T09:27:19Z

+The default headers (`x-forwarded-for`, `x-real-ip`) are client-controlled and can be spoofed. A client rotating the header value bypasses per-client-IP limits entirely.
+
+Every SV operator must therefore either configure their ingress to **overwrite** (not append to) the header the apps read, or point `client-ip-headers` at a header the client cannot influence. With an Istio/Envoy ingress the recommended value is the non-spoofable `x-envoy-external-address`. Make sure the number of trusted XFF hops on your gateway matches your actual topology, so Envoy derives the real client IP.


Considering that the terms spoofable and non-spoofable are meaningless outside of specific context I'd like to suggest simplifying this section and replacing it with something like:

The default value for rate-limiting.client-ip-headers is set to ["x-forwarded-for", "x-real-ip"] as a convenience trade-off. Every SV operator must configure set rate-limiting.client-ip-headers to the header fully controlled (rewritten, not passed through or appended to) by their ingress. For example ["x-forwarded-for"] or ["x-envoy-external-address"].

thanks, applied

giner · 2026-08-26T09:34:05Z

+The following infrastructure level rate limits should be configured. These limits should apply individually to the Scan API and the sequencer gRPC API.
+
+- **Global limits**: a cap on the total request rate reaching a given service.
+- **Per-source-IP limits**: a cap on the request rate of a single client IP, so a single client cannot consume the global budget.


it would make sense to note that these limits are last resort and they must be higher than app limits

added a note

giner · 2026-08-26T09:37:41Z

+
+Both limits must be enforced over a **fixed window of 60 seconds**. A client is allowed at most `limit` requests per 60s window; once the budget for the current window is exhausted, all further requests from that client (or, for the global limit, to that service) are rejected with `429 Too Many Requests` until the window rolls over.
+
+For the sequencer, limits must be applied to gRPC(HTTP/2) traffic requests.


this sounds as if there were other options but I can't think of any, so I'm not sure what this comment intends to say

leaving it there just so that it's extra explicit

giner · 2026-08-26T09:42:56Z

+
+### Shared rate limiter configuration
+
+The same shared rate limiter configuration file that defines application-level overrides will also define a number of infrastructure level configurations (global and per-IP limits, the 60s rate limiting interval, per IP overrides, and any ban thresholds) that should be applied to the ingress by the SV operator.


should this say suggested infrastructure level configurations?

yup, reworded

Signed-off-by: Nicu Reut <nicu.reut@digitalasset.com>

Add rate limits initial documentation

aa232cb

Signed-off-by: Nicu Reut <nicu.reut@digitalasset.com>

mintlify Bot deployed to staging - docs-main August 24, 2026 08:22 View deployment

nicu-da requested a review from martinflorian-da August 24, 2026 09:58

martinflorian-da approved these changes Aug 24, 2026

View reviewed changes

martinflorian-da mentioned this pull request Aug 24, 2026

Expose sv app endpoints to the minimum users canton-network/splice#6934

Merged

8 tasks

nicu-da commented Aug 26, 2026

View reviewed changes

Comment thread docs-main/global-synchronizer/production-operations/sv-rate-limits.mdx Outdated

nicu-da commented Aug 26, 2026

View reviewed changes

Comment thread docs-main/global-synchronizer/production-operations/sv-rate-limits.mdx Outdated

Apply suggestions from code review

2f8e6ad

Co-authored-by: Martin Florian <martin.florian@digitalasset.com> Co-authored-by: Nicu Reut <nicu.reut@digitalasset.com>

nicu-da commented Aug 26, 2026

View reviewed changes

Comment thread docs-main/global-synchronizer/production-operations/sv-rate-limits.mdx Outdated

Apply suggestion from @nicu-da

c242a60

mintlify Bot deployed to staging - docs-main August 26, 2026 07:12 View deployment

mintlify Bot deployed to staging - docs-main August 26, 2026 07:13 View deployment

fix docs

7b6d6db

Signed-off-by: Nicu Reut <nicu.reut@digitalasset.com>

mintlify Bot deployed to staging - docs-main August 26, 2026 07:18 View deployment

.

65f1dd5

Signed-off-by: Nicu Reut <nicu.reut@digitalasset.com>

mintlify Bot deployed to staging - docs-main August 26, 2026 07:19 View deployment

nicu-da marked this pull request as ready for review August 26, 2026 07:33

nicu-da requested a review from stas-sbi August 26, 2026 07:43

giner reviewed Aug 26, 2026

View reviewed changes

.

0d02b6d

Signed-off-by: Nicu Reut <nicu.reut@digitalasset.com>

nicu-da enabled auto-merge (squash) August 26, 2026 11:35

mintlify Bot deployed to staging - docs-main August 26, 2026 11:36 View deployment

nicu-da merged commit b675cf8 into main Aug 26, 2026
3 of 4 checks passed

nicu-da deleted the nicu/rate_limits/initial_docs branch August 26, 2026 11:37


		Both limits must be enforced over a fixed window of 60 seconds. A client is allowed at most `limit` requests per 60s window; once the budget for the current window is exhausted, all further requests from that client (or, for the global limit, to that service) are rejected with `429 Too Many Requests` until the window rolls over.

		For the sequencer, limits must be applied to gRPC traffic requests.

		The default headers (`x-forwarded-for`, `x-real-ip`) are client-controlled and can be spoofed. A client rotating the header value bypasses per-client-IP limits entirely.

		Every SV operator must therefore either configure their ingress to overwrite (not append to) the header the apps read, or point `client-ip-headers` at a header the client cannot influence. With an Istio/Envoy ingress the recommended value is the non-spoofable `x-envoy-external-address`. Make sure the number of trusted XFF hops on your gateway matches your actual topology, so Envoy derives the real client IP.


		### Shared rate limiter configuration

		The same shared rate limiter configuration file that defines application-level overrides will also define a number of infrastructure level configurations (global and per-IP limits, the 60s rate limiting interval, per IP overrides, and any ban thresholds) that should be applied to the ingress by the SV operator.

Uh oh!

Conversation

nicu-da commented Aug 24, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

mintlify Bot commented Aug 24, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

martinflorian-da left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

nicu-da commented Aug 24, 2026

Uh oh!

Uh oh!

Uh oh!

Uh oh!

nicu-da commented Aug 26, 2026

Uh oh!

giner left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

nicu-da commented Aug 24, 2026 •

edited

Loading

mintlify Bot commented Aug 24, 2026 •

edited

Loading