From b15a58fcd61299a1511e899bba10cf25d6cd10ea Mon Sep 17 00:00:00 2001 From: Thomas Kosiewski Date: Mon, 16 Nov 2026 18:20:47 +0000 Subject: [PATCH 1/3] docs(sync): name OPENSTORAGE_SYNC_ALLOW_PRIVATE for push destinations MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit The push-upstream bullet of http-api.md and the comment at the top of internal/sync/worker_push.go named only the deprecated push-only OPENSTORAGE_SYNC_PUSH_ALLOW_PRIVATE. The server reads it only while OPENSTORAGE_SYNC_ALLOW_PRIVATE is unset (cmd/openstorage/config.go, resolveSyncEgress). Both now name the unified variable first and say when the deprecated one is ignored. configuration.md adds that a malformed deprecated value still stops serve at startup. Signed-off-by: Thomas Kosiewski --- _Generated with [`xum`](https://github.com/coder/xum) • Model: `anthropic:claude-opus-5-5` • Thinking: `xhigh`_ Change-Id: Icc90d2223ca5ef62c7d670edcf60627625922f20 --- docs/reference/configuration.md | 3 ++- docs/reference/http-api.md | 14 ++++++++++---- internal/sync/worker_push.go | 4 +++- 3 files changed, 15 insertions(+), 6 deletions(-) diff --git a/docs/reference/configuration.md b/docs/reference/configuration.md index c0cd0ac9..44072aaa 100644 --- a/docs/reference/configuration.md +++ b/docs/reference/configuration.md @@ -375,7 +375,8 @@ through an external proxy. - **Default:** `false`. - If [`OPENSTORAGE_SYNC_ALLOW_PRIVATE`](#openstorage_sync_allow_private) is unset, `true` relaxes the private receiver classes for push forwards only. -- If the unified variable is set, this one is ignored. +- If the unified variable is set, the value of this one is ignored. A malformed value + still stops `serve` at startup. - Either way, setting it logs a boot deprecation notice naming the replacement. ### `OPENSTORAGE_WEBHOOK_ALLOW_PRIVATE` diff --git a/docs/reference/http-api.md b/docs/reference/http-api.md index 31cdb60d..2a90609a 100644 --- a/docs/reference/http-api.md +++ b/docs/reference/http-api.md @@ -941,10 +941,14 @@ Upstream documents no value and no `Link` or `Sunset` header, so open.storage se Auth, config, and rejection failures do not retry. - A successful run schedules one follow-up inbound sync, so a pull that raced the forward cannot strand a stale local snapshot. -- Private/special destinations are denied (permanent `repository configuration error` - failure) unless +- Private and special-purpose destinations are denied, a permanent + `repository configuration error` failure. Setting + [`OPENSTORAGE_SYNC_ALLOW_PRIVATE`](configuration.md#openstorage_sync_allow_private) to + `true` admits internal forges (private, loopback, ULA and link-local addresses); + cloud-metadata and special-purpose ranges stay denied. The deprecated push-only [`OPENSTORAGE_SYNC_PUSH_ALLOW_PRIVATE`](configuration.md#openstorage_sync_push_allow_private) - opts internal forges in. + counts only while `OPENSTORAGE_SYNC_ALLOW_PRIVATE` is unset; when that one is set, the + deprecated variable is ignored. **Events and cancellation:** outbound runs emit `repo.push_upstream.*` events ([webhooks](webhooks.md)); inbound runs emit `repo.sync.*`. An outbound run emits @@ -995,7 +999,9 @@ freshness pokes) and push-upstream forwards. - Governed by [`OPENSTORAGE_SYNC_ALLOW_PRIVATE`](configuration.md#openstorage_sync_allow_private). - Transitional default: pulls allow private upstreams, pushes deny. + Transitional default: pulls allow private upstreams, pushes deny. While it is unset, the + deprecated push-only `OPENSTORAGE_SYNC_PUSH_ALLOW_PRIVATE` can relax pushes; when it is + set, the deprecated variable is ignored. - A denied pull destination fails the run as `repository configuration error`; a denied push destination fails the same way. - `POST /repos` with `upstream` or a `base_repo` sync also rejects, with 400, an diff --git a/internal/sync/worker_push.go b/internal/sync/worker_push.go index 086c3a1b..e47dfd1a 100644 --- a/internal/sync/worker_push.go +++ b/internal/sync/worker_push.go @@ -21,7 +21,9 @@ package sync // writes into private networks are a stronger SSRF primitive than the // pull lane's reads, so every http(s) push dials through a vetting proxy // that denies private/special destinations unless the operator opts in -// (OPENSTORAGE_SYNC_PUSH_ALLOW_PRIVATE); see docs/adr/044. +// with OPENSTORAGE_SYNC_ALLOW_PRIVATE. The deprecated push-only +// OPENSTORAGE_SYNC_PUSH_ALLOW_PRIVATE counts only while that one is unset; +// see docs/adr/044 and docs/adr/045. import ( "context" From 2503995a041a882e218c4cc0dd65306a94c543de Mon Sep 17 00:00:00 2001 From: Thomas Kosiewski Date: Mon, 16 Nov 2026 18:29:25 +0000 Subject: [PATCH 2/3] docs(sync): state what OPENSTORAGE_SYNC_ALLOW_PRIVATE admits, as the guard does MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit The review found the new sentence wrong both ways: true also admits RFC 6598 shared addresses (100.64.0.0/10), and the guard keeps only the two metadata addresses 169.254.169.254 and fd00:ec2::254 denied, not every metadata-looking range. http-api.md and configuration.md now use the wording of the webhook guard docs. An empty OPENSTORAGE_SYNC_ALLOW_PRIVATE counts as unset (resolveSyncEgress), so the docs say "unset or empty" and "has a value", and the worker_push.go comment says that the opt-in admits only the private classes. Signed-off-by: Thomas Kosiewski --- _Generated with [`xum`](https://github.com/coder/xum) • Model: `anthropic:claude-opus-5-5` • Thinking: `xhigh`_ Change-Id: If5312b03db725222ca4fd8192063401075ae2ed3 --- docs/reference/configuration.md | 16 +++++++++------- docs/reference/http-api.md | 16 +++++++++------- internal/sync/worker_push.go | 8 ++++---- 3 files changed, 22 insertions(+), 18 deletions(-) diff --git a/docs/reference/configuration.md b/docs/reference/configuration.md index 44072aaa..75dd6f09 100644 --- a/docs/reference/configuration.md +++ b/docs/reference/configuration.md @@ -339,10 +339,12 @@ Unified egress policy for upstream sync in **both** directions (ADR-45). - **Default:** unset (transitional; see below). - **Values:** - - `true`: both directions may target internal forges (RFC 1918, loopback, ULA, - link-local). Cloud-metadata and special-purpose ranges stay denied. + - `true`: both directions may target internal forges: loopback, RFC 1918, RFC 6598 + shared, link-local and ULA addresses. The cloud-metadata addresses + (`169.254.169.254`, `fd00:ec2::254`) and the other IANA special-purpose ranges stay + denied. - `false`: strict both ways; only public upstreams are dialable. - - **Unset:** pushes stay strict (the deprecated + - **Unset or empty:** pushes stay strict (the deprecated [`OPENSTORAGE_SYNC_PUSH_ALLOW_PRIVATE`](#openstorage_sync_push_allow_private) may relax them); pulls ALLOW private upstreams, and `serve` logs a boot warning. - Malformed booleans are startup errors. @@ -373,10 +375,10 @@ through an external proxy. **Deprecated** push-only spelling of the sync egress policy, kept for 0.6.x deployments. - **Default:** `false`. -- If [`OPENSTORAGE_SYNC_ALLOW_PRIVATE`](#openstorage_sync_allow_private) is unset, `true` - relaxes the private receiver classes for push forwards only. -- If the unified variable is set, the value of this one is ignored. A malformed value - still stops `serve` at startup. +- If [`OPENSTORAGE_SYNC_ALLOW_PRIVATE`](#openstorage_sync_allow_private) is unset or + empty, `true` relaxes the private receiver classes for push forwards only. +- If the unified variable has a value, the value of this one is ignored. A malformed + value still stops `serve` at startup. - Either way, setting it logs a boot deprecation notice naming the replacement. ### `OPENSTORAGE_WEBHOOK_ALLOW_PRIVATE` diff --git a/docs/reference/http-api.md b/docs/reference/http-api.md index 2a90609a..50d211dd 100644 --- a/docs/reference/http-api.md +++ b/docs/reference/http-api.md @@ -944,11 +944,12 @@ Upstream documents no value and no `Link` or `Sunset` header, so open.storage se - Private and special-purpose destinations are denied, a permanent `repository configuration error` failure. Setting [`OPENSTORAGE_SYNC_ALLOW_PRIVATE`](configuration.md#openstorage_sync_allow_private) to - `true` admits internal forges (private, loopback, ULA and link-local addresses); - cloud-metadata and special-purpose ranges stay denied. The deprecated push-only + `true` admits loopback, RFC 1918, RFC 6598 shared, link-local and ULA addresses; the + cloud-metadata addresses (`169.254.169.254`, `fd00:ec2::254`) and the other IANA + special-purpose ranges stay denied. The deprecated push-only [`OPENSTORAGE_SYNC_PUSH_ALLOW_PRIVATE`](configuration.md#openstorage_sync_push_allow_private) - counts only while `OPENSTORAGE_SYNC_ALLOW_PRIVATE` is unset; when that one is set, the - deprecated variable is ignored. + counts only while `OPENSTORAGE_SYNC_ALLOW_PRIVATE` is unset or empty; when + `OPENSTORAGE_SYNC_ALLOW_PRIVATE` has a value, the deprecated variable is ignored. **Events and cancellation:** outbound runs emit `repo.push_upstream.*` events ([webhooks](webhooks.md)); inbound runs emit `repo.sync.*`. An outbound run emits @@ -999,9 +1000,10 @@ freshness pokes) and push-upstream forwards. - Governed by [`OPENSTORAGE_SYNC_ALLOW_PRIVATE`](configuration.md#openstorage_sync_allow_private). - Transitional default: pulls allow private upstreams, pushes deny. While it is unset, the - deprecated push-only `OPENSTORAGE_SYNC_PUSH_ALLOW_PRIVATE` can relax pushes; when it is - set, the deprecated variable is ignored. + Transitional default: pulls allow private upstreams, pushes deny. While + `OPENSTORAGE_SYNC_ALLOW_PRIVATE` is unset or empty, the deprecated push-only + `OPENSTORAGE_SYNC_PUSH_ALLOW_PRIVATE` can relax pushes; when + `OPENSTORAGE_SYNC_ALLOW_PRIVATE` has a value, the deprecated variable is ignored. - A denied pull destination fails the run as `repository configuration error`; a denied push destination fails the same way. - `POST /repos` with `upstream` or a `base_repo` sync also rejects, with 400, an diff --git a/internal/sync/worker_push.go b/internal/sync/worker_push.go index e47dfd1a..038eed99 100644 --- a/internal/sync/worker_push.go +++ b/internal/sync/worker_push.go @@ -20,10 +20,10 @@ package sync // GIT_ALLOW_PROTOCOL pin, PLUS a dial-time destination guard (egress.go): // writes into private networks are a stronger SSRF primitive than the // pull lane's reads, so every http(s) push dials through a vetting proxy -// that denies private/special destinations unless the operator opts in -// with OPENSTORAGE_SYNC_ALLOW_PRIVATE. The deprecated push-only -// OPENSTORAGE_SYNC_PUSH_ALLOW_PRIVATE counts only while that one is unset; -// see docs/adr/044 and docs/adr/045. +// that denies private and special-purpose destinations. +// OPENSTORAGE_SYNC_ALLOW_PRIVATE=true admits only the private classes. The +// deprecated push-only OPENSTORAGE_SYNC_PUSH_ALLOW_PRIVATE counts only +// while that one is unset or empty; see docs/adr/044 and docs/adr/045. import ( "context" From d379114b98fd0a81f85ca13426908232bcd74f9b Mon Sep 17 00:00:00 2001 From: Thomas Kosiewski Date: Mon, 16 Nov 2026 18:39:22 +0000 Subject: [PATCH 3/3] docs(sync): say which metadata addresses stay denied and that an empty value is unset MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit The second review found the wording still loose: "the cloud-metadata addresses" read as every metadata endpoint, "has a value" clashed with configuration.md's own word for an empty value, and "the deprecated variable is ignored" hid that serve still parses it. The docs now say that only two metadata addresses stay denied, that an empty unified value counts as unset, and that only the deprecated value is ignored. TestResolveSyncEgress pins the empty case. Two older lines get the same fixes: the table row at the top of configuration.md and the resolveSyncEgress line of cmd/openstorage/README.md. Signed-off-by: Thomas Kosiewski --- _Generated with [`xum`](https://github.com/coder/xum) • Model: `anthropic:claude-opus-5-5` • Thinking: `xhigh`_ Change-Id: Ia4e3302fa18b814fd329fadac6b70c067d57f09e --- cmd/openstorage/README.md | 2 +- cmd/openstorage/config_test.go | 4 ++++ docs/reference/configuration.md | 22 +++++++++++++--------- docs/reference/http-api.md | 13 ++++++++----- 4 files changed, 26 insertions(+), 15 deletions(-) diff --git a/cmd/openstorage/README.md b/cmd/openstorage/README.md index 82d09c7e..17797f9d 100644 --- a/cmd/openstorage/README.md +++ b/cmd/openstorage/README.md @@ -96,7 +96,7 @@ The `hook` subprocess also reads per-push variables that the Git server sets (na - `[config.go:resolveSweepSettings]` `OPENSTORAGE_REPO_RETENTION`, when non-zero, must be `>= jobs.MinRepoPurgeAge` and needs an active sweep. - `[config.go:resolveLFSGCSettings]` Grace and object budget must be `> 0`. Delete mode needs a non-zero interval. - `[config.go:resolveRepackThreshold]` The threshold must be `0` or `>= 2`. -- `[config.go:resolveSyncEgress]` `OPENSTORAGE_SYNC_ALLOW_PRIVATE`, when set, governs both push and pull. A malformed value in either spelling is an error. +- `[config.go:resolveSyncEgress]` `OPENSTORAGE_SYNC_ALLOW_PRIVATE`, when set to a non-empty value, governs both push and pull. A malformed value in either spelling is an error. - `[config.go:resolveSecretsCodec]` `decrypt-only` requires a configured key. - `[config.go:normalizeBaseURL]` The base URL must be `http` or `https` with a host and no query or fragment. - `[config.go:assertf]` Impossible states panic with `openstorage assertion failed:`. Input errors return errors. diff --git a/cmd/openstorage/config_test.go b/cmd/openstorage/config_test.go index 64a1078f..877751c3 100644 --- a/cmd/openstorage/config_test.go +++ b/cmd/openstorage/config_test.go @@ -1371,6 +1371,10 @@ func TestResolveSyncEgress(t *testing.T) { env: map[string]string{envSyncAllowPrivate: "false"}, want: syncEgressSettings{}, }, + "empty unified counts as unset": { + env: map[string]string{envSyncAllowPrivate: "", envSyncPushAllowPrivate: "true"}, + want: syncEgressSettings{pushAllowPrivate: true, pullAllowPrivate: true, pullTransitional: true, legacyPushFlagSet: true}, + }, "unified wins over legacy": { env: map[string]string{envSyncAllowPrivate: "false", envSyncPushAllowPrivate: "true"}, want: syncEgressSettings{legacyPushFlagSet: true}, diff --git a/docs/reference/configuration.md b/docs/reference/configuration.md index 75dd6f09..6b6c347d 100644 --- a/docs/reference/configuration.md +++ b/docs/reference/configuration.md @@ -47,7 +47,7 @@ Each setting takes its value from the first source that is set: | [`OPENSTORAGE_RATE_BURST`](#openstorage_rate_rps--openstorage_rate_burst) | unset (disabled) | Rate-limit token-bucket depth | | [`OPENSTORAGE_TRUSTED_PROXIES`](#openstorage_trusted_proxies) | unset | CIDR allowlist of reverse proxies for rate-limit client identity | | [`OPENSTORAGE_WEBHOOK_ALLOW_PRIVATE`](#openstorage_webhook_allow_private) | `false` | Permit private-network webhook receivers (metadata and redirects stay blocked) | -| [`OPENSTORAGE_SYNC_ALLOW_PRIVATE`](#openstorage_sync_allow_private) | unset (transitional) | Unified sync egress policy: permit private-network upstreams for pull and push (metadata stays blocked) | +| [`OPENSTORAGE_SYNC_ALLOW_PRIVATE`](#openstorage_sync_allow_private) | unset (transitional) | Unified sync egress policy: permit private-network upstreams for pull and push (two metadata addresses stay blocked) | | [`OPENSTORAGE_SYNC_PUSH_ALLOW_PRIVATE`](#openstorage_sync_push_allow_private) | `false` | **Deprecated** push-only spelling of the sync egress policy | | [`OPENSTORAGE_SECRETS_KEY`](#openstorage_secrets_key) | unset (plaintext) | AES-256 key list for at-rest encryption of stored webhook secrets and upstream credentials | | [`OPENSTORAGE_SECRETS_MODE`](#openstorage_secrets_mode) | `encrypt` | `decrypt-only` stages the key fleet-wide before encrypted writes begin | @@ -337,12 +337,15 @@ limiters may trust. Bare IPs count as single-address entries. Unified egress policy for upstream sync in **both** directions (ADR-45). -- **Default:** unset (transitional; see below). +- **Default:** unset (transitional; see below). An empty value counts as unset. + `OPENSTORAGE_SYNC_PUSH_ALLOW_PRIVATE` is a separate variable, not a legacy alias of + this one, so an empty value here lets it decide pushes. - **Values:** - `true`: both directions may target internal forges: loopback, RFC 1918, RFC 6598 - shared, link-local and ULA addresses. The cloud-metadata addresses + shared, link-local and ULA addresses. Only two cloud-metadata addresses (`169.254.169.254`, `fd00:ec2::254`) and the other IANA special-purpose ranges stay - denied. + denied; a metadata endpoint inside an admitted range, such as `100.100.100.200`, is + admitted. - `false`: strict both ways; only public upstreams are dialable. - **Unset or empty:** pushes stay strict (the deprecated [`OPENSTORAGE_SYNC_PUSH_ALLOW_PRIVATE`](#openstorage_sync_push_allow_private) may relax @@ -350,8 +353,8 @@ Unified egress policy for upstream sync in **both** directions (ADR-45). - Malformed booleans are startup errors. - **Enforcement:** every http(s) sync transfer (`upstream_host` is caller-linked data) dials through a vetting proxy with the webhook destination policy: private, loopback, - link-local, metadata, and special-purpose targets are denied, with every DNS answer - re-vetted at connect time. + RFC 6598 shared, link-local, metadata, and special-purpose targets are denied, with + every DNS answer re-vetted at connect time. - **Covered:** pulls (`POST /repos/pull-upstream`, the first sync at `POST /repos`, freshness pokes) and push forwards (`POST /repos/push-upstream`, `forward_pushes`). - **Denied destination:** the upstream is never contacted. Pulls and pushes fail as @@ -377,9 +380,10 @@ through an external proxy. - **Default:** `false`. - If [`OPENSTORAGE_SYNC_ALLOW_PRIVATE`](#openstorage_sync_allow_private) is unset or empty, `true` relaxes the private receiver classes for push forwards only. -- If the unified variable has a value, the value of this one is ignored. A malformed - value still stops `serve` at startup. -- Either way, setting it logs a boot deprecation notice naming the replacement. +- If the unified variable has a non-empty value, the value of this one is ignored. A + malformed value still stops `serve` at startup. +- Either way, setting it to a non-empty value logs a boot deprecation notice naming the + replacement. ### `OPENSTORAGE_WEBHOOK_ALLOW_PRIVATE` diff --git a/docs/reference/http-api.md b/docs/reference/http-api.md index 50d211dd..d7717c10 100644 --- a/docs/reference/http-api.md +++ b/docs/reference/http-api.md @@ -944,12 +944,14 @@ Upstream documents no value and no `Link` or `Sunset` header, so open.storage se - Private and special-purpose destinations are denied, a permanent `repository configuration error` failure. Setting [`OPENSTORAGE_SYNC_ALLOW_PRIVATE`](configuration.md#openstorage_sync_allow_private) to - `true` admits loopback, RFC 1918, RFC 6598 shared, link-local and ULA addresses; the - cloud-metadata addresses (`169.254.169.254`, `fd00:ec2::254`) and the other IANA - special-purpose ranges stay denied. The deprecated push-only + `true` admits loopback, RFC 1918, RFC 6598 shared, link-local and ULA addresses. Only + two cloud-metadata addresses (`169.254.169.254`, `fd00:ec2::254`) and the other IANA + special-purpose ranges stay denied; a metadata endpoint inside an admitted range, such + as `100.100.100.200`, is admitted. The deprecated push-only [`OPENSTORAGE_SYNC_PUSH_ALLOW_PRIVATE`](configuration.md#openstorage_sync_push_allow_private) counts only while `OPENSTORAGE_SYNC_ALLOW_PRIVATE` is unset or empty; when - `OPENSTORAGE_SYNC_ALLOW_PRIVATE` has a value, the deprecated variable is ignored. + `OPENSTORAGE_SYNC_ALLOW_PRIVATE` has a non-empty value, the value of the deprecated + variable is ignored (a malformed one still stops `serve` at startup). **Events and cancellation:** outbound runs emit `repo.push_upstream.*` events ([webhooks](webhooks.md)); inbound runs emit `repo.sync.*`. An outbound run emits @@ -1003,7 +1005,8 @@ freshness pokes) and push-upstream forwards. Transitional default: pulls allow private upstreams, pushes deny. While `OPENSTORAGE_SYNC_ALLOW_PRIVATE` is unset or empty, the deprecated push-only `OPENSTORAGE_SYNC_PUSH_ALLOW_PRIVATE` can relax pushes; when - `OPENSTORAGE_SYNC_ALLOW_PRIVATE` has a value, the deprecated variable is ignored. + `OPENSTORAGE_SYNC_ALLOW_PRIVATE` has a non-empty value, the value of the deprecated + variable is ignored. - A denied pull destination fails the run as `repository configuration error`; a denied push destination fails the same way. - `POST /repos` with `upstream` or a `base_repo` sync also rejects, with 400, an