diff --git a/cmd/openstorage/README.md b/cmd/openstorage/README.md index 82d09c7e..17797f9d 100644 --- a/cmd/openstorage/README.md +++ b/cmd/openstorage/README.md @@ -96,7 +96,7 @@ The `hook` subprocess also reads per-push variables that the Git server sets (na - `[config.go:resolveSweepSettings]` `OPENSTORAGE_REPO_RETENTION`, when non-zero, must be `>= jobs.MinRepoPurgeAge` and needs an active sweep. - `[config.go:resolveLFSGCSettings]` Grace and object budget must be `> 0`. Delete mode needs a non-zero interval. - `[config.go:resolveRepackThreshold]` The threshold must be `0` or `>= 2`. -- `[config.go:resolveSyncEgress]` `OPENSTORAGE_SYNC_ALLOW_PRIVATE`, when set, governs both push and pull. A malformed value in either spelling is an error. +- `[config.go:resolveSyncEgress]` `OPENSTORAGE_SYNC_ALLOW_PRIVATE`, when set to a non-empty value, governs both push and pull. A malformed value in either spelling is an error. - `[config.go:resolveSecretsCodec]` `decrypt-only` requires a configured key. - `[config.go:normalizeBaseURL]` The base URL must be `http` or `https` with a host and no query or fragment. - `[config.go:assertf]` Impossible states panic with `openstorage assertion failed:`. Input errors return errors. diff --git a/cmd/openstorage/config_test.go b/cmd/openstorage/config_test.go index 64a1078f..877751c3 100644 --- a/cmd/openstorage/config_test.go +++ b/cmd/openstorage/config_test.go @@ -1371,6 +1371,10 @@ func TestResolveSyncEgress(t *testing.T) { env: map[string]string{envSyncAllowPrivate: "false"}, want: syncEgressSettings{}, }, + "empty unified counts as unset": { + env: map[string]string{envSyncAllowPrivate: "", envSyncPushAllowPrivate: "true"}, + want: syncEgressSettings{pushAllowPrivate: true, pullAllowPrivate: true, pullTransitional: true, legacyPushFlagSet: true}, + }, "unified wins over legacy": { env: map[string]string{envSyncAllowPrivate: "false", envSyncPushAllowPrivate: "true"}, want: syncEgressSettings{legacyPushFlagSet: true}, diff --git a/docs/reference/configuration.md b/docs/reference/configuration.md index c0cd0ac9..6b6c347d 100644 --- a/docs/reference/configuration.md +++ b/docs/reference/configuration.md @@ -47,7 +47,7 @@ Each setting takes its value from the first source that is set: | [`OPENSTORAGE_RATE_BURST`](#openstorage_rate_rps--openstorage_rate_burst) | unset (disabled) | Rate-limit token-bucket depth | | [`OPENSTORAGE_TRUSTED_PROXIES`](#openstorage_trusted_proxies) | unset | CIDR allowlist of reverse proxies for rate-limit client identity | | [`OPENSTORAGE_WEBHOOK_ALLOW_PRIVATE`](#openstorage_webhook_allow_private) | `false` | Permit private-network webhook receivers (metadata and redirects stay blocked) | -| [`OPENSTORAGE_SYNC_ALLOW_PRIVATE`](#openstorage_sync_allow_private) | unset (transitional) | Unified sync egress policy: permit private-network upstreams for pull and push (metadata stays blocked) | +| [`OPENSTORAGE_SYNC_ALLOW_PRIVATE`](#openstorage_sync_allow_private) | unset (transitional) | Unified sync egress policy: permit private-network upstreams for pull and push (two metadata addresses stay blocked) | | [`OPENSTORAGE_SYNC_PUSH_ALLOW_PRIVATE`](#openstorage_sync_push_allow_private) | `false` | **Deprecated** push-only spelling of the sync egress policy | | [`OPENSTORAGE_SECRETS_KEY`](#openstorage_secrets_key) | unset (plaintext) | AES-256 key list for at-rest encryption of stored webhook secrets and upstream credentials | | [`OPENSTORAGE_SECRETS_MODE`](#openstorage_secrets_mode) | `encrypt` | `decrypt-only` stages the key fleet-wide before encrypted writes begin | @@ -337,19 +337,24 @@ limiters may trust. Bare IPs count as single-address entries. Unified egress policy for upstream sync in **both** directions (ADR-45). -- **Default:** unset (transitional; see below). +- **Default:** unset (transitional; see below). An empty value counts as unset. + `OPENSTORAGE_SYNC_PUSH_ALLOW_PRIVATE` is a separate variable, not a legacy alias of + this one, so an empty value here lets it decide pushes. - **Values:** - - `true`: both directions may target internal forges (RFC 1918, loopback, ULA, - link-local). Cloud-metadata and special-purpose ranges stay denied. + - `true`: both directions may target internal forges: loopback, RFC 1918, RFC 6598 + shared, link-local and ULA addresses. Only two cloud-metadata addresses + (`169.254.169.254`, `fd00:ec2::254`) and the other IANA special-purpose ranges stay + denied; a metadata endpoint inside an admitted range, such as `100.100.100.200`, is + admitted. - `false`: strict both ways; only public upstreams are dialable. - - **Unset:** pushes stay strict (the deprecated + - **Unset or empty:** pushes stay strict (the deprecated [`OPENSTORAGE_SYNC_PUSH_ALLOW_PRIVATE`](#openstorage_sync_push_allow_private) may relax them); pulls ALLOW private upstreams, and `serve` logs a boot warning. - Malformed booleans are startup errors. - **Enforcement:** every http(s) sync transfer (`upstream_host` is caller-linked data) dials through a vetting proxy with the webhook destination policy: private, loopback, - link-local, metadata, and special-purpose targets are denied, with every DNS answer - re-vetted at connect time. + RFC 6598 shared, link-local, metadata, and special-purpose targets are denied, with + every DNS answer re-vetted at connect time. - **Covered:** pulls (`POST /repos/pull-upstream`, the first sync at `POST /repos`, freshness pokes) and push forwards (`POST /repos/push-upstream`, `forward_pushes`). - **Denied destination:** the upstream is never contacted. Pulls and pushes fail as @@ -373,10 +378,12 @@ through an external proxy. **Deprecated** push-only spelling of the sync egress policy, kept for 0.6.x deployments. - **Default:** `false`. -- If [`OPENSTORAGE_SYNC_ALLOW_PRIVATE`](#openstorage_sync_allow_private) is unset, `true` - relaxes the private receiver classes for push forwards only. -- If the unified variable is set, this one is ignored. -- Either way, setting it logs a boot deprecation notice naming the replacement. +- If [`OPENSTORAGE_SYNC_ALLOW_PRIVATE`](#openstorage_sync_allow_private) is unset or + empty, `true` relaxes the private receiver classes for push forwards only. +- If the unified variable has a non-empty value, the value of this one is ignored. A + malformed value still stops `serve` at startup. +- Either way, setting it to a non-empty value logs a boot deprecation notice naming the + replacement. ### `OPENSTORAGE_WEBHOOK_ALLOW_PRIVATE` diff --git a/docs/reference/http-api.md b/docs/reference/http-api.md index 31cdb60d..d7717c10 100644 --- a/docs/reference/http-api.md +++ b/docs/reference/http-api.md @@ -941,10 +941,17 @@ Upstream documents no value and no `Link` or `Sunset` header, so open.storage se Auth, config, and rejection failures do not retry. - A successful run schedules one follow-up inbound sync, so a pull that raced the forward cannot strand a stale local snapshot. -- Private/special destinations are denied (permanent `repository configuration error` - failure) unless +- Private and special-purpose destinations are denied, a permanent + `repository configuration error` failure. Setting + [`OPENSTORAGE_SYNC_ALLOW_PRIVATE`](configuration.md#openstorage_sync_allow_private) to + `true` admits loopback, RFC 1918, RFC 6598 shared, link-local and ULA addresses. Only + two cloud-metadata addresses (`169.254.169.254`, `fd00:ec2::254`) and the other IANA + special-purpose ranges stay denied; a metadata endpoint inside an admitted range, such + as `100.100.100.200`, is admitted. The deprecated push-only [`OPENSTORAGE_SYNC_PUSH_ALLOW_PRIVATE`](configuration.md#openstorage_sync_push_allow_private) - opts internal forges in. + counts only while `OPENSTORAGE_SYNC_ALLOW_PRIVATE` is unset or empty; when + `OPENSTORAGE_SYNC_ALLOW_PRIVATE` has a non-empty value, the value of the deprecated + variable is ignored (a malformed one still stops `serve` at startup). **Events and cancellation:** outbound runs emit `repo.push_upstream.*` events ([webhooks](webhooks.md)); inbound runs emit `repo.sync.*`. An outbound run emits @@ -995,7 +1002,11 @@ freshness pokes) and push-upstream forwards. - Governed by [`OPENSTORAGE_SYNC_ALLOW_PRIVATE`](configuration.md#openstorage_sync_allow_private). - Transitional default: pulls allow private upstreams, pushes deny. + Transitional default: pulls allow private upstreams, pushes deny. While + `OPENSTORAGE_SYNC_ALLOW_PRIVATE` is unset or empty, the deprecated push-only + `OPENSTORAGE_SYNC_PUSH_ALLOW_PRIVATE` can relax pushes; when + `OPENSTORAGE_SYNC_ALLOW_PRIVATE` has a non-empty value, the value of the deprecated + variable is ignored. - A denied pull destination fails the run as `repository configuration error`; a denied push destination fails the same way. - `POST /repos` with `upstream` or a `base_repo` sync also rejects, with 400, an diff --git a/internal/sync/worker_push.go b/internal/sync/worker_push.go index 086c3a1b..038eed99 100644 --- a/internal/sync/worker_push.go +++ b/internal/sync/worker_push.go @@ -20,8 +20,10 @@ package sync // GIT_ALLOW_PROTOCOL pin, PLUS a dial-time destination guard (egress.go): // writes into private networks are a stronger SSRF primitive than the // pull lane's reads, so every http(s) push dials through a vetting proxy -// that denies private/special destinations unless the operator opts in -// (OPENSTORAGE_SYNC_PUSH_ALLOW_PRIVATE); see docs/adr/044. +// that denies private and special-purpose destinations. +// OPENSTORAGE_SYNC_ALLOW_PRIVATE=true admits only the private classes. The +// deprecated push-only OPENSTORAGE_SYNC_PUSH_ALLOW_PRIVATE counts only +// while that one is unset or empty; see docs/adr/044 and docs/adr/045. import ( "context"