Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion cmd/openstorage/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -96,7 +96,7 @@ The `hook` subprocess also reads per-push variables that the Git server sets (na
- `[config.go:resolveSweepSettings]` `OPENSTORAGE_REPO_RETENTION`, when non-zero, must be `>= jobs.MinRepoPurgeAge` and needs an active sweep.
- `[config.go:resolveLFSGCSettings]` Grace and object budget must be `> 0`. Delete mode needs a non-zero interval.
- `[config.go:resolveRepackThreshold]` The threshold must be `0` or `>= 2`.
- `[config.go:resolveSyncEgress]` `OPENSTORAGE_SYNC_ALLOW_PRIVATE`, when set, governs both push and pull. A malformed value in either spelling is an error.
- `[config.go:resolveSyncEgress]` `OPENSTORAGE_SYNC_ALLOW_PRIVATE`, when set to a non-empty value, governs both push and pull. A malformed value in either spelling is an error.
- `[config.go:resolveSecretsCodec]` `decrypt-only` requires a configured key.
- `[config.go:normalizeBaseURL]` The base URL must be `http` or `https` with a host and no query or fragment.
- `[config.go:assertf]` Impossible states panic with `openstorage assertion failed:`. Input errors return errors.
Expand Down
4 changes: 4 additions & 0 deletions cmd/openstorage/config_test.go
Original file line number Diff line number Diff line change
Expand Up @@ -1371,6 +1371,10 @@ func TestResolveSyncEgress(t *testing.T) {
env: map[string]string{envSyncAllowPrivate: "false"},
want: syncEgressSettings{},
},
"empty unified counts as unset": {
env: map[string]string{envSyncAllowPrivate: "", envSyncPushAllowPrivate: "true"},
want: syncEgressSettings{pushAllowPrivate: true, pullAllowPrivate: true, pullTransitional: true, legacyPushFlagSet: true},
},
"unified wins over legacy": {
env: map[string]string{envSyncAllowPrivate: "false", envSyncPushAllowPrivate: "true"},
want: syncEgressSettings{legacyPushFlagSet: true},
Expand Down
29 changes: 18 additions & 11 deletions docs/reference/configuration.md
Original file line number Diff line number Diff line change
Expand Up @@ -47,7 +47,7 @@ Each setting takes its value from the first source that is set:
| [`OPENSTORAGE_RATE_BURST`](#openstorage_rate_rps--openstorage_rate_burst) | unset (disabled) | Rate-limit token-bucket depth |
| [`OPENSTORAGE_TRUSTED_PROXIES`](#openstorage_trusted_proxies) | unset | CIDR allowlist of reverse proxies for rate-limit client identity |
| [`OPENSTORAGE_WEBHOOK_ALLOW_PRIVATE`](#openstorage_webhook_allow_private) | `false` | Permit private-network webhook receivers (metadata and redirects stay blocked) |
| [`OPENSTORAGE_SYNC_ALLOW_PRIVATE`](#openstorage_sync_allow_private) | unset (transitional) | Unified sync egress policy: permit private-network upstreams for pull and push (metadata stays blocked) |
| [`OPENSTORAGE_SYNC_ALLOW_PRIVATE`](#openstorage_sync_allow_private) | unset (transitional) | Unified sync egress policy: permit private-network upstreams for pull and push (two metadata addresses stay blocked) |
| [`OPENSTORAGE_SYNC_PUSH_ALLOW_PRIVATE`](#openstorage_sync_push_allow_private) | `false` | **Deprecated** push-only spelling of the sync egress policy |
| [`OPENSTORAGE_SECRETS_KEY`](#openstorage_secrets_key) | unset (plaintext) | AES-256 key list for at-rest encryption of stored webhook secrets and upstream credentials |
| [`OPENSTORAGE_SECRETS_MODE`](#openstorage_secrets_mode) | `encrypt` | `decrypt-only` stages the key fleet-wide before encrypted writes begin |
Expand Down Expand Up @@ -337,19 +337,24 @@ limiters may trust. Bare IPs count as single-address entries.

Unified egress policy for upstream sync in **both** directions (ADR-45).

- **Default:** unset (transitional; see below).
- **Default:** unset (transitional; see below). An empty value counts as unset.
`OPENSTORAGE_SYNC_PUSH_ALLOW_PRIVATE` is a separate variable, not a legacy alias of
this one, so an empty value here lets it decide pushes.
- **Values:**
- `true`: both directions may target internal forges (RFC 1918, loopback, ULA,
link-local). Cloud-metadata and special-purpose ranges stay denied.
- `true`: both directions may target internal forges: loopback, RFC 1918, RFC 6598
shared, link-local and ULA addresses. Only two cloud-metadata addresses
(`169.254.169.254`, `fd00:ec2::254`) and the other IANA special-purpose ranges stay
denied; a metadata endpoint inside an admitted range, such as `100.100.100.200`, is
admitted.
- `false`: strict both ways; only public upstreams are dialable.
- **Unset:** pushes stay strict (the deprecated
- **Unset or empty:** pushes stay strict (the deprecated
[`OPENSTORAGE_SYNC_PUSH_ALLOW_PRIVATE`](#openstorage_sync_push_allow_private) may relax
them); pulls ALLOW private upstreams, and `serve` logs a boot warning.
- Malformed booleans are startup errors.
- **Enforcement:** every http(s) sync transfer (`upstream_host` is caller-linked data)
dials through a vetting proxy with the webhook destination policy: private, loopback,
link-local, metadata, and special-purpose targets are denied, with every DNS answer
re-vetted at connect time.
RFC 6598 shared, link-local, metadata, and special-purpose targets are denied, with
every DNS answer re-vetted at connect time.
- **Covered:** pulls (`POST /repos/pull-upstream`, the first sync at `POST /repos`,
freshness pokes) and push forwards (`POST /repos/push-upstream`, `forward_pushes`).
- **Denied destination:** the upstream is never contacted. Pulls and pushes fail as
Expand All @@ -373,10 +378,12 @@ through an external proxy.
**Deprecated** push-only spelling of the sync egress policy, kept for 0.6.x deployments.

- **Default:** `false`.
- If [`OPENSTORAGE_SYNC_ALLOW_PRIVATE`](#openstorage_sync_allow_private) is unset, `true`
relaxes the private receiver classes for push forwards only.
- If the unified variable is set, this one is ignored.
- Either way, setting it logs a boot deprecation notice naming the replacement.
- If [`OPENSTORAGE_SYNC_ALLOW_PRIVATE`](#openstorage_sync_allow_private) is unset or
empty, `true` relaxes the private receiver classes for push forwards only.
- If the unified variable has a non-empty value, the value of this one is ignored. A
malformed value still stops `serve` at startup.
- Either way, setting it to a non-empty value logs a boot deprecation notice naming the
replacement.

### `OPENSTORAGE_WEBHOOK_ALLOW_PRIVATE`

Expand Down
19 changes: 15 additions & 4 deletions docs/reference/http-api.md
Original file line number Diff line number Diff line change
Expand Up @@ -941,10 +941,17 @@ Upstream documents no value and no `Link` or `Sunset` header, so open.storage se
Auth, config, and rejection failures do not retry.
- A successful run schedules one follow-up inbound sync, so a pull that raced the forward
cannot strand a stale local snapshot.
- Private/special destinations are denied (permanent `repository configuration error`
failure) unless
- Private and special-purpose destinations are denied, a permanent
`repository configuration error` failure. Setting
[`OPENSTORAGE_SYNC_ALLOW_PRIVATE`](configuration.md#openstorage_sync_allow_private) to
`true` admits loopback, RFC 1918, RFC 6598 shared, link-local and ULA addresses. Only
two cloud-metadata addresses (`169.254.169.254`, `fd00:ec2::254`) and the other IANA
special-purpose ranges stay denied; a metadata endpoint inside an admitted range, such
as `100.100.100.200`, is admitted. The deprecated push-only
[`OPENSTORAGE_SYNC_PUSH_ALLOW_PRIVATE`](configuration.md#openstorage_sync_push_allow_private)
opts internal forges in.
counts only while `OPENSTORAGE_SYNC_ALLOW_PRIVATE` is unset or empty; when
`OPENSTORAGE_SYNC_ALLOW_PRIVATE` has a non-empty value, the value of the deprecated
variable is ignored (a malformed one still stops `serve` at startup).

**Events and cancellation:** outbound runs emit `repo.push_upstream.*` events
([webhooks](webhooks.md)); inbound runs emit `repo.sync.*`. An outbound run emits
Expand Down Expand Up @@ -995,7 +1002,11 @@ freshness pokes) and push-upstream forwards.

- Governed by
[`OPENSTORAGE_SYNC_ALLOW_PRIVATE`](configuration.md#openstorage_sync_allow_private).
Transitional default: pulls allow private upstreams, pushes deny.
Transitional default: pulls allow private upstreams, pushes deny. While
`OPENSTORAGE_SYNC_ALLOW_PRIVATE` is unset or empty, the deprecated push-only
`OPENSTORAGE_SYNC_PUSH_ALLOW_PRIVATE` can relax pushes; when
`OPENSTORAGE_SYNC_ALLOW_PRIVATE` has a non-empty value, the value of the deprecated
variable is ignored.
- A denied pull destination fails the run as `repository configuration error`; a denied
push destination fails the same way.
- `POST /repos` with `upstream` or a `base_repo` sync also rejects, with 400, an
Expand Down
6 changes: 4 additions & 2 deletions internal/sync/worker_push.go
Original file line number Diff line number Diff line change
Expand Up @@ -20,8 +20,10 @@ package sync
// GIT_ALLOW_PROTOCOL pin, PLUS a dial-time destination guard (egress.go):
// writes into private networks are a stronger SSRF primitive than the
// pull lane's reads, so every http(s) push dials through a vetting proxy
// that denies private/special destinations unless the operator opts in
// (OPENSTORAGE_SYNC_PUSH_ALLOW_PRIVATE); see docs/adr/044.
// that denies private and special-purpose destinations.
// OPENSTORAGE_SYNC_ALLOW_PRIVATE=true admits only the private classes. The
// deprecated push-only OPENSTORAGE_SYNC_PUSH_ALLOW_PRIVATE counts only
// while that one is unset or empty; see docs/adr/044 and docs/adr/045.

import (
"context"
Expand Down
Loading