Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion .github/workflows/ci.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -23,7 +23,7 @@ jobs:
version: v3.17.0
- run: pip install -r requirements-dev.txt
- name: Install the Weir contract validator
run: GOBIN="$PWD/.bin" go install github.com/batchstream/weir/cmd/weir@5107986b882da60751403c7e05bbbebdb12b3cdb
run: GOBIN="$PWD/.bin" go install github.com/batchstream/weir/cmd/weir@56b193a2252a299d8a1d8661c7fae25bb78c2518
- name: Install promtool
run: |
curl -fsSLo "$RUNNER_TEMP/prometheus.tar.gz" https://github.com/prometheus/prometheus/releases/download/v3.15.1/prometheus-3.15.1.linux-amd64.tar.gz
Expand Down
2 changes: 1 addition & 1 deletion .github/workflows/release.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -32,7 +32,7 @@ jobs:
version: v3.17.0
- run: pip install -r requirements-dev.txt
- name: Install the Weir contract validator
run: GOBIN="$PWD/.bin" go install github.com/batchstream/weir/cmd/weir@5107986b882da60751403c7e05bbbebdb12b3cdb
run: GOBIN="$PWD/.bin" go install github.com/batchstream/weir/cmd/weir@56b193a2252a299d8a1d8661c7fae25bb78c2518
- name: Install promtool
run: |
curl -fsSLo "$RUNNER_TEMP/prometheus.tar.gz" https://github.com/prometheus/prometheus/releases/download/v3.15.1/prometheus-3.15.1.linux-amd64.tar.gz
Expand Down
6 changes: 3 additions & 3 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -27,9 +27,9 @@ MongoDB and Search data plane, on Kubernetes.

| Item | Version |
| --- | --- |
| Chart | `0.20.0-rc.1` |
| Server source | [commit `5107986b882d`](https://github.com/batchstream/weir/commit/5107986b882da60751403c7e05bbbebdb12b3cdb) |
| Server image | `ghcr.io/batchstream/weir@sha256:b5d949cefb0ba01e18cff2b5210cef954bc387c20cffa87784466e793e0e088e` |
| Chart | `0.20.0` |
| Server source | v0.3.0, [commit `56b193a2252a`](https://github.com/batchstream/weir/commit/56b193a2252a299d8a1d8661c7fae25bb78c2518) |
| Server image | `ghcr.io/batchstream/weir@sha256:6a84f2b3f3ff0776db0f01185d1b0e2cc108cf2e406fbc81b5824206106466b7` |
| Go SDK | [weir-go](https://github.com/batchstream/weir-go) v0.24.1 |

`Chart.yaml` records the server source and `values.yaml` pins the image. Both the
Expand Down
6 changes: 3 additions & 3 deletions charts/weir/Chart.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -2,10 +2,10 @@ apiVersion: v2
name: weir
description: Synchronous MongoDB and Search data plane
type: application
version: 0.20.0-rc.1
appVersion: "5107986b882d"
version: 0.20.0
appVersion: "56b193a2252a"
annotations:
weir.batchstream.io/source-revision: "5107986b882da60751403c7e05bbbebdb12b3cdb"
weir.batchstream.io/source-revision: "56b193a2252a299d8a1d8661c7fae25bb78c2518"
kubeVersion: ">=1.30.0-0"
home: https://github.com/batchstream/weir
sources:
Expand Down
2 changes: 1 addition & 1 deletion charts/weir/values.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -6,7 +6,7 @@ nameOverride: ""
fullnameOverride: ""
image:
repository: ghcr.io/batchstream/weir
digest: sha256:b5d949cefb0ba01e18cff2b5210cef954bc387c20cffa87784466e793e0e088e
digest: sha256:6a84f2b3f3ff0776db0f01185d1b0e2cc108cf2e406fbc81b5824206106466b7
pullPolicy: IfNotPresent
imagePullSecrets: []
# Weir Store definitions; backends and credentials are provisioned separately.
Expand Down
2 changes: 1 addition & 1 deletion docs/configuration.md
Original file line number Diff line number Diff line change
Expand Up @@ -45,7 +45,7 @@ The Chart generates `weir.yaml` with the listeners, `shutdown_timeout` and `stor

## Credentials

Helm retains values in release history. Inline backend credentials are rejected, and so is any `@` in `mongodb.uri` or `search.url`, since it can only carry userinfo. MongoDB and Search use `authentication.username_file/password_file`; mount an externally managed Secret read-only using `extraVolumes/extraVolumeMounts` and reference it by absolute path, since relative paths resolve from `/etc/weir`; [examples/credential-files.yaml](../examples/credential-files.yaml) shows the layout. MongoDB requires SCRAM-SHA-256, authSource and TLS in the URI when authenticating. Both adapters can use `tls.ca_file`, or system trust when it is absent. Credential and CA files are read at startup; roll the Pods after rotating them, for example by changing a `podAnnotations` entry.
Helm retains values in release history. Inline backend credentials are rejected, and so is any `@` in `mongodb.uri` or `search.url`, since it can only carry userinfo. MongoDB and Search use `authentication.username_file/password_file`; mount an externally managed Secret read-only using `extraVolumes/extraVolumeMounts` and reference it by absolute path, since relative paths resolve from `/etc/weir`; [examples/credential-files.yaml](../examples/credential-files.yaml) shows the layout. MongoDB requires SCRAM-SHA-256, authSource and an explicit `tls` option in the URI when authenticating, and `tls` requires credentials. Use `tls=true` unless the backend has no TLS. With `tls=false` the SCRAM handshake and every command travel in plaintext: anyone who can see the traffic can read and change commands, guess the password offline from a captured handshake, or take over the authenticated connection. Use it only when the network path to every replica-set member, including members Weir discovers, is trusted, and give the user a long random password. The Chart's NetworkPolicy limits which Pods can connect where the CNI enforces it, but does not hide traffic from anyone with node or host-network access. Both adapters can use `tls.ca_file`, or system trust when it is absent; MongoDB accepts a CA file only with `tls=true`. Credential and CA files are read at startup; roll the Pods after rotating them, for example by changing a `podAnnotations` entry.

## Validate before deployment

Expand Down
44 changes: 27 additions & 17 deletions scripts/check-contract.py
Original file line number Diff line number Diff line change
Expand Up @@ -49,6 +49,18 @@ def validate_config(binary, resources, directory):
return container, args


def expect_rejected(binary, resources, directory, message):
"""Require the binary to reject the configuration with message."""
try:
validate_config(binary, resources, directory)
except subprocess.CalledProcessError as error:
if message not in error.stderr:
raise RuntimeError("configuration rejected for another reason than "
+ repr(message) + ": " + error.stderr) from error
else:
raise RuntimeError("configuration unexpectedly accepted; expected " + repr(message))


def batching_policy():
"""Return valid values for every batching key."""
policy = {"max_operations": 64, "max_bytes": "2MiB", "max_wait": "0s", "max_in_flight": 8,
Expand Down Expand Up @@ -188,33 +200,31 @@ def main():
(directory / "test-user.txt").write_text("test-user\n")
(directory / "test-password.txt").write_text("test-password\n")
validate_config(binary, render(files), directory)
# SCRAM over an explicit tls=false is accepted for networks without TLS,
# but not with a CA file, and authentication needs an explicit tls.
store["mongodb"]["uri"] = ("mongodb://mongo:27017/?directConnection=true"
"&authMechanism=SCRAM-SHA-256&authSource=admin&tls=false")
validate_config(binary, render(files), directory)
store["tls"] = {"ca_file": "test-ca.pem"}
(directory / "test-ca.pem").write_text("")
expect_rejected(binary, render(files), directory, "MongoDB CA file requires TLS")
del store["tls"]
store["mongodb"]["uri"] = ("mongodb://mongo:27017/?directConnection=true"
"&authMechanism=SCRAM-SHA-256&authSource=admin")
expect_rejected(binary, render(files), directory,
"MongoDB URI authentication requires explicit SCRAM-SHA-256, authSource, and tls")
files = copy.deepcopy(fixtures[1][1])
store = files["stores"][0]
store["search"]["url"] = "https://search:9200"
store["authentication"] = {"username_file": "test-user.txt", "password_file": "test-password.txt"}
validate_config(binary, render(files, "search"), directory)
# check reads the referenced files rather than trusting their names.
store["authentication"]["password_file"] = "missing-password.txt"
try:
validate_config(binary, render(files, "search"), directory)
except subprocess.CalledProcessError as error:
if "credential file unavailable" not in error.stderr:
raise RuntimeError("missing credential file rejected for another reason: "
+ error.stderr) from error
else:
raise RuntimeError("missing credential file unexpectedly accepted")
expect_rejected(binary, render(files, "search"), directory, "credential file unavailable")
# Server validation owns backend semantics beyond the Chart's schema.
invalid = copy.deepcopy(fixtures[0][1])
invalid["stores"][0]["mongodb"]["uri"] = "mongodb://mongo:27017/?readPreference=secondary"
resources = render(invalid)
try:
validate_config(binary, resources, directory)
except subprocess.CalledProcessError as error:
if "unsupported MongoDB URI option" not in error.stderr:
raise RuntimeError("invalid Store configuration rejected for another reason: "
+ error.stderr) from error
else:
raise RuntimeError("invalid Store configuration unexpectedly accepted")
expect_rejected(binary, render(invalid), directory, "unsupported MongoDB URI option")
check_scale(binary, directory)
check_runtime(binary, directory)
print("Contract checks passed: MongoDB/Search configurations, batching, scale, "
Expand Down
6 changes: 3 additions & 3 deletions tests/test_chart.py
Original file line number Diff line number Diff line change
Expand Up @@ -519,11 +519,11 @@ def test_runtime_logging_failures_still_reap_child_and_check_ports(self):
listener.__exit__.assert_called_once()

def test_chart_identity_matches_pinned_image(self):
revision = "5107986b882da60751403c7e05bbbebdb12b3cdb"
digest = "sha256:b5d949cefb0ba01e18cff2b5210cef954bc387c20cffa87784466e793e0e088e"
revision = "56b193a2252a299d8a1d8661c7fae25bb78c2518"
digest = "sha256:6a84f2b3f3ff0776db0f01185d1b0e2cc108cf2e406fbc81b5824206106466b7"
chart = yaml.safe_load((ROOT / "charts/weir/Chart.yaml").read_text())
values = yaml.safe_load((ROOT / "charts/weir/values.yaml").read_text())
self.assertEqual(chart["version"], "0.20.0-rc.1")
self.assertEqual(chart["version"], "0.20.0")
self.assertEqual(chart["annotations"]["weir.batchstream.io/source-revision"], revision)
self.assertEqual(values["image"]["repository"], "ghcr.io/batchstream/weir")
self.assertEqual(values["image"]["digest"], digest)
Expand Down
Loading