From f345ecf12f28b56a133cec13164a1fe6beb64dc7 Mon Sep 17 00:00:00 2001 From: Randy <857271370@qq.com> Date: Wed, 4 Nov 2026 20:40:07 +0800 Subject: [PATCH 1/4] =?UTF-8?q?=F0=9F=93=9D=20docs:=20move=20reference=20d?= =?UTF-8?q?etail=20from=20the=20README=20to=20docs?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit --- CONTRIBUTING.md | 6 ++--- SECURITY.md | 2 +- docs/autoscaling.md | 15 +++++++++++++ docs/configuration.md | 52 +++++++++++++++++++++++++++++++++++++++++++ docs/development.md | 27 ++++++++++++++++++++++ docs/monitoring.md | 15 +++++++++++++ docs/operations.md | 39 ++++++++++++++++++++++++++++++++ 7 files changed, 152 insertions(+), 4 deletions(-) create mode 100644 docs/autoscaling.md create mode 100644 docs/configuration.md create mode 100644 docs/development.md create mode 100644 docs/monitoring.md create mode 100644 docs/operations.md diff --git a/CONTRIBUTING.md b/CONTRIBUTING.md index 56dc8d5..ea73034 100644 --- a/CONTRIBUTING.md +++ b/CONTRIBUTING.md @@ -15,7 +15,7 @@ For vulnerabilities, follow [SECURITY.md](SECURITY.md). ## Work locally Install Helm, Python 3 and the Go version the chart's workflows use, then follow -[Develop and release](README.md#develop-and-release): +[Development](docs/development.md): ```sh python3 -m venv .venv @@ -32,5 +32,5 @@ rendered alerts and the autoscaling query; without it, that test is skipped. - Keep the change focused and explain the observed problem and resulting behavior. - Add a regression test when fixing behavior. -- Update `values.schema.json`, the [examples](examples) and the README when values change. -- When pinning a new server, pin it in [weir-tests](https://github.com/batchstream/weir-tests) first, then change the source annotation, `appVersion`, image digest, workflow pins, the README's references and the values in `tests/test_chart.py` together. +- Update `values.schema.json`, the [examples](examples), the README and [docs](docs) when values change. +- When pinning a new server, pin it in [weir-tests](https://github.com/batchstream/weir-tests) first, then change the source annotation, `appVersion`, image digest, workflow pins, the references in the README and [docs](docs) and the values in `tests/test_chart.py` together. diff --git a/SECURITY.md b/SECURITY.md index 721b173..4c61620 100644 --- a/SECURITY.md +++ b/SECURITY.md @@ -18,4 +18,4 @@ vulnerabilities even if you cannot reproduce them on `main`. ## Deployment guidance Application traffic uses plaintext HTTP/2 inside a trusted, isolated network. See -[Network and operations](README.md#network-and-operations). +[Network](docs/operations.md#network). diff --git a/docs/autoscaling.md b/docs/autoscaling.md new file mode 100644 index 0000000..67ba5ad --- /dev/null +++ b/docs/autoscaling.md @@ -0,0 +1,15 @@ +# Autoscaling + +How to scale a Store group with KEDA and Prometheus. + +## Example overlay + +Overlay [examples/prometheus-autoscaling.yaml](../examples/prometheus-autoscaling.yaml) on the `mongo` release in namespace `application`. Adjust the namespace, release identity, Prometheus endpoint and scraper labels for your deployment. Its query sums this exact group's Weir process CPU; `AverageValue` lets HPA divide that total across replicas, and the threshold of 1.4 cores is 70% of the default per-replica limit, so change it with `resources`. Scale on Weir's own CPU, not on queued operations or active executions: those rise whenever the backend slows down, and replicas added then bring no capacity, so a queue trigger holds the group at its maximum. Each replica runs fewer record batches at once while executions queue in its backend, so the group's combined requests follow what the backend can take rather than the replica count; a new replica starts at two batches per operation for a little over one probe interval after its first execution, so replicas added in front of a queueing backend do not flood it; the dashboard shows that limit and the latency ratio behind it. Set `maxReplicaCount` from the backend's connection budget: each replica runs up to 16 record batches and 64 Natives per Store at once. + +## Query guard and coverage + +The example guards its aggregate with `and on() (min(up{namespace="application",weir_group="application/mongo"}) == 1)`. A failed discovered scrape target or no discovered targets produces an empty result; `ignoreNullValues: "false"` then leaves KEDA in error instead of treating an incomplete lower workload as a valid scale-down signal. The HTTP timeout is 10 seconds, and no `or vector(0)` supplies a false zero. This guard covers the targets Prometheus has discovered; it cannot prove that scraper configuration includes every group Pod. EndpointSlice discovery keeps a terminating Pod as a target until it is deleted, so its last scrape after exit fails and briefly turns the query into an error; drop endpoints whose `terminating` condition is true, as the default Endpoints discovery already does. Before enabling autoscaling, compare the per-Pod targets with all running/Ready group replicas and verify the labels, `process_cpu_seconds_total` and `up` are present; repeat that coverage check after scale and rollout changes. Configure an existing `TriggerAuthentication`/`ClusterTriggerAuthentication` through `authenticationRef` if needed. No controller credentials are rendered or read by this Chart. KEDA's operator, metrics server and relevant network rules must already allow its Prometheus query. See [the Prometheus scaler contract](https://keda.sh/docs/2.20/scalers/prometheus/) and [ScaledObject specification](https://keda.sh/docs/2.20/reference/scaledobject-spec/). + +## KEDA settings and turning autoscaling on + +`autoscaling.triggers` accepts native KEDA triggers, including Prometheus and other supported scalers. Prometheus triggers require metrics to be enabled; CRDs/controllers are external prerequisites. `pollingInterval`, `cooldownPeriod` and `scaleDown` configure polling and HPA scale-down behavior. The cooldown setting concerns KEDA activation/scale-to-zero; ordinary reduction between running replica counts is controlled by HPA behavior. Minimum replicas must stay at least one: SDK connections cannot wake a group with zero replicas. A PDB requires at least two minimum replicas. Every upgrade or rollback that turns autoscaling on removes the Chart's replica count, so Kubernetes resets the Deployment to one replica until KEDA's HPA scales it to `minReplicaCount`: from three replicas with a minimum of two, two Pods were terminating within 20 seconds and the second Ready replica returned after about a minute. Turn autoscaling on when one replica can carry the load, with `minReplicaCount` at least the running count. `kubectl apply` and GitOps tools reset the count the same way when they turn autoscaling on. Upgrades that keep autoscaling on leave the count to KEDA. Verify the ScaledObject becomes Ready and HPA reaches its minimum before admitting production traffic; Helm readiness alone does not establish controller readiness. Disabling KEDA resumes the configured `replicaCount`. diff --git a/docs/configuration.md b/docs/configuration.md new file mode 100644 index 0000000..4a52ee5 --- /dev/null +++ b/docs/configuration.md @@ -0,0 +1,52 @@ +# Configuration + +How to declare Store groups, connect clients and supply credentials. + +## Store groups + +One Helm release is one Store group. Every replica hosts the same Store definitions; a group can host several Stores, and each group is released and scaled independently. Replicas share nothing with each other. Each Pod runs one Weir process with Lua inside that process. Backends, collections, indices, credentials and backups are provisioned separately. + +### Stores + +Declare this group's Stores directly in `stores`: + +```yaml +stores: + - name: primary + mongodb: + uri: mongodb://mongo:27017/?directConnection=true + - name: search + search: + url: http://search:9200 + batching: + max_wait: 5ms +``` + +Each entry uses Weir's own Store schema: `name`, exactly one of `mongodb.uri` or `search.url`, optional file-based `authentication`, optional `tls.ca_file` and optional `batching` (`max_operations`, `max_bytes`, `max_wait`, `max_in_flight`, `min_in_flight`). Database, collection and index targets come from request URIs; Chart configuration declares the backend connection for each Store. Multiple replicas within a group use identical Stores and settings. A request for a Store the group does not host fails with NotFound. + +### Install several groups + +Install [examples/mongo.yaml](../examples/mongo.yaml) and [examples/search.yaml](../examples/search.yaml) as separate groups in the same namespace: + +```sh +helm upgrade --install mongo ./charts/weir --namespace application \ + --kube-context your-context -f examples/mongo.yaml --atomic --wait --timeout 5m +helm upgrade --install search ./charts/weir --namespace application \ + --kube-context your-context -f examples/search.yaml --atomic --wait --timeout 5m +``` + +## Client connections + +The Chart generates `weir.yaml` with the listeners, `shutdown_timeout` and `stores`. Clients use [weir-go](https://github.com/batchstream/weir-go) v0.23.0, which speaks this server's protocol, and connect with `weir.New` to the group's headless Service by its full name with a trailing dot, `-weir..svc.cluster.local.:`, with the cluster's own domain if it is not `cluster.local`; without the dot every lookup first tries the Pod's DNS search domains. The name resolves to every Ready Pod, and the SDK spreads business RPCs across them. The SDK looks the name up every 5 seconds, so Pods added by a scale-out or rollout take their share of calls within seconds of becoming Ready, and a terminating Pod stops receiving calls once it leaves DNS, before SIGTERM. It also pings a replica it has heard nothing from for 10 seconds and leaves one that does not answer; this server accepts those pings, so upgrade the group before its clients. Generated `[::]` listeners accept IPv4 and IPv6 on the supported Go runtime. Group identity is `namespace/release`, so metric labels remain stable when `fullnameOverride` changes. The Service selects only its release. Allow clients to reach each group's application port. + +## Generated configuration + +The Chart renders one `weir.yaml` into its ConfigMap and mounts it read-only at `/etc/weir`. Its schema rejects unknown keys; the selected Weir binary validates backend semantics. Weir fixes transport, memory-pressure, Lua, queue and retry policy internally, so only listeners, the shutdown drain and Stores are configurable. Weir sets its Go memory limit to 90% of its container's cgroup v2 `memory.max` and refuses new work as live memory nears it, so keep a memory limit in `resources`. Without a limit, or on cgroup v1 nodes, Weir does not shed load. + +## Credentials + +Helm retains values in release history. Inline backend credentials are rejected, and so is any `@` in `mongodb.uri` or `search.url`, since it can only carry userinfo. MongoDB and Search use `authentication.username_file/password_file`; mount an externally managed Secret read-only using `extraVolumes/extraVolumeMounts` and reference it by absolute path, since relative paths resolve from `/etc/weir`; [examples/credential-files.yaml](../examples/credential-files.yaml) shows the layout. MongoDB requires SCRAM-SHA-256, authSource and TLS in the URI when authenticating. Both adapters can use `tls.ca_file`, or system trust when it is absent. Credential and CA files are read at startup; roll the Pods after rotating them, for example by changing a `podAnnotations` entry. + +## Validate before deployment + +Validate the generated `weir.yaml` with `weir check --config ...` from the pinned Weir source before deployment. Credential-file validation reads those supplied files and requires their authorization. ConfigMap updates automatically change the Pod checksum. diff --git a/docs/development.md b/docs/development.md new file mode 100644 index 0000000..8f7026b --- /dev/null +++ b/docs/development.md @@ -0,0 +1,27 @@ +# Development + +How to test the chart and publish a version. + +## Test and package + +```sh +python3 -m venv .venv +.venv/bin/pip install -r requirements-dev.txt +GOBIN="$PWD/.bin" go install github.com/batchstream/weir/cmd/weir@SOURCE_REVISION +make test PYTHON=.venv/bin/python WEIR_BIN="$PWD/.bin/weir" +make package PYTHON=.venv/bin/python WEIR_BIN="$PWD/.bin/weir" +``` + +`SOURCE_REVISION` is the `weir.batchstream.io/source-revision` annotation in `Chart.yaml`. `make test` lints the Chart, renders every example and validates the generated configuration with the selected Weir binary, including a scale check with many Stores and replicas and an isolated loopback-only serve, HTTP probe and shutdown run. With an official `promtool` on `PATH` or named by `PROMTOOL`, it also runs the rendered alerts and the autoscaling query against their PromQL fixtures; CI installs a checksum-verified promtool. It opens no backend connections, reads no existing Secrets and does not exercise CNI enforcement, Prometheus collection or KEDA scaling in a real cluster. + +## Upgrade test + +CI also runs `scripts/kind-upgrade.sh` on every change. In a kind cluster of its own, under load from `weir-soak` with `-no-errors` on both the current and the previous SDK, it upgrades a release from the Chart of the previous server through the Chart a change replaces to the new one, back and forward again at one replica, replaces the replica with one that starts 30 seconds late, scales to three replicas, takes the three back and forward again and scales back down. The change fails if a rollout does not complete, a caller saw any error or an outcome did not hold. While the previous SDK is v0.22.0 its client skips held streams, which that version fails in a rollout. Every rollout setting stays the Chart's default, and the test measures no throughput. The job fails if the weir-tests revision it builds the clients from pins other servers than these two Charts, so pin a new server there first. To run it locally, build `weir-soak` from [weir-tests](https://github.com/batchstream/weir-tests) for the cluster's platform and check out the commit to upgrade from; the script uses a kubeconfig of its own and ignores your current context and Helm settings. A second binary, built with the previous SDK and its protocol version as in the workflow, is optional; `PREVIOUS_PROFILES` sets its profiles, and `THROUGH_CHECKOUT` a Chart to pass through: + +```sh +scripts/kind-upgrade.sh FROM_CHECKOUT . path/to/weir-soak [path/to/previous-weir-soak] +``` + +## Release + +A reviewed `weir-VERSION` tag on `main` publishes the OCI Chart and a GitHub release with the archive and SHA256SUMS. The workflow refuses a version whose OCI Chart already exists and creates the release before pushing, so a rerun never overwrites a published Chart. If the push fails after the release exists, verify the release archive against its SHA256SUMS and `helm push` it by hand. Source annotation, image digest and CI source pins must identify the same server revision. diff --git a/docs/monitoring.md b/docs/monitoring.md new file mode 100644 index 0000000..9acd32b --- /dev/null +++ b/docs/monitoring.md @@ -0,0 +1,15 @@ +# Monitoring + +How to collect metrics and enable alerts and the dashboard. + +## Enable metrics + +Enable `metrics.enabled` and authorize scraper sources in `metrics.ingress`. The diagnostics listener is always bound for the probes; `metrics.enabled` adds the NetworkPolicy ingress for scrapers and a separate metrics Service. Reference the diagnostics port by its `metrics` name in ingress rules so a changed `diagnosticsPort` keeps matching. With default NetworkPolicy, metrics enablement requires explicit scraper rules. Annotations can support an existing scraper; `metrics.serviceMonitor.enabled` creates a ServiceMonitor when Prometheus Operator is already installed. Its selector matches exactly this release's metrics Service, and Prometheus discovers each Pod endpoint separately. `metrics.serviceMonitor.labels` must match your Prometheus instance's monitor selector; Chart identity/component labels cannot be overridden. + +## Labels and scrapers + +ServiceMonitor target relabeling supplies `namespace` and `weir_group="namespace/release"`. Native Weir metrics also carry `store` where applicable. With a custom scraper, supply the same stable labels and avoid duplicate scrape jobs. Scrape all group replicas; querying only the ordinary metrics ClusterIP would sample one Pod. + +## Alerts and dashboard + +`metrics.prometheusRule.enabled` adds a PrometheusRule with this group's alerts, and `metrics.grafanaDashboard.enabled` a ConfigMap holding a Grafana dashboard for the group. Both require `metrics.enabled`; their `labels` must match your Prometheus rule selector and Grafana's dashboard sidecar. Every expression selects the group by `namespace` and `weir_group`, so a custom scraper must supply those labels. [examples/prometheus-alerts.yaml](../examples/prometheus-alerts.yaml) enables the ServiceMonitor, alerts and dashboard together. `WeirGroupUnavailable` is critical: Prometheus sees no Ready replica of the group, which also covers failed scrapes. The other alerts are warnings: a replica Prometheus cannot scrape, sustained memory pressure, admission or Store rejections other than draining, unknown mutation outcomes, more than 5% of records ending with a failure, more than 5% of Native calls ending without a complete response, a p99 queue wait above one second and stalled RPCs. Each alert links to its runbook in Weir's [operations guide](https://github.com/batchstream/weir/blob/main/docs/operations.md#alerts). diff --git a/docs/operations.md b/docs/operations.md new file mode 100644 index 0000000..6161c7e --- /dev/null +++ b/docs/operations.md @@ -0,0 +1,39 @@ +# Operations + +How the chart handles traffic, rollouts, capacity, shutdown and probes. + +## Network + +Application traffic uses plaintext HTTP/2 inside a trusted, isolated network. Default NetworkPolicy permits same-namespace clients labeled `weir-client: "true"` to call the application port, plus DNS egress. Its named `grpc` rule follows `service.port` automatically (default 7447). Backend egress remains explicit. Verify the CNI enforces both allowed and denied traffic; additive allow-all policies can bypass restrictions. + +## Rolling updates + +A rolling update starts each replacement and waits for it before stopping a replica (`maxSurge: 1`, `maxUnavailable: 0`), at any replica count and under autoscaling, so an upgrade neither interrupts a single replica nor reduces capacity. A replacement that cannot start, such as a bad image or a cluster without room, leaves the old replicas serving, and the Deployment reports failure after `progressDeadlineSeconds`. A rollout needs room for one more Pod, for which a cluster autoscaler may add a node. Where a cluster or quota has none, set `strategy.rollingUpdate` to `maxSurge: 0` and `maxUnavailable: 1`: the replacement then races the old replica's shutdown, capacity drops by one during each replacement, and a single replica whose replacement is late has no server for as long as it is late, while calls fail with errors. + +## Replicas and disruption + +One replica still has downtime when its process or node fails. For three replicas use [examples/three-replicas.yaml](../examples/three-replicas.yaml); update its release-specific spread selector. Its `nodeTaintsPolicy: Honor` excludes untolerated nodes from the spread calculation. The optional PDB limits voluntary disruptions and does not establish physical fault tolerance by itself. + +## Capacity + +Kubernetes resource names and DNS endpoints follow their native format limits. Size groups and replicas from measured process, network and backend capacity, including starting and terminating instances. + +Each default Pod requests and limits 2 CPU / 3 GiB. Budget starting, running and terminating processes and backend connection demand together: a rollout runs one replica more than the group's count, plus Pods still terminating. Kubernetes does not impose a hard bound on concurrent terminating Pods, and old remote work can outlast a lost connection. Large workloads need measured admission/latency and backend capacity verification. + +## Rollout timing + +The Deployment waits `minReadySeconds: 15` after each replacement becomes Ready. Weir bounds its own startup at 30 seconds and usually listens within a second, so the startup probe runs every second and a replica becomes Ready as soon as it starts. The first probe can land anywhere in the first period, so the probe allows 32 failures, killing a Pod that has not started 31 to 32 seconds after its container starts. `progressDeadlineSeconds` is an explicit Deployment rollout budget (Chart default 180 seconds), including time for scheduling and image pulls. It must strictly exceed `minReadySeconds` plus that 32-second startup budget. Insufficient budgets fail validation; the Chart never silently changes the configured deadline. Deployment timing fields retain Kubernetes' int32 bounds. Minimum readiness time leaves space for clients to re-resolve the headless Service while old replicas remain. + +## Shutdown + +Before SIGTERM, kubelet's `preStop.sleep: 15` lets terminating Pod addresses leave Ready headless DNS without another shell/worker process. Weir then withdraws readiness and drains admitted work within `shutdownTimeoutSeconds` (Chart default 5). Each Store then receives Weir's fixed 2-second cleanup budget to cancel remaining work, end backend sessions and close connections; Stores clean up in parallel. With `terminationGracePeriodSeconds: 0`, the Pod grace is derived as preStop plus shutdown timeout plus 2 seconds of cleanup plus one second for final process exit (23 seconds by default). That extra second is the smallest integer slack above the bounded lifecycle phases; reserve more measured exit overhead with a positive override, which must cover the full derived budget. `shutdownTimeoutSeconds` is a positive integer and becomes Weir's `shutdown_timeout` duration string. The native sleep hook requires Kubernetes 1.30+'s default PodLifecycleSleepAction feature. Disabling the hook removes its retirement protection. + +## Probes + +Kubelet probes are HTTP GETs of `/readyz` (startup and readiness) and `/livez` (liveness) on `diagnosticsPort`, which Weir always binds on `[::]`. The same port serves `/metrics`; without `metrics.enabled`, the default NetworkPolicy admits no other sources to it. Readiness describes lifecycle; validate real business operations separately. A sent mutation without a terminal response has an unknown outcome and requires reconciliation before retrying. + +Readiness and liveness use Kubernetes' native 10-second probe period, one-second timeout and three consecutive failures. Startup uses the one-second period and the threshold above. Deployment revision retention uses Kubernetes' native default. These defaults are documented in the [probe API](https://kubernetes.io/docs/reference/kubernetes-api/workload-resources/pod-v1/#Probe) and [Deployment API](https://kubernetes.io/docs/concepts/workloads/controllers/deployment/#clean-up-policy). + +## Store and backend changes + +Changes to Store membership or backend identity require a coordinated cutover. An ordinary rolling update temporarily runs old and new definitions in one group and does not provide a transactional migration. Stop or redirect affected clients, drain the old replicas, apply the reviewed configuration/image pair and verify actual data operations before resuming. From f387bd668ff69cc0662cea5e78ee3eb6c71bee8f Mon Sep 17 00:00:00 2001 From: Randy <857271370@qq.com> Date: Wed, 4 Nov 2026 20:40:07 +0800 Subject: [PATCH 2/4] =?UTF-8?q?=F0=9F=93=9D=20docs:=20restructure=20the=20?= =?UTF-8?q?README=20around=20a=20runnable=20quick=20start?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit --- README.md | 168 +++++++++++++++++++++++++++++++++++------------------- 1 file changed, 110 insertions(+), 58 deletions(-) diff --git a/README.md b/README.md index 848d215..2f0330c 100644 --- a/README.md +++ b/README.md @@ -2,99 +2,151 @@ [![CI](https://github.com/batchstream/weir-charts/actions/workflows/ci.yaml/badge.svg?branch=main)](https://github.com/batchstream/weir-charts/actions/workflows/ci.yaml) [![License: MIT](https://img.shields.io/badge/License-MIT-blue.svg)](LICENSE) -Deploy [Weir](https://github.com/batchstream/weir), a synchronous MongoDB and Search data plane, on Kubernetes. Helm 3.17+ and Kubernetes 1.30+ are required. Chart `0.20.0-rc.1` targets server source [commit `5107986b882d`](https://github.com/batchstream/weir/commit/5107986b882da60751403c7e05bbbebdb12b3cdb). `Chart.yaml` records that source contract and `values.yaml` pins the immutable image `ghcr.io/batchstream/weir@sha256:b5d949cefb0ba01e18cff2b5210cef954bc387c20cffa87784466e793e0e088e`; both validation and release workflows install that exact source with `go install`. +Helm chart that deploys [Weir](https://github.com/batchstream/weir), a synchronous +MongoDB and Search data plane, on Kubernetes. -One Helm release is one Store group. Every replica hosts the same Store definitions; a group can host several Stores, and each group is released and scaled independently. Replicas share nothing with each other. Each Pod runs one Weir process with Lua inside that process. Backends, collections, indices, credentials and backups are provisioned separately. +## Overview -## Store groups +### Terms -Declare this group's Stores directly in `stores`: +| Term | Meaning | +| --- | --- | +| Store | A named backend connection, declared in `stores`. | +| Search | Elasticsearch or OpenSearch. | +| Store group | One Helm release. A group can host several Stores and is released and scaled independently. | +| Replica | One Pod running one Weir process. Every replica of a group hosts the same Stores. Replicas share nothing. | -```yaml -stores: - - name: primary - mongodb: - uri: mongodb://mongo:27017/?directConnection=true - - name: search - search: - url: http://search:9200 - batching: - max_wait: 5ms -``` - -Each entry uses Weir's own Store schema: `name`, exactly one of `mongodb.uri` or `search.url`, optional file-based `authentication`, optional `tls.ca_file` and optional `batching` (`max_operations`, `max_bytes`, `max_wait`, `max_in_flight`, `min_in_flight`). Database, collection and index targets come from request URIs; Chart configuration declares the backend connection for each Store. Multiple replicas within a group use identical Stores and settings. A request for a Store the group does not host fails with NotFound. +### Responsibilities -Install [examples/mongo.yaml](examples/mongo.yaml) and [examples/search.yaml](examples/search.yaml) as separate groups in the same namespace: +| Provided by | Items | +| --- | --- | +| The chart | Deployment, headless Service, ConfigMap, NetworkPolicy. Optional: metrics Service, ServiceMonitor, PrometheusRule, Grafana dashboard, PodDisruptionBudget, KEDA ScaledObject. | +| You | Backends, collections, indices, credentials and backups. | -```sh -helm upgrade --install mongo ./charts/weir --namespace application \ - --kube-context your-context -f examples/mongo.yaml --atomic --wait --timeout 5m -helm upgrade --install search ./charts/weir --namespace application \ - --kube-context your-context -f examples/search.yaml --atomic --wait --timeout 5m -``` +### Pinned versions -The Chart generates `weir.yaml` with the listeners, `shutdown_timeout` and `stores`. Clients use [weir-go](https://github.com/batchstream/weir-go) v0.23.0, which speaks this server's protocol, and connect with `weir.New` to the group's headless Service by its full name with a trailing dot, `-weir..svc.cluster.local.:`, with the cluster's own domain if it is not `cluster.local`; without the dot every lookup first tries the Pod's DNS search domains. The name resolves to every Ready Pod, and the SDK spreads business RPCs across them. The SDK looks the name up every 5 seconds, so Pods added by a scale-out or rollout take their share of calls within seconds of becoming Ready, and a terminating Pod stops receiving calls once it leaves DNS, before SIGTERM. It also pings a replica it has heard nothing from for 10 seconds and leaves one that does not answer; this server accepts those pings, so upgrade the group before its clients. Generated `[::]` listeners accept IPv4 and IPv6 on the supported Go runtime. Group identity is `namespace/release`, so metric labels remain stable when `fullnameOverride` changes. The Service selects only its release. Allow clients to reach each group's application port. +| Item | Version | +| --- | --- | +| Chart | `0.20.0-rc.1` | +| Server source | [commit `5107986b882d`](https://github.com/batchstream/weir/commit/5107986b882da60751403c7e05bbbebdb12b3cdb) | +| Server image | `ghcr.io/batchstream/weir@sha256:b5d949cefb0ba01e18cff2b5210cef954bc387c20cffa87784466e793e0e088e` | +| Go SDK | [weir-go](https://github.com/batchstream/weir-go) v0.23.0 | -## Configuration and credentials +`Chart.yaml` records the server source and `values.yaml` pins the image. Both the +validation and release workflows install that exact source with `go install`. -The Chart renders one `weir.yaml` into its ConfigMap and mounts it read-only at `/etc/weir`. Its schema rejects unknown keys; the selected Weir binary validates backend semantics. Weir fixes transport, memory-pressure, Lua, queue and retry policy internally, so only listeners, the shutdown drain and Stores are configurable. Weir sets its Go memory limit to 90% of its container's cgroup v2 `memory.max` and refuses new work as live memory nears it, so keep a memory limit in `resources`. Without a limit, or on cgroup v1 nodes, Weir does not shed load. +## Requirements -Helm retains values in release history. Inline backend credentials are rejected, and so is any `@` in `mongodb.uri` or `search.url`, since it can only carry userinfo. MongoDB and Search use `authentication.username_file/password_file`; mount an externally managed Secret read-only using `extraVolumes/extraVolumeMounts` and reference it by absolute path, since relative paths resolve from `/etc/weir`; [examples/credential-files.yaml](examples/credential-files.yaml) shows the layout. MongoDB requires SCRAM-SHA-256, authSource and TLS in the URI when authenticating. Both adapters can use `tls.ca_file`, or system trust when it is absent. Credential and CA files are read at startup; roll the Pods after rotating them, for example by changing a `podAnnotations` entry. +| Item | Requirement | +| --- | --- | +| Helm | 3.17 or later | +| Kubernetes | 1.30 or later | +| Backend | MongoDB or Search, reachable from the namespace | +| Network | The application port uses plaintext HTTP/2. Use an isolated network and a CNI that enforces NetworkPolicy. | +| Optional | Prometheus Operator for monitoring. KEDA for autoscaling. | -Validate the generated `weir.yaml` with `weir check --config ...` from the pinned Weir source before deployment. Credential-file validation reads those supplied files and requires their authorization. ConfigMap updates automatically change the Pod checksum. +## Quick start -## Prometheus and KEDA +This installs one Store group with one MongoDB Store in the namespace +`application`. -Enable `metrics.enabled` and authorize scraper sources in `metrics.ingress`. The diagnostics listener is always bound for the probes; `metrics.enabled` adds the NetworkPolicy ingress for scrapers and a separate metrics Service. Reference the diagnostics port by its `metrics` name in ingress rules so a changed `diagnosticsPort` keeps matching. With default NetworkPolicy, metrics enablement requires explicit scraper rules. Annotations can support an existing scraper; `metrics.serviceMonitor.enabled` creates a ServiceMonitor when Prometheus Operator is already installed. Its selector matches exactly this release's metrics Service, and Prometheus discovers each Pod endpoint separately. `metrics.serviceMonitor.labels` must match your Prometheus instance's monitor selector; Chart identity/component labels cannot be overridden. +**Step 1.** Get the chart. -ServiceMonitor target relabeling supplies `namespace` and `weir_group="namespace/release"`. Native Weir metrics also carry `store` where applicable. With a custom scraper, supply the same stable labels and avoid duplicate scrape jobs. Scrape all group replicas; querying only the ordinary metrics ClusterIP would sample one Pod. - -`metrics.prometheusRule.enabled` adds a PrometheusRule with this group's alerts, and `metrics.grafanaDashboard.enabled` a ConfigMap holding a Grafana dashboard for the group. Both require `metrics.enabled`; their `labels` must match your Prometheus rule selector and Grafana's dashboard sidecar. Every expression selects the group by `namespace` and `weir_group`, so a custom scraper must supply those labels. [examples/prometheus-alerts.yaml](examples/prometheus-alerts.yaml) enables the ServiceMonitor, alerts and dashboard together. `WeirGroupUnavailable` is critical: Prometheus sees no Ready replica of the group, which also covers failed scrapes. The other alerts are warnings: a replica Prometheus cannot scrape, sustained memory pressure, admission or Store rejections other than draining, unknown mutation outcomes, more than 5% of records ending with a failure, more than 5% of Native calls ending without a complete response, a p99 queue wait above one second and stalled RPCs. Each alert links to its runbook in Weir's [operations guide](https://github.com/batchstream/weir/blob/main/docs/operations.md#alerts). - -Overlay [examples/prometheus-autoscaling.yaml](examples/prometheus-autoscaling.yaml) on the `mongo` release in namespace `application`. Adjust the namespace, release identity, Prometheus endpoint and scraper labels for your deployment. Its query sums this exact group's Weir process CPU; `AverageValue` lets HPA divide that total across replicas, and the threshold of 1.4 cores is 70% of the default per-replica limit, so change it with `resources`. Scale on Weir's own CPU, not on queued operations or active executions: those rise whenever the backend slows down, and replicas added then bring no capacity, so a queue trigger holds the group at its maximum. Each replica runs fewer record batches at once while executions queue in its backend, so the group's combined requests follow what the backend can take rather than the replica count; a new replica starts at two batches per operation for a little over one probe interval after its first execution, so replicas added in front of a queueing backend do not flood it; the dashboard shows that limit and the latency ratio behind it. Set `maxReplicaCount` from the backend's connection budget: each replica runs up to 16 record batches and 64 Natives per Store at once. - -The example guards its aggregate with `and on() (min(up{namespace="application",weir_group="application/mongo"}) == 1)`. A failed discovered scrape target or no discovered targets produces an empty result; `ignoreNullValues: "false"` then leaves KEDA in error instead of treating an incomplete lower workload as a valid scale-down signal. The HTTP timeout is 10 seconds, and no `or vector(0)` supplies a false zero. This guard covers the targets Prometheus has discovered; it cannot prove that scraper configuration includes every group Pod. EndpointSlice discovery keeps a terminating Pod as a target until it is deleted, so its last scrape after exit fails and briefly turns the query into an error; drop endpoints whose `terminating` condition is true, as the default Endpoints discovery already does. Before enabling autoscaling, compare the per-Pod targets with all running/Ready group replicas and verify the labels, `process_cpu_seconds_total` and `up` are present; repeat that coverage check after scale and rollout changes. Configure an existing `TriggerAuthentication`/`ClusterTriggerAuthentication` through `authenticationRef` if needed. No controller credentials are rendered or read by this Chart. KEDA's operator, metrics server and relevant network rules must already allow its Prometheus query. See [the Prometheus scaler contract](https://keda.sh/docs/2.20/scalers/prometheus/) and [ScaledObject specification](https://keda.sh/docs/2.20/reference/scaledobject-spec/). +```sh +git clone https://github.com/batchstream/weir-charts.git +cd weir-charts +``` -`autoscaling.triggers` accepts native KEDA triggers, including Prometheus and other supported scalers. Prometheus triggers require metrics to be enabled; CRDs/controllers are external prerequisites. `pollingInterval`, `cooldownPeriod` and `scaleDown` configure polling and HPA scale-down behavior. The cooldown setting concerns KEDA activation/scale-to-zero; ordinary reduction between running replica counts is controlled by HPA behavior. Minimum replicas must stay at least one: SDK connections cannot wake a group with zero replicas. A PDB requires at least two minimum replicas. Every upgrade or rollback that turns autoscaling on removes the Chart's replica count, so Kubernetes resets the Deployment to one replica until KEDA's HPA scales it to `minReplicaCount`: from three replicas with a minimum of two, two Pods were terminating within 20 seconds and the second Ready replica returned after about a minute. Turn autoscaling on when one replica can carry the load, with `minReplicaCount` at least the running count. `kubectl apply` and GitOps tools reset the count the same way when they turn autoscaling on. Upgrades that keep autoscaling on leave the count to KEDA. Verify the ScaledObject becomes Ready and HPA reaches its minimum before admitting production traffic; Helm readiness alone does not establish controller readiness. Disabling KEDA resumes the configured `replicaCount`. +**Step 2.** Check `examples/mongo.yaml`. It expects a MongoDB replica-set member +at `mongo:27017` on Pods labeled `app: mongo`. Edit the file if yours differ. -## Network and operations +```yaml +stores: +- name: mongo + mongodb: + uri: mongodb://mongo:27017/?directConnection=true +networkPolicy: + egress: + - to: + - podSelector: + matchLabels: + app: mongo + ports: + - protocol: TCP + port: 27017 +``` -Application traffic uses plaintext HTTP/2 inside a trusted, isolated network. Default NetworkPolicy permits same-namespace clients labeled `weir-client: "true"` to call the application port, plus DNS egress. Its named `grpc` rule follows `service.port` automatically (default 7447). Backend egress remains explicit. Verify the CNI enforces both allowed and denied traffic; additive allow-all policies can bypass restrictions. +**Step 3.** Install the release. Replace `your-context` with your cluster context. -A rolling update starts each replacement and waits for it before stopping a replica (`maxSurge: 1`, `maxUnavailable: 0`), at any replica count and under autoscaling, so an upgrade neither interrupts a single replica nor reduces capacity. A replacement that cannot start, such as a bad image or a cluster without room, leaves the old replicas serving, and the Deployment reports failure after `progressDeadlineSeconds`. A rollout needs room for one more Pod, for which a cluster autoscaler may add a node. Where a cluster or quota has none, set `strategy.rollingUpdate` to `maxSurge: 0` and `maxUnavailable: 1`: the replacement then races the old replica's shutdown, capacity drops by one during each replacement, and a single replica whose replacement is late has no server for as long as it is late, while calls fail with errors. +```sh +helm upgrade --install mongo ./charts/weir --namespace application --create-namespace \ + --kube-context your-context -f examples/mongo.yaml --atomic --wait --timeout 5m +``` -One replica still has downtime when its process or node fails. For three replicas use [examples/three-replicas.yaml](examples/three-replicas.yaml); update its release-specific spread selector. Its `nodeTaintsPolicy: Honor` excludes untolerated nodes from the spread calculation. The optional PDB limits voluntary disruptions and does not establish physical fault tolerance by itself. +**Step 4.** Allow your client Pods to connect. They must run in the same namespace +and carry this label. -Kubernetes resource names and DNS endpoints follow their native format limits. Size groups and replicas from measured process, network and backend capacity, including starting and terminating instances. +```yaml +metadata: + labels: + weir-client: "true" +``` -Each default Pod requests and limits 2 CPU / 3 GiB. Budget starting, running and terminating processes and backend connection demand together: a rollout runs one replica more than the group's count, plus Pods still terminating. Kubernetes does not impose a hard bound on concurrent terminating Pods, and old remote work can outlast a lost connection. Large workloads need measured admission/latency and backend capacity verification. +**Step 5.** Connect with the SDK, using the address that Helm prints. Keep the +trailing dot. -The Deployment waits `minReadySeconds: 15` after each replacement becomes Ready. Weir bounds its own startup at 30 seconds and usually listens within a second, so the startup probe runs every second and a replica becomes Ready as soon as it starts. The first probe can land anywhere in the first period, so the probe allows 32 failures, killing a Pod that has not started 31 to 32 seconds after its container starts. `progressDeadlineSeconds` is an explicit Deployment rollout budget (Chart default 180 seconds), including time for scheduling and image pulls. It must strictly exceed `minReadySeconds` plus that 32-second startup budget. Insufficient budgets fail validation; the Chart never silently changes the configured deadline. Deployment timing fields retain Kubernetes' int32 bounds. Minimum readiness time leaves space for clients to re-resolve the headless Service while old replicas remain. +```go +client, err := weir.New("mongo-weir.application.svc.cluster.local.:7447") +``` -Before SIGTERM, kubelet's `preStop.sleep: 15` lets terminating Pod addresses leave Ready headless DNS without another shell/worker process. Weir then withdraws readiness and drains admitted work within `shutdownTimeoutSeconds` (Chart default 5). Each Store then receives Weir's fixed 2-second cleanup budget to cancel remaining work, end backend sessions and close connections; Stores clean up in parallel. With `terminationGracePeriodSeconds: 0`, the Pod grace is derived as preStop plus shutdown timeout plus 2 seconds of cleanup plus one second for final process exit (23 seconds by default). That extra second is the smallest integer slack above the bounded lifecycle phases; reserve more measured exit overhead with a positive override, which must cover the full derived budget. `shutdownTimeoutSeconds` is a positive integer and becomes Weir's `shutdown_timeout` duration string. The native sleep hook requires Kubernetes 1.30+'s default PodLifecycleSleepAction feature. Disabling the hook removes its retirement protection. +See the [Go SDK quick start](https://github.com/batchstream/weir-go#quick-start) +for a complete client. -Kubelet probes are HTTP GETs of `/readyz` (startup and readiness) and `/livez` (liveness) on `diagnosticsPort`, which Weir always binds on `[::]`. The same port serves `/metrics`; without `metrics.enabled`, the default NetworkPolicy admits no other sources to it. Readiness describes lifecycle; validate real business operations separately. A sent mutation without a terminal response has an unknown outcome and requires reconciliation before retrying. +## Documentation -Readiness and liveness use Kubernetes' native 10-second probe period, one-second timeout and three consecutive failures. Startup uses the one-second period and the threshold above. Deployment revision retention uses Kubernetes' native default. These defaults are documented in the [probe API](https://kubernetes.io/docs/reference/kubernetes-api/workload-resources/pod-v1/#Probe) and [Deployment API](https://kubernetes.io/docs/concepts/workloads/controllers/deployment/#clean-up-policy). +| Guide | Contents | +| --- | --- | +| [Configuration](docs/configuration.md) | Store groups, client connections, generated configuration and credentials | +| [Monitoring](docs/monitoring.md) | Metrics, labels, alerts and the Grafana dashboard | +| [Autoscaling](docs/autoscaling.md) | KEDA and Prometheus setup | +| [Operations](docs/operations.md) | Network, rolling updates, capacity, shutdown and probes | +| [Development](docs/development.md) | Chart tests, the upgrade test and releases | -Changes to Store membership or backend identity require a coordinated cutover. An ordinary rolling update temporarily runs old and new definitions in one group and does not provide a transactional migration. Stop or redirect affected clients, drain the old replicas, apply the reviewed configuration/image pair and verify actual data operations before resuming. +| Example | Shows | +| --- | --- | +| [mongo.yaml](examples/mongo.yaml) | A group with a MongoDB Store | +| [search.yaml](examples/search.yaml) | A group with a Search Store | +| [credential-files.yaml](examples/credential-files.yaml) | Backend credentials from a mounted Secret | +| [three-replicas.yaml](examples/three-replicas.yaml) | Three replicas spread across nodes | +| [prometheus-alerts.yaml](examples/prometheus-alerts.yaml) | ServiceMonitor, alerts and dashboard | +| [prometheus-autoscaling.yaml](examples/prometheus-autoscaling.yaml) | KEDA autoscaling on Weir's own CPU | -## Develop and release +## Development ```sh python3 -m venv .venv .venv/bin/pip install -r requirements-dev.txt GOBIN="$PWD/.bin" go install github.com/batchstream/weir/cmd/weir@SOURCE_REVISION make test PYTHON=.venv/bin/python WEIR_BIN="$PWD/.bin/weir" -make package PYTHON=.venv/bin/python WEIR_BIN="$PWD/.bin/weir" ``` -`SOURCE_REVISION` is the `weir.batchstream.io/source-revision` annotation in `Chart.yaml`. `make test` lints the Chart, renders every example and validates the generated configuration with the selected Weir binary, including a scale check with many Stores and replicas and an isolated loopback-only serve, HTTP probe and shutdown run. With an official `promtool` on `PATH` or named by `PROMTOOL`, it also runs the rendered alerts and the autoscaling query against their PromQL fixtures; CI installs a checksum-verified promtool. It opens no backend connections, reads no existing Secrets and does not exercise CNI enforcement, Prometheus collection or KEDA scaling in a real cluster. +`SOURCE_REVISION` is the `weir.batchstream.io/source-revision` annotation in +`Chart.yaml`. See [Development](docs/development.md) for packaging and the upgrade +test. -CI also runs `scripts/kind-upgrade.sh` on every change. In a kind cluster of its own, under load from `weir-soak` with `-no-errors` on both the current and the previous SDK, it upgrades a release from the Chart of the previous server through the Chart a change replaces to the new one, back and forward again at one replica, replaces the replica with one that starts 30 seconds late, scales to three replicas, takes the three back and forward again and scales back down. The change fails if a rollout does not complete, a caller saw any error or an outcome did not hold. While the previous SDK is v0.22.0 its client skips held streams, which that version fails in a rollout. Every rollout setting stays the Chart's default, and the test measures no throughput. The job fails if the weir-tests revision it builds the clients from pins other servers than these two Charts, so pin a new server there first. To run it locally, build `weir-soak` from [weir-tests](https://github.com/batchstream/weir-tests) for the cluster's platform and check out the commit to upgrade from; the script uses a kubeconfig of its own and ignores your current context and Helm settings. A second binary, built with the previous SDK and its protocol version as in the workflow, is optional; `PREVIOUS_PROFILES` sets its profiles, and `THROUGH_CHECKOUT` a Chart to pass through: +- Contributing: [CONTRIBUTING.md](CONTRIBUTING.md) +- Security reports: [SECURITY.md](SECURITY.md) -```sh -scripts/kind-upgrade.sh FROM_CHECKOUT . path/to/weir-soak [path/to/previous-weir-soak] -``` +## Related repositories + +| Repository | Purpose | +| --- | --- | +| [weir](https://github.com/batchstream/weir) | Server | +| [weir-go](https://github.com/batchstream/weir-go) | Go client SDK | +| [weir-protocol](https://github.com/batchstream/weir-protocol) | Public gRPC schema and validation | +| **[weir-charts](https://github.com/batchstream/weir-charts)** | Helm chart (this repository) | +| [weir-tests](https://github.com/batchstream/weir-tests) | System tests and benchmarks | + +## License -A reviewed `weir-VERSION` tag on `main` publishes the OCI Chart and a GitHub release with the archive and SHA256SUMS. The workflow refuses a version whose OCI Chart already exists and creates the release before pushing, so a rerun never overwrites a published Chart. If the push fails after the release exists, verify the release archive against its SHA256SUMS and `helm push` it by hand. Source annotation, image digest and CI source pins must identify the same server revision. +MIT. See [LICENSE](LICENSE). From 7087f702adb8f8a4ad5e5b6eba98ef1965c9fb2f Mon Sep 17 00:00:00 2001 From: Randy <857271370@qq.com> Date: Wed, 4 Nov 2026 20:50:29 +0800 Subject: [PATCH 3/4] =?UTF-8?q?=F0=9F=93=9D=20docs:=20state=20what=20the?= =?UTF-8?q?=20chart=20quick=20start=20needs=20from=20the=20backend?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit --- README.md | 42 ++++++++++++++++++++++++++++++------------ docs/configuration.md | 4 ++-- 2 files changed, 32 insertions(+), 14 deletions(-) diff --git a/README.md b/README.md index 2f0330c..cab7bbe 100644 --- a/README.md +++ b/README.md @@ -39,11 +39,15 @@ validation and release workflows install that exact source with `go install`. | Item | Requirement | | --- | --- | -| Helm | 3.17 or later | +| Helm | 3.17 or later. CI tests 3.17.0. | | Kubernetes | 1.30 or later | -| Backend | MongoDB or Search, reachable from the namespace | +| Cluster capacity | Each Pod requests and limits 2 CPU and 3 GiB by default. A rollout runs one extra Pod. | +| MongoDB | A nonsharded replica set with the client `bulkWrite` command (MongoDB 8.0 or later). | +| Search | Elasticsearch or OpenSearch. | +| Collections and indices | Must exist before typed reads and mutations. | | Network | The application port uses plaintext HTTP/2. Use an isolated network and a CNI that enforces NetworkPolicy. | | Optional | Prometheus Operator for monitoring. KEDA for autoscaling. | +| Development | Go 1.27.3 and Python 3, to run the chart tests. | ## Quick start @@ -57,8 +61,10 @@ git clone https://github.com/batchstream/weir-charts.git cd weir-charts ``` -**Step 2.** Check `examples/mongo.yaml`. It expects a MongoDB replica-set member -at `mongo:27017` on Pods labeled `app: mongo`. Edit the file if yours differ. +**Step 2.** Check `examples/mongo.yaml`. It expects MongoDB in the namespace +`application`, with its primary at `mongo:27017` on Pods labeled `app: mongo`, and +no authentication. Edit the file if yours differ. For authentication, see +[Credentials](docs/configuration.md#credentials). ```yaml stores: @@ -76,20 +82,30 @@ networkPolicy: port: 27017 ``` +This example uses MongoDB direct mode, which has no failover. For production, use +a replica-set URI. See +[MongoDB connection modes](https://github.com/batchstream/weir/blob/main/docs/configuration.md#mongodb). + **Step 3.** Install the release. Replace `your-context` with your cluster context. ```sh -helm upgrade --install mongo ./charts/weir --namespace application --create-namespace \ +helm upgrade --install mongo ./charts/weir --namespace application \ --kube-context your-context -f examples/mongo.yaml --atomic --wait --timeout 5m ``` +Helm reports `STATUS: deployed` and prints the SDK address. With `--atomic`, Helm +removes a failed install after the timeout. Omit `--atomic` to keep the Pods for +inspection. + **Step 4.** Allow your client Pods to connect. They must run in the same namespace -and carry this label. +and carry this label on the Pod template. ```yaml -metadata: - labels: - weir-client: "true" +spec: + template: + metadata: + labels: + weir-client: "true" ``` **Step 5.** Connect with the SDK, using the address that Helm prints. Keep the @@ -99,8 +115,10 @@ trailing dot. client, err := weir.New("mongo-weir.application.svc.cluster.local.:7447") ``` -See the [Go SDK quick start](https://github.com/batchstream/weir-go#quick-start) -for a complete client. +To try it, run the program from the +[Go SDK quick start](https://github.com/batchstream/weir-go#quick-start) in that Pod +with this address. The program uses the collection `weir_example.records`, which +must exist first. ## Documentation @@ -117,7 +135,7 @@ for a complete client. | [mongo.yaml](examples/mongo.yaml) | A group with a MongoDB Store | | [search.yaml](examples/search.yaml) | A group with a Search Store | | [credential-files.yaml](examples/credential-files.yaml) | Backend credentials from a mounted Secret | -| [three-replicas.yaml](examples/three-replicas.yaml) | Three replicas spread across nodes | +| [three-replicas.yaml](examples/three-replicas.yaml) | Three replicas spread across nodes. Update its selector for your release. | | [prometheus-alerts.yaml](examples/prometheus-alerts.yaml) | ServiceMonitor, alerts and dashboard | | [prometheus-autoscaling.yaml](examples/prometheus-autoscaling.yaml) | KEDA autoscaling on Weir's own CPU | diff --git a/docs/configuration.md b/docs/configuration.md index 4a52ee5..d99b1fb 100644 --- a/docs/configuration.md +++ b/docs/configuration.md @@ -37,11 +37,11 @@ helm upgrade --install search ./charts/weir --namespace application \ ## Client connections -The Chart generates `weir.yaml` with the listeners, `shutdown_timeout` and `stores`. Clients use [weir-go](https://github.com/batchstream/weir-go) v0.23.0, which speaks this server's protocol, and connect with `weir.New` to the group's headless Service by its full name with a trailing dot, `-weir..svc.cluster.local.:`, with the cluster's own domain if it is not `cluster.local`; without the dot every lookup first tries the Pod's DNS search domains. The name resolves to every Ready Pod, and the SDK spreads business RPCs across them. The SDK looks the name up every 5 seconds, so Pods added by a scale-out or rollout take their share of calls within seconds of becoming Ready, and a terminating Pod stops receiving calls once it leaves DNS, before SIGTERM. It also pings a replica it has heard nothing from for 10 seconds and leaves one that does not answer; this server accepts those pings, so upgrade the group before its clients. Generated `[::]` listeners accept IPv4 and IPv6 on the supported Go runtime. Group identity is `namespace/release`, so metric labels remain stable when `fullnameOverride` changes. The Service selects only its release. Allow clients to reach each group's application port. +Clients use [weir-go](https://github.com/batchstream/weir-go) v0.23.0, which speaks this server's protocol, and connect with `weir.New` to the group's headless Service by its full name with a trailing dot, `-weir..svc.cluster.local.:`, with the cluster's own domain if it is not `cluster.local`; without the dot every lookup first tries the Pod's DNS search domains. The name resolves to every Ready Pod, and the SDK spreads business RPCs across them. The SDK looks the name up every 5 seconds, so Pods added by a scale-out or rollout take their share of calls within seconds of becoming Ready, and a terminating Pod stops receiving calls once it leaves DNS, before SIGTERM. It also pings a replica it has heard nothing from for 10 seconds and leaves one that does not answer; this server accepts those pings, so upgrade the group before its clients. Generated `[::]` listeners accept IPv4 and IPv6 on the supported Go runtime. Group identity is `namespace/release`, so metric labels remain stable when `fullnameOverride` changes. The Service selects only its release. Allow clients to reach each group's application port. ## Generated configuration -The Chart renders one `weir.yaml` into its ConfigMap and mounts it read-only at `/etc/weir`. Its schema rejects unknown keys; the selected Weir binary validates backend semantics. Weir fixes transport, memory-pressure, Lua, queue and retry policy internally, so only listeners, the shutdown drain and Stores are configurable. Weir sets its Go memory limit to 90% of its container's cgroup v2 `memory.max` and refuses new work as live memory nears it, so keep a memory limit in `resources`. Without a limit, or on cgroup v1 nodes, Weir does not shed load. +The Chart generates `weir.yaml` with the listeners, `shutdown_timeout` and `stores`. The Chart renders one `weir.yaml` into its ConfigMap and mounts it read-only at `/etc/weir`. Its schema rejects unknown keys; the selected Weir binary validates backend semantics. Weir fixes transport, memory-pressure, Lua, queue and retry policy internally, so only listeners, the shutdown drain and Stores are configurable. Weir sets its Go memory limit to 90% of its container's cgroup v2 `memory.max` and refuses new work as live memory nears it, so keep a memory limit in `resources`. Without a limit, or on cgroup v1 nodes, Weir does not shed load. ## Credentials From c9cb63fc2cadfd6c7839cc6f66872a352d2ac187 Mon Sep 17 00:00:00 2001 From: Randy <857271370@qq.com> Date: Wed, 4 Nov 2026 20:52:12 +0800 Subject: [PATCH 4/4] =?UTF-8?q?=F0=9F=93=9D=20docs:=20name=20the=20license?= =?UTF-8?q?d=20SDK=20release?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit --- README.md | 2 +- docs/configuration.md | 2 +- 2 files changed, 2 insertions(+), 2 deletions(-) diff --git a/README.md b/README.md index cab7bbe..f6fb064 100644 --- a/README.md +++ b/README.md @@ -30,7 +30,7 @@ MongoDB and Search data plane, on Kubernetes. | Chart | `0.20.0-rc.1` | | Server source | [commit `5107986b882d`](https://github.com/batchstream/weir/commit/5107986b882da60751403c7e05bbbebdb12b3cdb) | | Server image | `ghcr.io/batchstream/weir@sha256:b5d949cefb0ba01e18cff2b5210cef954bc387c20cffa87784466e793e0e088e` | -| Go SDK | [weir-go](https://github.com/batchstream/weir-go) v0.23.0 | +| Go SDK | [weir-go](https://github.com/batchstream/weir-go) v0.23.1 | `Chart.yaml` records the server source and `values.yaml` pins the image. Both the validation and release workflows install that exact source with `go install`. diff --git a/docs/configuration.md b/docs/configuration.md index d99b1fb..1b6a725 100644 --- a/docs/configuration.md +++ b/docs/configuration.md @@ -37,7 +37,7 @@ helm upgrade --install search ./charts/weir --namespace application \ ## Client connections -Clients use [weir-go](https://github.com/batchstream/weir-go) v0.23.0, which speaks this server's protocol, and connect with `weir.New` to the group's headless Service by its full name with a trailing dot, `-weir..svc.cluster.local.:`, with the cluster's own domain if it is not `cluster.local`; without the dot every lookup first tries the Pod's DNS search domains. The name resolves to every Ready Pod, and the SDK spreads business RPCs across them. The SDK looks the name up every 5 seconds, so Pods added by a scale-out or rollout take their share of calls within seconds of becoming Ready, and a terminating Pod stops receiving calls once it leaves DNS, before SIGTERM. It also pings a replica it has heard nothing from for 10 seconds and leaves one that does not answer; this server accepts those pings, so upgrade the group before its clients. Generated `[::]` listeners accept IPv4 and IPv6 on the supported Go runtime. Group identity is `namespace/release`, so metric labels remain stable when `fullnameOverride` changes. The Service selects only its release. Allow clients to reach each group's application port. +Clients use [weir-go](https://github.com/batchstream/weir-go) v0.23.1, which speaks this server's protocol, and connect with `weir.New` to the group's headless Service by its full name with a trailing dot, `-weir..svc.cluster.local.:`, with the cluster's own domain if it is not `cluster.local`; without the dot every lookup first tries the Pod's DNS search domains. The name resolves to every Ready Pod, and the SDK spreads business RPCs across them. The SDK looks the name up every 5 seconds, so Pods added by a scale-out or rollout take their share of calls within seconds of becoming Ready, and a terminating Pod stops receiving calls once it leaves DNS, before SIGTERM. It also pings a replica it has heard nothing from for 10 seconds and leaves one that does not answer; this server accepts those pings, so upgrade the group before its clients. Generated `[::]` listeners accept IPv4 and IPv6 on the supported Go runtime. Group identity is `namespace/release`, so metric labels remain stable when `fullnameOverride` changes. The Service selects only its release. Allow clients to reach each group's application port. ## Generated configuration