Skip to content
Open
8 changes: 7 additions & 1 deletion docs/security.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -89,7 +89,7 @@ When `postgresql.credentials.existingSecret` is empty, the chart generates a Sec
- The Metadata Service Deployment loads the credentials by mounting the Secret at `postgresql.credentials.mountPath`.
- When `postgresql.local_enabled: true`, the bundled PostgreSQL StatefulSet also reads this generated Secret.

To use a Secret you create yourself, use an external PostgreSQL database and set `postgresql.local_enabled: false` with `postgresql.credentials.existingSecret`.
For either bundled or external PostgreSQL, set `postgresql.credentials.existingSecret` to use a Secret you create yourself. The bundled database requires `username`, `password`, and `database` keys, with username and database matching the chart values. Updating the Secret does not change credentials inside an already-initialized database.

### External PostgreSQL

Expand Down Expand Up @@ -122,3 +122,9 @@ When engine query-listener TLS is enabled (`tls.engine.enabled`), the gateway's
- [Monitoring](./monitoring): metrics surface and Prometheus Operator integration.
- [Operator upgrade path](./operator-upgrade-path): when continuous admission, drift correction, and CR-status surfaces are required.
- [`SECURITY.md`](https://github.com/firebolt-db/firebolt-instance-helm/blob/main/SECURITY.md): vulnerability reporting policy for this repository.

## Render-time safeguards

The chart rejects duplicate Engine names and signing key IDs, generated pod or Service names longer than 63 characters, and custom environment variables, volumes, mounts, or annotations that collide with chart-owned fields. Mount validation also rejects duplicate destinations and paths that shadow a chart-owned directory or one of its subdirectories. Annotations under `checksum/` and `firebolt.io/` are reserved for rollout tracking. Contents of `envFrom` Secrets are not inspected at render time.

Treat `customEngineConfig.instance.id` as installation identity. Online Helm upgrades reject changing an existing identity. Offline/GitOps rendering cannot inspect the installed ConfigMap, so your deployment process must preserve the value. Changing the ID is not an account migration procedure.
18 changes: 17 additions & 1 deletion docs/usage/external-postgres.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -61,5 +61,21 @@ kubectl -n firebolt logs deployment/firebolt-metadata-service | grep -i postgres
## Caveats

- TLS to PostgreSQL is not currently exposed. There is no `sslmode` value or CA-bundle mount.
- `host`, `database`, and `schema` are pattern-restricted by `values.schema.json` and XML-escaped at render time. User input cannot break the rendered Metadata Service config.
- `host`, `database`, and `schema` are pattern-restricted by `values.schema.json` and quoted as YAML scalars at render time. User input cannot break the rendered Metadata Service config.
- The Metadata Service creates its schema on first connect and migrates the schema in place when pointed at a database from an older Metadata Service version. No manual migration step is required.

## Credential changes

Update the PostgreSQL role password and its Secret as a coordinated database operation. After the new credentials are available, restart the Metadata Service and wait for its rollout. Updating a Secret alone does not restart the workload or rerun Helm.

For the `firebolt` release in the `firebolt` namespace, restart the Metadata Service:

```bash
kubectl rollout restart deployment/firebolt-metadata-service --namespace firebolt
kubectl rollout status deployment/firebolt-metadata-service --namespace firebolt --timeout=10m
```

Online Helm upgrades also track the existing credential Secret's resource version. Offline rendering cannot read live Secrets. You can instead configure an externally installed Secret reload controller through `metadata.annotations`. Such a controller restarts the workload but does not coordinate the database password change.


The bundled PostgreSQL path accepts `credentials.existingSecret` too. Supply `username`, `password`, and `database` keys, with username and database matching the chart values. Updating that Secret does not change a password inside an already-initialized database.
1 change: 1 addition & 0 deletions helm/AGENTS.md
Original file line number Diff line number Diff line change
Expand Up @@ -47,6 +47,7 @@ The Helm chart itself, packaged and published as `firebolt-instance` to `oci://g

- `Chart.yaml` — chart and app version, name, icon, sources. `version` is owned by release-please; `appVersion` is bumped by the upstream image-release automation (see project-specific rules in the root `AGENTS.md`).
- `values.yaml` — configuration surface AND the source for the generated `README.md`. Annotate every value with `# --` for `helm-docs`.
- `templates/_validation.tpl` — cross-field/name and chart-owned-field checks; online upgrade identity guard. `templates/validate.yaml` invokes these checks without rendering a resource.
- `values.schema.json` — JSON Schema validating the value surface at `helm install`/`helm lint` time. Permissive by design (unknown keys pass); constrains only enums, ranges, and patterns. See "Adding a new value".
- `values-dev.yaml` — committed overlay used by `make dev`: pins engine/metadata to the mutable `:dev` tag (instead of the chart's pinned `appVersion`) and points `customEngineConfig.storage` at the floci S3 emulator (so the dedicated-Pensieve managed-storage check is satisfied). The pinned `appVersion` stays the default for reproducible installs via plain `make install`.
- `README.md` — **generated.** Do not hand-edit. Run `make docs` from the repo root.
Expand Down
5 changes: 3 additions & 2 deletions helm/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -68,7 +68,7 @@ Firebolt Instance on Kubernetes — Envoy gateway, metadata, auth, and engines
| engines[0].priorityClassName | string | `""` | Priority class name for engine pods. |
| engines[0].replicas | int | `1` | Number of nodes in this engine group (one StatefulSet replica per node). |
| engines[0].resources | object | `{"limits":{"memory":"4Gi"},"requests":{"cpu":"1","memory":"4Gi"}}` | Resource requests and limits for engine containers. Firebolt Core is memory-bound: more RAM directly improves cache hit rates and query throughput. CPU governs parallel query execution threads. Typical sizing guidance: Development / functional testing: 2 vCPU / 8 Gi (request) Small production workload: 4 vCPU / 32 Gi Medium production workload: 8 vCPU / 64 Gi Large production workload: 16 vCPU / 128 Gi Storage I/O is also significant — use an SSD-backed StorageClass and size the PVC to hold your working dataset plus ~30 % headroom. |
| engines[0].storage | object | `{"accessModes":["ReadWriteOnce"],"resources":{"requests":{"storage":"100Gi"}}}` | PVC storage configuration for this engine. Falls back to `engineSpec.defaultStorage` if omitted. |
| engines[0].storage | object | `{"accessModes":["ReadWriteOnce"],"resources":{"requests":{"storage":"100Gi"}}}` | PVC storage configuration for this engine. Falls back to `engineSpec.defaultStorage` if omitted. Set `storageClassName` to select a class, or an empty string to disable dynamic provisioning. Existing claim templates cannot be resized or switched by a normal Helm upgrade. |
| engines[0].tolerations | list | `[]` | Tolerations for engine pod scheduling. |
| engines[0].topologySpreadConstraints | list | `[]` | Topology spread constraints for engine pod scheduling. Overrides `engineSpec.topologySpreadConstraints` for this engine when set. |
| extraLabels | object | `{"firebolt/product":"core"}` | Extra labels applied to all resources and pods. |
Expand Down Expand Up @@ -109,6 +109,7 @@ Firebolt Instance on Kubernetes — Envoy gateway, metadata, auth, and engines
| gateway.service.type | string | `"ClusterIP"` | Service type. One of `ClusterIP`, `LoadBalancer`, or `NodePort`. |
| imagePullSecrets | list | `[]` | Registry credentials. Must be a pre-created docker-registry Secret in the deployment namespace. Leave empty if nodes have ambient registry access (e.g. node IAM role). |
| metadata | object | {} | Metadata service configuration. |
| metadata.annotations | object | {} | Annotations on the Metadata Service Deployment, for example to opt into an externally installed Secret reload controller. These do not annotate pods. |
| metadata.deployment | object | {} | Deployment-level settings for the metadata service. |
| metadata.deployment.terminationGracePeriodSeconds | int | `30` | Termination grace period in seconds. |
| metadata.image.pullPolicy | string | `"IfNotPresent"` | Image pull policy. |
Expand Down Expand Up @@ -145,7 +146,7 @@ Firebolt Instance on Kubernetes — Envoy gateway, metadata, auth, and engines
| postgresql | object | {} | PostgreSQL configuration. When `local_enabled: true` the chart deploys a single-replica `postgres:16-alpine` StatefulSet. Set `local_enabled: false` and supply connection details for an external database. |
| postgresql.connect_timeout_sec | int | `5` | Connection timeout in seconds. |
| postgresql.credentials | object | {} | PostgreSQL credentials Secret configuration. |
| postgresql.credentials.existingSecret | string | `""` | Reference an externally-managed Secret (e.g. via ESO). When set, the chart will not create its own Secret. Ignored when `postgresql.local_enabled` is true. |
| postgresql.credentials.existingSecret | string | `""` | Reference an externally-managed Secret (e.g. via ESO). When set, the chart will not create its own Secret. Both bundled PostgreSQL and the Metadata Service use it. Require username/password keys, plus database for bundled PostgreSQL; username/database must match the corresponding postgresql values. Changing Secret data does not alter an already-initialized PostgreSQL role password. |
| postgresql.credentials.mountPath | string | `"/secrets/postgres"` | Mount path for the credentials Secret inside the metadata service container. |
| postgresql.database | string | `"firebolt_metadata"` | Database name. |
| postgresql.host | string | `""` | PostgreSQL host. Auto-derived when `local_enabled` is true; must be set explicitly for external databases. |
Expand Down
67 changes: 67 additions & 0 deletions helm/scripts/test_render.py
Original file line number Diff line number Diff line change
Expand Up @@ -44,5 +44,72 @@ def test_retries_do_not_replay_delivered_requests(self):
self.assertEqual(policy["retriable_headers"], [{"name": "X-Firebolt-Drained", "present_match": True}])


class ConfigurationTests(unittest.TestCase):
def test_existing_database_secret_reaches_both_consumers(self):
docs = render({"postgresql": {"credentials": {"existingSecret": "database-creds"}}})
pg = resource(docs, "StatefulSet", "audit-metadata-pg")["spec"]["template"]["spec"]
refs = [e["valueFrom"]["secretKeyRef"]["name"] for e in pg["containers"][0]["env"] if "valueFrom" in e]
self.assertEqual(refs, ["database-creds"] * 3)
metadata = resource(docs, "Deployment", "audit-metadata-service")["spec"]["template"]["spec"]
self.assertIn("database-creds", [v["secret"]["secretName"] for v in metadata["volumes"] if "secret" in v])
self.assertFalse(any(d["kind"] == "Secret" for d in docs))

def test_storage_class_absent_empty_and_named(self):
for storage_class in (None, "", "fast"):
with self.subTest(storage_class=storage_class):
storage = {"accessModes": ["ReadWriteOnce"], "resources": {"requests": {"storage": "1Gi"}}}
if storage_class is not None:
storage["storageClassName"] = storage_class
docs = render({"engines": [{"name": "default", "replicas": 1, "storage": storage}]})
claim = resource(docs, "StatefulSet", "audit-engine-default-node-0")["spec"]["volumeClaimTemplates"][0]["spec"]
self.assertEqual(claim.get("storageClassName"), storage_class)
self.assertEqual("storageClassName" in claim, storage_class is not None)

def test_metadata_rolls_on_identity(self):
def annotations(values):
return resource(render(values), "Deployment", "audit-metadata-service")["spec"]["template"]["metadata"]["annotations"]
baseline = annotations({})
changed_id = annotations({"customEngineConfig": {"instance": {"id": "01kp98j0000000000000000001"}}})
self.assertNotEqual(baseline["checksum/config"], changed_id["checksum/config"])

def test_mounts_cannot_shadow_owned_paths_or_duplicate_destinations(self):
for path in ("/secrets/auth/admin", "/var", "/var/lib/firebolt/../firebolt", "/etc/envoy/tls/gateway"):
component = "gateway" if path.startswith("/etc/envoy") else "engineSpec"
extras = {"customVolumes": [{"name": "custom", "emptyDir": {}}],
"customVolumeMounts": [{"name": "custom", "mountPath": path}]}
values = {component: extras} if component == "engineSpec" else {
component: {"podTemplate": {"volumes": extras["customVolumes"], "volumeMounts": extras["customVolumeMounts"]}}}
self.assertIn("chart-owned mount", render(values, expect_error=True))
self.assertIn("duplicate volume mount", render({"engineSpec": {
"customVolumeMounts": [{"name": "custom", "mountPath": "/custom"},
{"name": "other", "mountPath": "/custom/"}]}}, expect_error=True))
render({"engineSpec": {"customVolumes": [{"name": "custom", "emptyDir": {}}],
"customVolumeMounts": [{"name": "custom", "mountPath": "/custom"}]}})

def test_metadata_workload_annotations_do_not_change_pod_template(self):
baseline = resource(render(), "Deployment", "audit-metadata-service")
custom = {"secret.reloader.stakater.com/auto": "true"}
updated = resource(render({"metadata": {"annotations": custom}}), "Deployment", "audit-metadata-service")
self.assertEqual(updated["metadata"]["annotations"], custom)
self.assertEqual(updated["spec"]["template"], baseline["spec"]["template"])
self.assertNotIn("firebolt.io/restart-token", baseline["spec"]["template"]["metadata"]["annotations"])
render({"metadata": {"annotations": {"invalid": True}}}, expect_error=True)

def test_reject_invalid_combinations(self):
cases = [
({"engines": [{"name": "same", "replicas": 1}] * 2}, "duplicate engine"),
({"engines": [{"name": "x" * 63, "replicas": 1}]}, "exceeds 63"),
({"engineSpec": {"extraEnv": [{"name": "FIREBOLT_CORE_NODE", "value": "99"}]}}, "chart-owned environment"),
({"engineSpec": {"customVolumes": [{"name": "data", "emptyDir": {}}]}}, "chart-owned volume"),
({"engineSpec": {"customVolumeMounts": [{"name": "custom", "mountPath": "/var/lib/firebolt/config.yaml"}]}}, "chart-owned mount"),
({"gateway": {"podTemplate": {"podAnnotations": {"checksum/config": "override"}}}}, "reserved"),
({"auth": {"signingKeys": [{"id": "same"}] * 2}}, "duplicate signing"),
({"postgresql": {"local_enabled": False}}, "postgresql.host is required"),
]
for values, message in cases:
with self.subTest(values=values):
self.assertIn(message, render(values, expect_error=True))


if __name__ == "__main__":
unittest.main()
5 changes: 5 additions & 0 deletions helm/templates/_helpers.tpl
Original file line number Diff line number Diff line change
Expand Up @@ -395,3 +395,8 @@ log() { printf '\n\033[1;36m=== %s ===\033[0m\n' "$*"; }
pass() { printf '\033[1;32mPASS:\033[0m %s\n' "$*"; }
fail() { printf '\033[1;31mFAIL:\033[0m %s\n' "$*" >&2; exit 1; }
{{- end }}

{{/* One credential source for both the database and Metadata Service. */}}
{{- define "fbinstance.postgresSecretName" -}}
{{- .Values.postgresql.credentials.existingSecret | default (printf "%s-metadata-postgres-creds" (include "fbinstance.fullname" .)) -}}
{{- end -}}
Loading