Skip to content

[CRDB-42772] tls: Add enableSighupRotation option for zero-downtime cert rotation#605

Open
nameisbhaskar wants to merge 1 commit intomasterfrom
crdb-42772-cert-rotation-clean
Open

[CRDB-42772] tls: Add enableSighupRotation option for zero-downtime cert rotation#605
nameisbhaskar wants to merge 1 commit intomasterfrom
crdb-42772-cert-rotation-clean

Conversation

@nameisbhaskar
Copy link
Copy Markdown
Contributor

@nameisbhaskar nameisbhaskar commented Mar 11, 2026

Implements CRDB-42772 to enable zero-downtime certificate rotation in CockroachDB Helm charts using SIGHUP signal.

Changes:

  • Add tls.enableSighupRotation configuration option (default: false)
  • When tls.enableSighupRotation=true, node certificates mount directly from secrets in a projected volume
  • Enables live certificate reload via SIGHUP without pod restarts
  • Certificate permissions set to 0440 (vs 0400 in traditional mode) to allow group read access, since secrets mount with root:1000 ownership when securityContext.enabled=true
  • Only node certificates (ca.crt, node.crt, node.key) are mounted in the DB container's --certs-dir
  • Client certificates remain in a separate volume for use by visus and other components that need them
  • Conditional copy-certs initContainer (skipped when tls.enableSighupRotation=true)

Documentation:

  • Add zero-downtime certificate rotation section to README.md
  • Document SIGHUP-based rotation with deployment and usage examples
  • Explain 0440 permissions requirement due to Kubernetes fsGroup behavior
  • Update CHANGELOG.md with feature description

All procedures validated against live GKE cluster deployment.

@nameisbhaskar nameisbhaskar force-pushed the crdb-42772-cert-rotation-clean branch 2 times, most recently from 7ec6551 to 8fdb98e Compare March 11, 2026 07:56
@nameisbhaskar nameisbhaskar changed the title feat: add enableSighupRotation option for zero-downtime certificate r… feat: add enableSighupRotation option for zero-downtime cert rotation Mar 11, 2026
@nameisbhaskar nameisbhaskar force-pushed the crdb-42772-cert-rotation-clean branch from 8fdb98e to cdc1321 Compare March 12, 2026 10:29
@nameisbhaskar nameisbhaskar changed the title feat: add enableSighupRotation option for zero-downtime cert rotation [CRDB-42772] tls: Add enableSighupRotation option for zero-downtime cert rotation Mar 13, 2026
@NishanthNalluri NishanthNalluri requested a review from Copilot March 16, 2026 08:46
Copy link
Copy Markdown

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Adds an opt-in Helm chart mode to support zero-downtime TLS certificate rotation by mounting certs directly from Secrets and relying on CockroachDB’s SIGHUP-triggered certificate reload.

Changes:

  • Introduces tls.enableSighupRotation (default false) to switch between “copy-certs initContainer” mode and “direct Secret mount + SIGHUP reload” mode.
  • Updates the StatefulSet to use a combined projected volume (certs-combined) with 0440 permissions when SIGHUP rotation is enabled.
  • Documents the feature and usage in README.md and adds a chart-level CHANGELOG.md entry.

Reviewed changes

Copilot reviewed 6 out of 6 changed files in this pull request and generated 2 comments.

Show a summary per file
File Description
cockroachdb/values.yaml Adds the new tls.enableSighupRotation configuration flag and explains the behavioral difference.
cockroachdb/templates/statefulset.yaml Skips copy-certs initContainer in SIGHUP mode; adds certs-combined projected volume + conditional mounts.
cockroachdb/README.md Documents zero-downtime certificate rotation via SIGHUP with example commands.
cockroachdb/CHANGELOG.md Adds an “Unreleased” entry documenting the new option.
build/templates/cockroachdb/values.yaml Mirrors the new value + comments for the build template output.
build/templates/cockroachdb/README.md Mirrors the new documentation section for the build template output.
Comments suppressed due to low confidence (1)

cockroachdb/templates/statefulset.yaml:554

  • Even when tls.enableSighupRotation=true, the chart still defines the separate client-secret volume, but nothing mounts it in SIGHUP mode. Keeping an unused volume makes the TLS volume logic harder to reason about and increases the chance of future drift (e.g., changes applied to one path but not the other). Consider gating client-secret under not .Values.tls.enableSighupRotation (similar to certs-secret) or removing it entirely in SIGHUP mode.
        {{- if or .Values.tls.certs.provided .Values.tls.certs.certManager .Values.tls.certs.selfSigner.enabled }}
        - name: client-secret
        {{- if or .Values.tls.certs.tlsSecret .Values.tls.certs.certManager .Values.tls.certs.selfSigner.enabled }}
          projected:
            sources:
            - secret:
                {{- if .Values.tls.certs.selfSigner.enabled }}
                name: {{ template "cockroachdb.fullname" . }}-client-secret
                {{ else }}
                name: {{ .Values.tls.certs.clientRootSecret }}
                {{ end -}}
                items:
                - key: ca.crt
                  path: ca.crt
                  mode: 0400
                - key: tls.crt
                  path: client.root.crt
                  mode: 0400
                - key: tls.key
                  path: client.root.key
                  mode: 0400
        {{- else }}
          secret:
            secretName: {{ .Values.tls.certs.clientRootSecret }}
            defaultMode: 0400
        {{- end }}

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Copy link
Copy Markdown

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Adds an opt-in Helm value to support zero-downtime TLS certificate rotation for CockroachDB by mounting certs directly from Kubernetes secrets and reloading via SIGHUP (instead of requiring Pod restarts).

Changes:

  • Introduces tls.enableSighupRotation (default false) and documents the behavior and permission model (0440 vs 0400).
  • Updates the StatefulSet to optionally skip the copy-certs initContainer and mount a combined projected secret volume for certs.
  • Adds chart-level documentation and a new chart CHANGELOG entry describing the feature.

Reviewed changes

Copilot reviewed 6 out of 6 changed files in this pull request and generated 3 comments.

Show a summary per file
File Description
cockroachdb/values.yaml Adds the new tls.enableSighupRotation value and documents the mode/permission differences.
cockroachdb/templates/statefulset.yaml Implements conditional initContainer behavior and projected secret volume mounts for SIGHUP-based rotation.
cockroachdb/README.md Documents zero-downtime rotation workflow, examples, and permission rationale.
cockroachdb/CHANGELOG.md Introduces a chart changelog entry for the new TLS rotation option.
build/templates/cockroachdb/values.yaml Mirrors the values.yaml documentation for the build/templates output.
build/templates/cockroachdb/README.md Mirrors the README documentation for the build/templates output.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

You can also share your feedback on Copilot code review. Take the survey.

nameisbhaskar added a commit that referenced this pull request Mar 16, 2026
- Clarify that root:1000 ownership depends on fsGroup:1000 in pod securityContext
- Document that users who override securityContext must ensure correct cert permissions
- Explain that client certs are intentionally included in SIGHUP mode to support
  administrative operations (cockroach init, node status, etc.) from within pods

Addresses copilot review comments on PR #605.

Co-Authored-By: roachdev-claude <roachdev-claude-bot@cockroachlabs.com>
@nameisbhaskar nameisbhaskar requested a review from Copilot March 16, 2026 13:36
Copy link
Copy Markdown

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Adds an opt-in Helm value to support zero-downtime TLS certificate rotation for CockroachDB by mounting certs directly from Secrets and reloading via SIGHUP, along with accompanying documentation and changelog entry.

Changes:

  • Introduces tls.enableSighupRotation (default false) and wires it into the StatefulSet to skip copy-certs and use a projected Secret volume with group-readable cert perms.
  • Adds/updates README documentation with rotation steps and rationale for 0440 permissions when using fsGroup.
  • Adds a chart CHANGELOG.md entry describing the feature.

Reviewed changes

Copilot reviewed 6 out of 6 changed files in this pull request and generated 2 comments.

Show a summary per file
File Description
cockroachdb/values.yaml Adds tls.enableSighupRotation configuration and explanatory comments.
cockroachdb/templates/statefulset.yaml Implements SIGHUP-rotation behavior via conditional initContainer behavior and new projected volume mounts.
cockroachdb/README.md Documents zero-downtime rotation workflow and configuration.
cockroachdb/CHANGELOG.md Adds changelog entry for the new option/feature.
build/templates/cockroachdb/values.yaml Mirrors values change for build template output.
build/templates/cockroachdb/README.md Mirrors README change for build template output.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

You can also share your feedback on Copilot code review. Take the survey.

nameisbhaskar added a commit that referenced this pull request Mar 16, 2026
- Clarify that root:1000 ownership depends on fsGroup:1000 in pod securityContext
- Document that users who override securityContext must ensure correct cert permissions
- Explain that client certs are intentionally included in SIGHUP mode to support
  administrative operations (cockroach init, node status, etc.) from within pods

Addresses copilot review comments on PR #605.

Co-Authored-By: roachdev-claude <roachdev-claude-bot@cockroachlabs.com>
@nameisbhaskar nameisbhaskar force-pushed the crdb-42772-cert-rotation-clean branch from f1c7b9f to 70bed87 Compare March 16, 2026 14:02
@nameisbhaskar nameisbhaskar requested a review from Copilot March 16, 2026 14:04
Copy link
Copy Markdown

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Adds an opt-in Helm chart mode to support zero-downtime TLS certificate rotation for CockroachDB by mounting certs directly from Secrets and reloading via SIGHUP.

Changes:

  • Introduces tls.enableSighupRotation (default false) to switch between initContainer-copied certs (0400) vs direct Secret mounts (0440).
  • Updates the StatefulSet template to conditionally skip copy-certs and use a projected Secret volume for node certs when SIGHUP rotation is enabled.
  • Documents the rotation workflow in the chart README and records the feature in a new chart CHANGELOG.

Reviewed changes

Copilot reviewed 6 out of 6 changed files in this pull request and generated 4 comments.

Show a summary per file
File Description
cockroachdb/values.yaml Adds the tls.enableSighupRotation configuration flag and explains the behavior/permissions.
cockroachdb/templates/statefulset.yaml Implements conditional initContainer/volumeMount/volume logic for SIGHUP-based TLS rotation.
cockroachdb/README.md Adds “Zero-downtime certificate rotation” section and values table entry.
cockroachdb/CHANGELOG.md Adds a new changelog file with the feature entry under Unreleased.
build/templates/cockroachdb/values.yaml Mirrors the new values flag in the build template copy.
build/templates/cockroachdb/README.md Mirrors the new documentation section and values table entry in the build template copy.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

You can also share your feedback on Copilot code review. Take the survey.

nameisbhaskar added a commit that referenced this pull request Mar 16, 2026
- Clarify that root:1000 ownership depends on fsGroup:1000 in pod securityContext
- Document that users who override securityContext must ensure correct cert permissions
- Explain that client certs are intentionally included in SIGHUP mode to support
  administrative operations (cockroach init, node status, etc.) from within pods

Addresses copilot review comments on PR #605.

Co-Authored-By: roachdev-claude <roachdev-claude-bot@cockroachlabs.com>
@nameisbhaskar nameisbhaskar force-pushed the crdb-42772-cert-rotation-clean branch 2 times, most recently from e45132c to 260d83c Compare March 16, 2026 14:22
@nameisbhaskar nameisbhaskar requested a review from Copilot March 16, 2026 14:23
Copy link
Copy Markdown

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Adds an opt-in Helm value to support zero-downtime TLS certificate rotation for CockroachDB by mounting certs directly from Kubernetes Secrets and reloading them via SIGHUP, avoiding pod restarts.

Changes:

  • Introduces tls.enableSighupRotation (default false) and documents the rotation workflow.
  • Updates the StatefulSet template to skip the copy-certs initContainer and mount node certs directly from a projected Secret volume (0440) when enabled.
  • Adds a chart CHANGELOG entry describing the new option.

Reviewed changes

Copilot reviewed 6 out of 6 changed files in this pull request and generated 4 comments.

Show a summary per file
File Description
cockroachdb/values.yaml Adds the tls.enableSighupRotation configuration and explains permission behavior.
cockroachdb/templates/statefulset.yaml Conditionally mounts node certs directly via projected Secret + skips copy-certs initContainer.
cockroachdb/README.md Documents SIGHUP-based zero-downtime cert rotation and adds the new values table entry.
cockroachdb/CHANGELOG.md Adds a changelog entry for the new rotation feature.
build/templates/cockroachdb/values.yaml Mirrors the values.yaml update in the build template copy.
build/templates/cockroachdb/README.md Mirrors README documentation updates in the build template copy.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

You can also share your feedback on Copilot code review. Take the survey.

@nameisbhaskar nameisbhaskar force-pushed the crdb-42772-cert-rotation-clean branch 2 times, most recently from 4267247 to 960137a Compare March 16, 2026 15:35
@nameisbhaskar nameisbhaskar force-pushed the crdb-42772-cert-rotation-clean branch 3 times, most recently from eee87f4 to 6cf48ab Compare March 16, 2026 15:45
@nameisbhaskar nameisbhaskar requested a review from Copilot March 16, 2026 15:47
Copy link
Copy Markdown

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Adds an opt-in Helm value to support zero-downtime TLS node certificate rotation by mounting node certs directly from Secrets and reloading them via SIGHUP, avoiding pod restarts.

Changes:

  • Introduces tls.enableSighupRotation (default false) and documents the behavior and operational steps.
  • Updates StatefulSet templating to (optionally) skip the copy-certs initContainer and mount node certs via a projected Secret volume with group-readable permissions.
  • Adds a chart-specific cockroachdb/CHANGELOG.md entry describing the new option.

Reviewed changes

Copilot reviewed 6 out of 6 changed files in this pull request and generated 1 comment.

Show a summary per file
File Description
cockroachdb/values.yaml Adds the new tls.enableSighupRotation configuration flag and describes the behavior/permissions.
cockroachdb/templates/statefulset.yaml Implements conditional initContainer omission and node/client cert mounting changes for SIGHUP-based rotation.
cockroachdb/README.md Documents zero-downtime rotation workflow and adds the new value to the configuration table.
cockroachdb/CHANGELOG.md Adds an “Unreleased” entry noting the new SIGHUP rotation option.
build/templates/cockroachdb/values.yaml Mirrors the new value in the generated chart templates source.
build/templates/cockroachdb/README.md Mirrors the new documentation in the generated chart templates source.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

You can also share your feedback on Copilot code review. Take the survey.

…otation

Implements CRDB-42772 to enable zero-downtime certificate rotation in
CockroachDB Helm charts using SIGHUP signal.

Changes:
- Add tls.enableSighupRotation configuration option (default: false)
- When enableSighupRotation=true, certificates mount directly from secrets
- Enables live certificate reload via SIGHUP without pod restarts
- Certificate permissions set to 0440 (vs 0400 in traditional mode) to allow
  group read access since secrets mount with root:1000 ownership
- Mount node certificates in combined projected volume for SIGHUP rotation
- Conditional copy-certs initContainer (skipped when enableSighupRotation=true)
- Only render initContainers section when actually needed (tls without sighup,
  visus, or custom initContainers)
- Fix secret key mappings to handle different formats:
  - Use tls.crt/tls.key for tlsSecret, certManager, and selfSigner modes
  - Use node.crt/node.key for user-provided certificates
  - Apply same logic for client certificates
- Use consistent decimal mode notation (288 for 0440, matching existing 256 for 0400)
- Fix securityContext.enabled reference in documentation
- Restore client-secret mount for visus container (needed for DB connections)

Documentation:
- Add zero-downtime certificate rotation section to README.md
- Document SIGHUP-based rotation with deployment and usage examples
- Explain 0440 permissions requirement due to Kubernetes fsGroup behavior
- Update CHANGELOG.md with feature description

Co-Authored-By: roachdev-claude <roachdev-claude-bot@cockroachlabs.com>
@nameisbhaskar nameisbhaskar force-pushed the crdb-42772-cert-rotation-clean branch from 6cf48ab to 2da5a44 Compare March 17, 2026 03:25
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants