Skip to content

check: make key-value store check release-aware#2924

Merged
berendt merged 1 commit into
mainfrom
fix-valkey-infrastructure-check
Jul 6, 2026
Merged

check: make key-value store check release-aware#2924
berendt merged 1 commit into
mainfrom
fix-valkey-infrastructure-check

Conversation

@ideaship

@ideaship ideaship commented Jul 6, 2026

Copy link
Copy Markdown
Contributor

What

Make the post-upgrade key-value store check in
scripts/check/200-infrastructure.sh release-aware instead of hardcoding a
redis-era check.

Why

The infrastructure check sweep validated the key-value store with a single
hardcoded line that always spoke redis on 192.168.16.10:6379 using the
redis-era master password. Unlike the MariaDB and RabbitMQ checks in the same
file, it had no release conditional.

At OpenStack 2025.2, upstream kolla-ansible replaced redis with valkey, and the
valkey role uses a distinct valkey_master_password. After the upgrade the
check's hardcoded redis password fails AUTH, check_tcp reports an unexpected
response, and the whole build fails on the only non-conditional check in the
file.

First observed on testbed-upgrade-stable-next-ubuntu-24.04 (periodic-midnight)
build 3c209b90 — the first run in which the redis→valkey upgrade wall cleared
and the check was ever exercised against valkey. Elasticsearch, MariaDB,
Prometheus, and RabbitMQ all passed; the redis check was the sole failure.

How

  • Source include.sh and select the active service with valkey_or_redis()
    the same OpenStack-release signal deploy/200 and upgrade/200 already use.
  • Read the matching ${store}_master_password from secrets.yml instead of
    embedding a literal. Using the single source of truth removes the duplication
    that caused the drift, so the check follows the deployed service and won't
    re-break the next time a password changes.

Port, host, and INFO replication expectations are unchanged: valkey preserves
the redis wire format and the testbed topology is the same.

Verification

  • shellcheck clean; bash -n OK.
  • Password extraction confirmed against secrets.yml: redis resolves to the
    exact value previously hardcoded (redis-era releases unaffected), valkey
    resolves to its distinct password.
  • Full live valkey check_tcp round-trip still needs a
    testbed-upgrade-stable-next-ubuntu-24.04 run to exercise end-to-end — hence
    draft.

🤖 Generated with Claude Code

The post-upgrade infrastructure sweep checked the key-value store with a
single hardcoded line that always spoke redis on 192.168.16.10:6379 with
the redis-era master password. Unlike the MariaDB and RabbitMQ checks in
the same file, it had no release conditional.

At OpenStack 2025.2 upstream kolla-ansible replaced redis with valkey.
The valkey role gets a distinct valkey_master_password, so after the
upgrade the check's hardcoded redis password fails AUTH and check_tcp
reports an unexpected response, failing the whole build on the only
non-conditional check in the file.

Select the active service the same way deploy/200 and upgrade/200 do, via
valkey_or_redis() from include.sh, and read the matching
${store}_master_password from secrets.yml instead of embedding a literal.
Reading the password from the single source of truth removes the
duplication that caused the drift, so the check follows the deployed
service instead of re-breaking the next time a password changes. The
port, host, and INFO replication expectations are unchanged; valkey
preserves the redis wire format and the testbed topology is the same.

Assisted-by: Claude:claude-opus-4-8
Signed-off-by: Roger Luethi <luethi@osism.tech>
@ideaship ideaship marked this pull request as ready for review July 6, 2026 09:36
@ideaship ideaship requested a review from berendt July 6, 2026 09:36
@berendt berendt merged commit 1363931 into main Jul 6, 2026
3 checks passed
@berendt berendt deleted the fix-valkey-infrastructure-check branch July 6, 2026 11:11
@github-project-automation github-project-automation Bot moved this from New to Done in Human Board Jul 6, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

Status: Done

Development

Successfully merging this pull request may close these issues.

3 participants