Skip to content

OCPBUGS-81500: fix: udpate healthcheck previous state only after successful operator status update#1614

Open
fracappa wants to merge 1 commit into
openshift:mainfrom
fracappa:fca/fix-healthcheck-degraded-condition-not-cleared
Open

OCPBUGS-81500: fix: udpate healthcheck previous state only after successful operator status update#1614
fracappa wants to merge 1 commit into
openshift:mainfrom
fracappa:fca/fix-healthcheck-degraded-condition-not-cleared

Conversation

@fracappa

@fracappa fracappa commented May 8, 2026

Copy link
Copy Markdown
Contributor

Previously, getPacemakerStatus updated c.previous before updateOperatorStatus was called. If the status update failed, subsequent syncs would skip processing because c.previous already reflected the new CR timestamp, leaving PacemakerHealthCheckDegraded stuck a True indefinitely.

Summary by CodeRabbit

Bug Fixes

  • Corrected Pacemaker health check status tracking to ensure accurate grace-period timing and state transitions
  • Fixed handling of unknown status values during operator condition updates
  • Improved reliability of health check status persistence logic

… status update

Previously, getPacemakerStatus updated c.previous before updateOperatorStatus
was called. If the status update failed, subsequent syncs would skip processing
because c.previous already reflected the new CR timestamp, leaving
PacemakerHealthCheckDegraded stuck a True indefinitely.
@openshift-ci openshift-ci Bot added the do-not-merge/work-in-progress Indicates that a PR should not merge because it is a work in progress. label May 8, 2026
@openshift-ci

openshift-ci Bot commented May 8, 2026

Copy link
Copy Markdown
Contributor

Skipping CI for Draft Pull Request.
If you want CI signal for your change, please convert it to an actual PR.
You can still manually trigger a test run with /test all

@openshift-ci-robot openshift-ci-robot added the jira/valid-reference Indicates that this PR references a valid Jira ticket of any type. label May 8, 2026
@openshift-ci-robot

Copy link
Copy Markdown

@fracappa: This pull request references Jira Issue OCPBUGS-81500, which is valid. The bug has been moved to the POST state.

3 validation(s) were run on this bug
  • bug is open, matching expected state (open)
  • bug target version (5.0.0) matches configured target version for branch (5.0.0)
  • bug is in the state ASSIGNED, which is one of the valid states (NEW, ASSIGNED, POST)

The bug has been updated to refer to the pull request using the external bug tracker.

Details

In response to this:

Previously, getPacemakerStatus updated c.previous before updateOperatorStatus was called. If the status update failed, subsequent syncs would skip processing because c.previous already reflected the new CR timestamp, leaving PacemakerHealthCheckDegraded stuck a True indefinitely.

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the openshift-eng/jira-lifecycle-plugin repository.

@openshift-ci-robot openshift-ci-robot added the jira/valid-bug Indicates that a referenced Jira bug is valid for the branch this PR is targeting. label May 8, 2026
@coderabbitai

coderabbitai Bot commented May 8, 2026

Copy link
Copy Markdown

No actionable comments were generated in the recent review. 🎉

ℹ️ Recent review info
⚙️ Run configuration

Configuration used: Repository: openshift/coderabbit/.coderabbit.yaml

Review profile: CHILL

Plan: Enterprise

Run ID: aaece5c7-3584-4772-815e-5ad683fabcc3

📥 Commits

Reviewing files that changed from the base of the PR and between c0614ca and e897f46.

📒 Files selected for processing (2)
  • pkg/tnf/pkg/pacemaker/healthcheck.go
  • pkg/tnf/pkg/pacemaker/healthcheck_test.go

Walkthrough

The pacemaker health check controller now defers recording the previous status until after operator status updates succeed and the current status is not Unknown, instead of updating it during status retrieval. This refines when the "last valid" status baseline is established.

Changes

Pacemaker Previous Status Tracking

Layer / File(s) Summary
Core Implementation
pkg/tnf/pkg/pacemaker/healthcheck.go
The sync method defers updating c.previous to after updateOperatorStatus succeeds and only when currentStatus.OverallStatus is not Unknown, removing the prior update logic from getPacemakerStatus.
Test Alignment
pkg/tnf/pkg/pacemaker/healthcheck_test.go
TestHealthCheck_getPacemakerStatus_UnchangedTimestamp now sets controller.previous = current1 after the first getPacemakerStatus call to simulate the state update that occurs during normal sync() flow, ensuring correct behavior on the second invocation.

Estimated code review effort

🎯 2 (Simple) | ⏱️ ~10 minutes

🚥 Pre-merge checks | ✅ 11 | ❌ 1

❌ Failed checks (1 warning)

Check name Status Explanation Resolution
Docstring Coverage ⚠️ Warning Docstring coverage is 0.00% which is insufficient. The required threshold is 80.00%. Write docstrings for the functions missing them to satisfy the coverage threshold.
✅ Passed checks (11 passed)
Check name Status Explanation
Description Check ✅ Passed Check skipped - CodeRabbit’s high-level summary is enabled.
Title check ✅ Passed The title directly describes the main change: moving healthcheck previous state update to occur only after successful operator status update, which is the core fix in this PR.
Linked Issues check ✅ Passed Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check ✅ Passed Check skipped because no linked issues were found for this pull request.
Stable And Deterministic Test Names ✅ Passed The PR uses Go standard testing package, not Ginkgo. No Ginkgo test declarations (It, Describe, Context, When) found. All test names are static and deterministic. Check is not applicable.
Test Structure And Quality ✅ Passed The PR modifies standard Go unit tests, not Ginkgo tests. The custom check is for Ginkgo structure (It blocks, BeforeEach), which is not applicable here.
Microshift Test Compatibility ✅ Passed No new Ginkgo e2e tests added. PR modifies only a source file and unit tests using standard Go testing, not Ginkgo framework. Check not applicable.
Single Node Openshift (Sno) Test Compatibility ✅ Passed The PR modifies unit tests in pkg/tnf/pkg/pacemaker/, not e2e tests. These are standard Go tests, not Ginkgo e2e tests. The custom check only applies to Ginkgo e2e tests.
Topology-Aware Scheduling Compatibility ✅ Passed This PR modifies only healthcheck state management logic. No scheduling constraints (affinity, node selectors, topology spread, replicas, etc.) are introduced or modified.
Ote Binary Stdout Contract ✅ Passed PR introduces no OTE Binary Stdout Contract violations. Changes are library code using only klog.v2 for logging, with no direct stdout writes or unredirected logging.
Ipv6 And Disconnected Network Test Compatibility ✅ Passed No Ginkgo e2e tests were added in this PR. The changes are only to standard Go unit tests (healthcheck_test.go uses testing.T, not Ginkgo patterns). Check is not applicable.

✏️ Tip: You can configure your own custom pre-merge checks in the settings.

✨ Finishing Touches
🧪 Generate unit tests (beta)
  • Create PR with unit tests

Warning

There were issues while running some tools. Please review the errors and either fix the tool's configuration or disable the tool if it's a critical failure.

🔧 golangci-lint (2.12.1)

Error: can't load config: unsupported version of the configuration: "" See https://golangci-lint.run/docs/product/migration-guide for migration instructions
The command is terminated due to an error: can't load config: unsupported version of the configuration: "" See https://golangci-lint.run/docs/product/migration-guide for migration instructions


Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands and usage tips.

@openshift-ci

openshift-ci Bot commented May 8, 2026

Copy link
Copy Markdown
Contributor

[APPROVALNOTIFIER] This PR is NOT APPROVED

This pull-request has been approved by:
Once this PR has been reviewed and has the lgtm label, please assign jaypoulz for approval. For more information see the Code Review Process.

The full list of commands accepted by this bot can be found here.

Details Needs approval from an approver in each of these files:

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

@openshift-ci-robot

Copy link
Copy Markdown

@fracappa: This pull request references Jira Issue OCPBUGS-81500, which is valid.

3 validation(s) were run on this bug
  • bug is open, matching expected state (open)
  • bug target version (5.0.0) matches configured target version for branch (5.0.0)
  • bug is in the state POST, which is one of the valid states (NEW, ASSIGNED, POST)
Details

In response to this:

Previously, getPacemakerStatus updated c.previous before updateOperatorStatus was called. If the status update failed, subsequent syncs would skip processing because c.previous already reflected the new CR timestamp, leaving PacemakerHealthCheckDegraded stuck a True indefinitely.

Summary by CodeRabbit

Bug Fixes

  • Corrected Pacemaker health check status tracking to ensure accurate grace-period timing and state transitions
  • Fixed handling of unknown status values during operator condition updates
  • Improved reliability of health check status persistence logic

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the openshift-eng/jira-lifecycle-plugin repository.

@fracappa fracappa marked this pull request as ready for review May 20, 2026 13:39
@openshift-ci openshift-ci Bot removed the do-not-merge/work-in-progress Indicates that a PR should not merge because it is a work in progress. label May 20, 2026
@openshift-ci openshift-ci Bot requested review from fonta-rh and jaypoulz May 20, 2026 13:40
@openshift-ci

openshift-ci Bot commented May 20, 2026

Copy link
Copy Markdown
Contributor

@fracappa: all tests passed!

Full PR test history. Your PR dashboard.

Details

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository. I understand the commands that are listed here.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

jira/valid-bug Indicates that a referenced Jira bug is valid for the branch this PR is targeting. jira/valid-reference Indicates that this PR references a valid Jira ticket of any type.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants