UPSTREAM: 1873: evictions: fix assumePod silently dropping success metric on informer race#637
Conversation
…tric on informer race When KubeVirt sets EvictionInProgressAnnotationKey before returning TooManyRequests, the informer's UpdateFunc can call addPod (evictionAssumed=false) before evictPod's assumePod call arrives. assumePod found the entry already present and returned early, leaving evictionAssumed=false. DeleteFunc then skipped the "success" metric. Fix: if the existing entry has evictionAssumed=false (added by addPod), upgrade it in place without double-counting the pod in the counters. Adds TestEvictionInBackgroundMetrics_InformerRace to reproduce the race deterministically. Signed-off-by: Simone Tiraboschi <stirabos@redhat.com>
|
@ingvagabund: The following test failed, say
Full PR test history. Your PR dashboard. DetailsInstructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository. I understand the commands that are listed here. |
|
/lgtm |
|
[APPROVALNOTIFIER] This PR is APPROVED This pull-request has been approved by: ingvagabund, tiraboschi The full list of commands accepted by this bot can be found here. The pull request process is described here DetailsNeeds approval from an approver in each of these files:
Approvers can indicate their approval by writing |
|
/cherry-pick main |
|
@tiraboschi: new pull request created: #789 DetailsIn response to this:
Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository. |
When KubeVirt sets EvictionInProgressAnnotationKey before returning TooManyRequests, the informer's UpdateFunc can call addPod (evictionAssumed=false) before evictPod's assumePod call arrives. assumePod found the entry already present and returned early, leaving evictionAssumed=false. DeleteFunc then skipped the "success" metric.
Fix: if the existing entry has evictionAssumed=false (added by addPod), upgrade it in place without double-counting the pod in the counters.
Adds TestEvictionInBackgroundMetrics_InformerRace to reproduce the race deterministically.
Pulling kubernetes-sigs#1873.
Description
Checklist
Please ensure your pull request meets the following criteria before submitting
for review, these items will be used by reviewers to assess the quality and
completeness of your changes: