Skip to content

OAPE-695:codecov integration changes using dockerfile and .sh file#425

Open
siddhibhor-56 wants to merge 1 commit into
openshift:masterfrom
siddhibhor-56:codecov
Open

OAPE-695:codecov integration changes using dockerfile and .sh file#425
siddhibhor-56 wants to merge 1 commit into
openshift:masterfrom
siddhibhor-56:codecov

Conversation

@siddhibhor-56
Copy link
Copy Markdown
Contributor

@siddhibhor-56 siddhibhor-56 commented May 14, 2026

Summary by CodeRabbit

  • Chores
    • Added E2E code coverage collection infrastructure with automated setup and artifact collection workflows.
    • Introduced coverage-instrumented container image and build/push targets for test coverage reporting in CI and local environments.

@openshift-ci-robot openshift-ci-robot added the jira/valid-reference Indicates that this PR references a valid Jira ticket of any type. label May 14, 2026
@openshift-ci-robot
Copy link
Copy Markdown

openshift-ci-robot commented May 14, 2026

@siddhibhor-56: This pull request references OAPE-695 which is a valid jira issue.

Warning: The referenced jira issue has an invalid target version for the target branch this PR targets: expected the sub-task to target the "5.0.0" version, but no target version was set.

Details

In response to this:

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the openshift-eng/jira-lifecycle-plugin repository.

@coderabbitai
Copy link
Copy Markdown

coderabbitai Bot commented May 14, 2026

Walkthrough

This PR introduces E2E coverage infrastructure for cert-manager-operator, comprising a coverage-instrumented Docker image, a bash lifecycle script for setup and collection, and Makefile targets to orchestrate the workflow. The setup phase patches the operator deployment to use the instrumented image and inject coverage environment variables; the collect phase extracts coverage data and optionally uploads it to Codecov.

Changes

E2E Coverage Infrastructure

Layer / File(s) Summary
Coverage-instrumented image and build targets
images/ci/Dockerfile.coverage, Makefile (lines 420–452)
Multi-stage Dockerfile compiles the operator binary with Go coverage flags (-cover, -covermode=count, -coverpkg=./...), prepares /tmp/e2e-cover for coverage data, and sets GOCOVERDIR. Makefile adds COVERAGE_IMG variable and image-build-coverage, image-push-coverage targets to build and push the instrumented image.
E2E coverage lifecycle script and integration
hack/e2e-coverage.sh, Makefile (e2e-coverage-collect target)
Bash script implements setup command to detect and patch the operator deployment (CSV-aware or direct), injecting coverage environment and volume mounts, and collect command to extract coverage data from the operator pod, optionally convert to text format, and upload to Codecov. Makefile e2e-coverage-collect target invokes the collect command with configurable ARTIFACT_DIR.

Estimated code review effort

🎯 3 (Moderate) | ⏱️ ~25 minutes

🚥 Pre-merge checks | ✅ 12
✅ Passed checks (12 passed)
Check name Status Explanation
Description Check ✅ Passed Check skipped - CodeRabbit’s high-level summary is enabled.
Title check ✅ Passed The title mentions Codecov integration and references the files being added (Dockerfile and .sh file), which aligns with the PR's main objective of integrating Codecov functionality.
Docstring Coverage ✅ Passed No functions found in the changed files to evaluate docstring coverage. Skipping docstring coverage check.
Linked Issues check ✅ Passed Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check ✅ Passed Check skipped because no linked issues were found for this pull request.
Stable And Deterministic Test Names ✅ Passed This PR only adds codecov integration files (Makefile targets, shell script, Dockerfile). It does not modify or add any Ginkgo test files. The check is not applicable to non-test files.
Test Structure And Quality ✅ Passed The custom check reviews Ginkgo test code quality, but this PR adds only infrastructure files: Makefile, bash script, and Dockerfile. No Ginkgo test files are included, so the check is not applicable.
Microshift Test Compatibility ✅ Passed No new Ginkgo e2e tests were added. Changes are build infrastructure only (Makefile, Dockerfile, shell scripts). MicroShift compatibility check not applicable.
Single Node Openshift (Sno) Test Compatibility ✅ Passed No new Ginkgo e2e tests in this PR. Changes are build/infrastructure only (Makefile, shell script, Dockerfile). SNO compatibility check not applicable.
Topology-Aware Scheduling Compatibility ✅ Passed PR adds E2E coverage infrastructure: Makefile targets, shell script, Dockerfile. No scheduling constraints, affinity rules, or operator deployment code changes.
Ote Binary Stdout Contract ✅ Passed PR adds no Go source code changes. Only adds Makefile build targets, bash CI script, and coverage Dockerfile. No process-level code writes to stdout.
Ipv6 And Disconnected Network Test Compatibility ✅ Passed PR adds infrastructure files only (Makefile, bash script, Dockerfile). No Ginkgo e2e tests are introduced, so this check does not apply.

✏️ Tip: You can configure your own custom pre-merge checks in the settings.

✨ Finishing Touches
🧪 Generate unit tests (beta)
  • Create PR with unit tests

Comment @coderabbitai help to get the list of available commands and usage tips.

@openshift-ci openshift-ci Bot requested review from TrilokGeer and bharath-b-rh May 14, 2026 12:39
@openshift-ci
Copy link
Copy Markdown
Contributor

openshift-ci Bot commented May 14, 2026

[APPROVALNOTIFIER] This PR is NOT APPROVED

This pull-request has been approved by: siddhibhor-56
Once this PR has been reviewed and has the lgtm label, please assign mytreya-rh for approval. For more information see the Code Review Process.

The full list of commands accepted by this bot can be found here.

Details Needs approval from an approver in each of these files:

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

Copy link
Copy Markdown

@coderabbitai coderabbitai Bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 2

🧹 Nitpick comments (1)
images/ci/Dockerfile.coverage (1)

24-24: ⚡ Quick win

Document the rationale for UID/GID 65534.

The numeric UID/GID 65534 corresponds to the nobody user/group, which is a common choice for running unprivileged containers. Consider adding a brief inline comment to clarify this choice for future maintainers.

📝 Suggested comment addition
-RUN mkdir -p /tmp/e2e-cover && chown 65534:65534 /tmp/e2e-cover && chmod 700 /tmp/e2e-cover
+# 65534 is the 'nobody' user/group - a standard unprivileged user for containers
+RUN mkdir -p /tmp/e2e-cover && chown 65534:65534 /tmp/e2e-cover && chmod 700 /tmp/e2e-cover
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@images/ci/Dockerfile.coverage` at line 24, Add a brief inline comment to the
RUN line that changes ownership to UID/GID 65534 so future maintainers know
65534 maps to the unprivileged "nobody" user/group; update the RUN command in
Dockerfile.coverage (the line that creates /tmp/e2e-cover and runs chown
65534:65534 and chmod 700) to include a short comment explaining that 65534 is
used to run as an unprivileged user (nobody) to minimize permissions and improve
security.
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Inline comments:
In `@hack/e2e-coverage.sh`:
- Around line 94-99: The fixed "sleep 10" after sending SIGTERM is flaky;
replace the static delay with an adaptive wait that polls pod readiness. Remove
the sleep and instead loop (or call oc wait directly) to check the pod
referenced by "${pod}" in "${NAMESPACE}" for phase=Running and container ready
(or use oc wait pod/"${pod}" --for=condition=Ready --timeout=120s without the
prior sleep), retrying until success or timeout; ensure the logic surrounds the
oc exec kill -TERM 1 call and uses the same pod variable so it handles variable
restart times instead of a fixed 10s pause.
- Around line 122-128: The checksum verification currently does a plain cd and
then runs sha256sum, which hides failures and still attempts cd -; change this
to run verification in a safe context and abort on failure: use a subshell or
pushd/popd so directory changes cannot leak (e.g., (cd "$(dirname
"${codecov_bin}")" && sha256sum -c "$(basename "${codecov_bin}").SHA256SUM"))
and check the exit status of sha256sum, exiting the script on non-zero; only run
chmod +x "${codecov_bin}" after the verification succeeds. Reference:
codecov_bin, sha256sum -c, and chmod +x "${codecov_bin}".

---

Nitpick comments:
In `@images/ci/Dockerfile.coverage`:
- Line 24: Add a brief inline comment to the RUN line that changes ownership to
UID/GID 65534 so future maintainers know 65534 maps to the unprivileged "nobody"
user/group; update the RUN command in Dockerfile.coverage (the line that creates
/tmp/e2e-cover and runs chown 65534:65534 and chmod 700) to include a short
comment explaining that 65534 is used to run as an unprivileged user (nobody) to
minimize permissions and improve security.
🪄 Autofix (Beta)

Fix all unresolved CodeRabbit comments on this PR:

  • Push a commit to this branch (recommended)
  • Create a new PR with the fixes

ℹ️ Review info
⚙️ Run configuration

Configuration used: Repository: openshift/coderabbit/.coderabbit.yaml

Review profile: CHILL

Plan: Enterprise

Run ID: d26f2852-30f2-42db-96f6-5f664ed4041a

📥 Commits

Reviewing files that changed from the base of the PR and between 5608078 and a4efab4.

📒 Files selected for processing (3)
  • Makefile
  • hack/e2e-coverage.sh
  • images/ci/Dockerfile.coverage

Comment thread hack/e2e-coverage.sh
Comment on lines +94 to +99
echo "Sending SIGTERM to operator process to flush coverage data..."
oc exec -n "${NAMESPACE}" "${pod}" -c cert-manager-operator -- /bin/sh -c 'kill -TERM 1' || true

echo "Waiting for container to restart..."
sleep 10
oc wait pod/"${pod}" --for=condition=Ready -n "${NAMESPACE}" --timeout=120s
Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

⚠️ Potential issue | 🟡 Minor | ⚡ Quick win

Hardcoded sleep may be insufficient for container restart.

The script waits exactly 10 seconds after sending SIGTERM before checking pod readiness. Container restart time can vary based on image pull policies, resource constraints, and cluster load. This fixed delay creates a race condition that may cause premature readiness checks.

⏱️ Proposed fix to add adaptive wait logic
     echo "Sending SIGTERM to operator process to flush coverage data..."
     oc exec -n "${NAMESPACE}" "${pod}" -c cert-manager-operator -- /bin/sh -c 'kill -TERM 1' || true
 
-    echo "Waiting for container to restart..."
-    sleep 10
-    oc wait pod/"${pod}" --for=condition=Ready -n "${NAMESPACE}" --timeout=120s
+    echo "Waiting for pod to become NotReady (container restarting)..."
+    for i in {1..30}; do
+        if ! oc get pod/"${pod}" -n "${NAMESPACE}" -o jsonpath='{.status.conditions[?(@.type=="Ready")].status}' | grep -q "True"; then
+            echo "Pod is restarting..."
+            break
+        fi
+        sleep 1
+    done
+    
+    echo "Waiting for pod to become Ready again..."
+    oc wait pod/"${pod}" --for=condition=Ready -n "${NAMESPACE}" --timeout=120s
📝 Committable suggestion

‼️ IMPORTANT
Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation. Thoroughly test & benchmark the code to ensure it meets the requirements.

Suggested change
echo "Sending SIGTERM to operator process to flush coverage data..."
oc exec -n "${NAMESPACE}" "${pod}" -c cert-manager-operator -- /bin/sh -c 'kill -TERM 1' || true
echo "Waiting for container to restart..."
sleep 10
oc wait pod/"${pod}" --for=condition=Ready -n "${NAMESPACE}" --timeout=120s
echo "Sending SIGTERM to operator process to flush coverage data..."
oc exec -n "${NAMESPACE}" "${pod}" -c cert-manager-operator -- /bin/sh -c 'kill -TERM 1' || true
echo "Waiting for pod to become NotReady (container restarting)..."
for i in {1..30}; do
if ! oc get pod/"${pod}" -n "${NAMESPACE}" -o jsonpath='{.status.conditions[?(@.type=="Ready")].status}' | grep -q "True"; then
echo "Pod is restarting..."
break
fi
sleep 1
done
echo "Waiting for pod to become Ready again..."
oc wait pod/"${pod}" --for=condition=Ready -n "${NAMESPACE}" --timeout=120s
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@hack/e2e-coverage.sh` around lines 94 - 99, The fixed "sleep 10" after
sending SIGTERM is flaky; replace the static delay with an adaptive wait that
polls pod readiness. Remove the sleep and instead loop (or call oc wait
directly) to check the pod referenced by "${pod}" in "${NAMESPACE}" for
phase=Running and container ready (or use oc wait pod/"${pod}"
--for=condition=Ready --timeout=120s without the prior sleep), retrying until
success or timeout; ensure the logic surrounds the oc exec kill -TERM 1 call and
uses the same pod variable so it handles variable restart times instead of a
fixed 10s pause.

Comment thread hack/e2e-coverage.sh
Comment on lines +122 to +128
curl -sS -o "${codecov_bin}" \
https://uploader.codecov.io/latest/linux/codecov
curl -sS -o "${codecov_bin}.SHA256SUM" \
https://uploader.codecov.io/latest/linux/codecov.SHA256SUM

cd "$(dirname "${codecov_bin}")" && sha256sum -c "$(basename "${codecov_bin}").SHA256SUM" && cd - >/dev/null
chmod +x "${codecov_bin}"
Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

⚠️ Potential issue | 🟡 Minor | ⚡ Quick win

Improve error handling in checksum verification.

The directory change for sha256sum verification (line 127) does not properly handle failures. If cd fails or the checksum verification fails, the script continues to cd - which may fail or return to the wrong directory, and the error is suppressed by redirecting to /dev/null.

🔒 Proposed fix for safer checksum verification
-            cd "$(dirname "${codecov_bin}")" && sha256sum -c "$(basename "${codecov_bin}").SHA256SUM" && cd - >/dev/null
+            if ! (cd "$(dirname "${codecov_bin}")" && sha256sum -c "$(basename "${codecov_bin}").SHA256SUM"); then
+                echo "Error: Codecov binary checksum verification failed"
+                exit 1
+            fi
📝 Committable suggestion

‼️ IMPORTANT
Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation. Thoroughly test & benchmark the code to ensure it meets the requirements.

Suggested change
curl -sS -o "${codecov_bin}" \
https://uploader.codecov.io/latest/linux/codecov
curl -sS -o "${codecov_bin}.SHA256SUM" \
https://uploader.codecov.io/latest/linux/codecov.SHA256SUM
cd "$(dirname "${codecov_bin}")" && sha256sum -c "$(basename "${codecov_bin}").SHA256SUM" && cd - >/dev/null
chmod +x "${codecov_bin}"
curl -sS -o "${codecov_bin}" \
https://uploader.codecov.io/latest/linux/codecov
curl -sS -o "${codecov_bin}.SHA256SUM" \
https://uploader.codecov.io/latest/linux/codecov.SHA256SUM
if ! (cd "$(dirname "${codecov_bin}")" && sha256sum -c "$(basename "${codecov_bin}").SHA256SUM"); then
echo "Error: Codecov binary checksum verification failed"
exit 1
fi
chmod +x "${codecov_bin}"
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@hack/e2e-coverage.sh` around lines 122 - 128, The checksum verification
currently does a plain cd and then runs sha256sum, which hides failures and
still attempts cd -; change this to run verification in a safe context and abort
on failure: use a subshell or pushd/popd so directory changes cannot leak (e.g.,
(cd "$(dirname "${codecov_bin}")" && sha256sum -c "$(basename
"${codecov_bin}").SHA256SUM")) and check the exit status of sha256sum, exiting
the script on non-zero; only run chmod +x "${codecov_bin}" after the
verification succeeds. Reference: codecov_bin, sha256sum -c, and chmod +x
"${codecov_bin}".

@openshift-ci
Copy link
Copy Markdown
Contributor

openshift-ci Bot commented May 14, 2026

@siddhibhor-56: all tests passed!

Full PR test history. Your PR dashboard.

Details

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository. I understand the commands that are listed here.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

jira/valid-reference Indicates that this PR references a valid Jira ticket of any type.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants