Skip to content

OAPE-693:adds the e2e coverage collector for the operator using codecov#138

Open
siddhibhor-56 wants to merge 2 commits into
openshift:mainfrom
siddhibhor-56:codecov
Open

OAPE-693:adds the e2e coverage collector for the operator using codecov#138
siddhibhor-56 wants to merge 2 commits into
openshift:mainfrom
siddhibhor-56:codecov

Conversation

@siddhibhor-56
Copy link
Copy Markdown
Contributor

@siddhibhor-56 siddhibhor-56 commented Apr 12, 2026

Adds the e2e coverage collector for the operator using codecov

Summary by CodeRabbit

  • Chores
    • Added end-to-end coverage support: builds and publishes a coverage-instrumented image and collects coverage data during E2E runs.
    • Added tooling to extract, convert, and optionally upload E2E coverage reports to Codecov (upload skipped if no token).
    • Improved CI/local workflows for gathering E2E coverage artifacts.
    • Expanded the set of ignored vulnerability IDs in the vulnerability scan tooling.

@openshift-ci
Copy link
Copy Markdown

openshift-ci Bot commented Apr 12, 2026

[APPROVALNOTIFIER] This PR is NOT APPROVED

This pull-request has been approved by: siddhibhor-56
Once this PR has been reviewed and has the lgtm label, please assign mytreya-rh for approval. For more information see the Code Review Process.

The full list of commands accepted by this bot can be found here.

Details Needs approval from an approver in each of these files:

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

@coderabbitai
Copy link
Copy Markdown

coderabbitai Bot commented Apr 12, 2026

Note

Reviews paused

It looks like this branch is under active development. To avoid overwhelming you with review comments due to an influx of new commits, CodeRabbit has automatically paused this review. You can configure this behavior by changing the reviews.auto_review.auto_pause_after_reviewed_commits setting.

Use the following commands to manage reviews:

  • @coderabbitai resume to resume automatic reviews.
  • @coderabbitai review to trigger a single review.

Use the checkboxes below for quick actions:

  • ▶️ Resume reviews
  • 🔍 Trigger review

Walkthrough

Adds E2E coverage support: Makefile targets and variable, a coverage-builder Dockerfile, and a new hack/e2e-coverage.sh script with setup/collect modes to deploy an instrumented operator, mount a PVC for GOCOVERDIR, extract coverage artifacts, and optionally upload to Codecov.

Changes

E2E Coverage integration

Layer / File(s) Summary
Makefile variables & help
Makefile
Adds COVERAGE_IMG ?= $(IMG)-e2e-coverage and a Makefile help block describing E2E coverage targets.
Build image
images/ci/Dockerfile.coverage
New multi-stage Dockerfile: builds operator with Go coverage flags and produces runtime image that sets GOCOVERDIR=/tmp/e2e-cover and runs as non-root.
Orchestration script (setup)
hack/e2e-coverage.sh
setup() creates a PVC, discovers operator CSV/deployment, patches CSV to use the coverage image, injects GOCOVERDIR and mounts PVC into the controller pod, waits for rollout, and verifies env/pod state.
Orchestration script (collect)
hack/e2e-coverage.sh
collect() determines artifact paths, reads CODECOV_TOKEN if available, scales operator to trigger coverage flush, creates a short-lived extractor pod to copy coverage files from the PVC, converts covmeta data to a go coverage profile, and optionally uploads to Codecov (upload failures non-fatal).
Makefile targets / wiring
Makefile
Adds phony targets: docker-build-coverage, docker-push-coverage, and e2e-coverage-collect (runs hack/e2e-coverage.sh collect), with optional ARTIFACT_DIR usage.
Govulncheck ignore list update
hack/govulncheck.sh
Expands KNOWN_VULNS_PATTERN to include GO-2026-4971 and GO-2026-4918.*

* Small unrelated addition included in the same PR.

Sequence Diagram

sequenceDiagram
    participant CI as CI Runner
    participant K8s as Kubernetes API
    participant Operator as Operator Pod
    participant PVC as Coverage PVC
    participant Extractor as Extractor Pod
    participant Codecov as Codecov

    CI->>K8s: hack/e2e-coverage.sh setup (create PVC, patch CSV to coverage image)
    K8s->>Operator: rollout controller with coverage image and PVC mounted (GOCOVERDIR set)
    CI->>K8s: run E2E tests against instrumented Operator
    CI->>K8s: hack/e2e-coverage.sh collect (scale down operator)
    Operator->>PVC: write coverage files to PVC on shutdown
    CI->>K8s: create Extractor Pod mounting PVC
    Extractor->>PVC: read and copy coverage files
    Extractor-->>CI: copy artifacts to CI workspace
    CI->>CI: convert covmeta -> go coverage profile
    CI->>Codecov: upload profile if `CODECOV_TOKEN` present
    Codecov-->>CI: upload response
Loading

Estimated code review effort

🎯 3 (Moderate) | ⏱️ ~25 minutes

🚥 Pre-merge checks | ✅ 11 | ❌ 1

❌ Failed checks (1 warning)

Check name Status Explanation Resolution
Docstring Coverage ⚠️ Warning Docstring coverage is 0.00% which is insufficient. The required threshold is 80.00%. Write docstrings for the functions missing them to satisfy the coverage threshold.
✅ Passed checks (11 passed)
Check name Status Explanation
Description Check ✅ Passed Check skipped - CodeRabbit’s high-level summary is enabled.
Linked Issues check ✅ Passed Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check ✅ Passed Check skipped because no linked issues were found for this pull request.
Stable And Deterministic Test Names ✅ Passed Check is not applicable to this PR. The PR modifies only build/infrastructure files (Makefile, Docker build files, CI scripts) and does not add or modify any Ginkgo test definitions or test names.
Test Structure And Quality ✅ Passed PR does not contain Ginkgo test code. Changes are Makefile, bash scripts, and Docker image—no test files to review.
Microshift Test Compatibility ✅ Passed No new Ginkgo e2e tests were added. The PR only adds infrastructure files: Makefile targets, a Dockerfile for coverage instrumentation, and bash scripts for coverage collection.
Single Node Openshift (Sno) Test Compatibility ✅ Passed No new Ginkgo e2e tests are added. PR changes are limited to infrastructure files (Makefile, bash scripts, Dockerfile) for E2E coverage collection. Check is not applicable.
Topology-Aware Scheduling Compatibility ✅ Passed PR adds E2E coverage infrastructure only. No scheduling constraints, affinity rules, nodeSelectors, or topology-aware configurations introduced.
Ote Binary Stdout Contract ✅ Passed No Go code modified. Infrastructure-only changes (Makefile, shell scripts, Dockerfile). Existing logging uses stderr correctly. Test suite uses GinkgoWriter for stdout.
Ipv6 And Disconnected Network Test Compatibility ✅ Passed No new Ginkgo e2e tests are added in this PR. The changes are infrastructure for E2E coverage collection only (Makefile, Docker, scripts). The custom check is not applicable.
Title check ✅ Passed The title clearly describes the main change: adding E2E coverage collection for the operator using Codecov, which aligns with the core modifications (Makefile targets, coverage script, coverage Dockerfile).

✏️ Tip: You can configure your own custom pre-merge checks in the settings.

✨ Finishing Touches
🧪 Generate unit tests (beta)
  • Create PR with unit tests

Tip

💬 Introducing Slack Agent: The best way for teams to turn conversations into code.

Slack Agent is built on CodeRabbit's deep understanding of your code, so your team can collaborate across the entire SDLC without losing context.

  • Generate code and open pull requests
  • Plan features and break down work
  • Investigate incidents and troubleshoot customer tickets together
  • Automate recurring tasks and respond to alerts with triggers
  • Summarize progress and report instantly

Built for teams:

  • Shared memory across your entire org—no repeating context
  • Per-thread sandboxes to safely plan and execute work
  • Governance built-in—scoped access, auditability, and budget controls

One agent for your entire SDLC. Right inside Slack.

👉 Get started


Comment @coderabbitai help to get the list of available commands and usage tips.

Copy link
Copy Markdown

@coderabbitai coderabbitai Bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 3

🤖 Prompt for all review comments with AI agents
Verify each finding against the current code and only fix it if needed.

Inline comments:
In `@images/ci/Dockerfile`:
- Around line 30-31: The Dockerfile currently uses "RUN mkdir -p /tmp/e2e-cover
&& chmod 777 /tmp/e2e-cover" which creates a world-writable directory; instead
create the directory, chown it to UID/GID 65534 and set restrictive permissions
(e.g. 0700 or 0750) so only the intended user can write; update the RUN that
creates /tmp/e2e-cover to use chown 65534:65534 and chmod 700 (or 750 if group
write is needed) rather than chmod 777.

In `@Makefile`:
- Around line 234-243: The kubectl cp invocation in the Makefile is causing a
nested e2e-cover directory, so change the kubectl cp usage to copy the contents
of /tmp/e2e-cover directly into $(E2E_COVERAGE_DIR); update the kubectl cp line
that references $(KUBECTL) cp "$(E2E_COVERAGE_NAMESPACE)/$$POD:/tmp/e2e-cover"
$(E2E_COVERAGE_DIR) to use a trailing dot on the source (e.g.
"$(E2E_COVERAGE_NAMESPACE)/$$POD:/tmp/e2e-cover/." $(E2E_COVERAGE_DIR)) so
covmeta and covcounters land directly in $(E2E_COVERAGE_DIR) for go tool covdata
to consume.
- Around line 245-248: Replace the unsafe "latest" downloader invocation in the
Makefile (the curl/chmod/./codecov sequence) with a pinned release workflow:
download a specific versioned uploader binary URL (not "latest") and its
corresponding SHA256SUM (or .sha256) file, verify the binary against the
checksum (using sha256sum -c or equivalent) before making it executable and
running it (the ./codecov invocation), and fail the step if verification fails;
keep the existing flags (--file=$(OUTPUTS_PATH)/coverage-e2e.out --flags=e2e
--name="E2E Coverage" --verbose) and still remove the binary afterwards. Ensure
the Makefile references the chosen version string so it is explicit and
reproducible.
🪄 Autofix (Beta)

Fix all unresolved CodeRabbit comments on this PR:

  • Push a commit to this branch (recommended)
  • Create a new PR with the fixes

ℹ️ Review info
⚙️ Run configuration

Configuration used: Organization UI

Review profile: CHILL

Plan: Pro

Run ID: 9ff32fe0-a38f-4c43-ab77-dfdadbf0a804

📥 Commits

Reviewing files that changed from the base of the PR and between 2a65cb8 and 5670473.

📒 Files selected for processing (2)
  • Makefile
  • images/ci/Dockerfile

Comment thread images/ci/Dockerfile Outdated
Comment thread Makefile Outdated
Comment thread Makefile Outdated
Copy link
Copy Markdown

@coderabbitai coderabbitai Bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🧹 Nitpick comments (10)
test/utils/bitwarden_resources.go (2)

26-32: Constants are duplicated with aws_resources.go.

The constants externalSecretsAPIVersionV1 and clusterSecretStoreKind are also defined in test/utils/aws_resources.go. Consider extracting these to a shared constants file within the utils package to avoid drift.

🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed.

In `@test/utils/bitwarden_resources.go` around lines 26 - 32, The constants
externalSecretsAPIVersionV1 and clusterSecretStoreKind are duplicated across
bitwarden_resources.go and aws_resources.go; extract these into a shared utils
package constants file (e.g., constants.go) and replace the local definitions in
both bitwarden_resources.go and aws_resources.go with references to the new
shared constants (externalSecretsAPIVersionV1, clusterSecretStoreKind), removing
the duplicate declarations so both files import/use the single source of truth.

70-96: Consider handling SetNestedField errors.

While SetNestedField typically only fails if the path is structurally invalid (which won't happen with these compile-time known paths), explicitly checking the error would make the code more defensive.

-	_ = unstructured.SetNestedField(u.Object, []interface{}{
+	if err := unstructured.SetNestedField(u.Object, []interface{}{
 		map[string]interface{}{
 			"secretKey": "value",
 			"remoteRef": map[string]interface{}{
 				"key": remoteKey,
 			},
 		},
-	}, "spec", "data")
+	}, "spec", "data"); err != nil {
+		panic(fmt.Sprintf("failed to set spec.data: %v", err))
+	}
🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed.

In `@test/utils/bitwarden_resources.go` around lines 70 - 96, The SetNestedField
calls in BitwardenExternalSecretByName and BitwardenExternalSecretByUUID ignore
returned errors; update both functions to capture the error from
unstructured.SetNestedField and handle it (e.g., if err != nil {
panic(fmt.Errorf("failed to set spec.data for %s: %w", name, err)) }) so
failures are surfaced; reference the SetNestedField call sites in
BitwardenExternalSecretByName and BitwardenExternalSecretByUUID and include the
original error text in the panic/log message for diagnosability.
test/utils/bitwarden_api_runner.go (1)

35-38: Consider pinning the curl image tag for reproducibility.

Using docker.io/curlimages/curl:latest can lead to non-reproducible test behavior if the image is updated with breaking changes. Consider pinning to a specific version.

 const (
-	bitwardenAPITestRunnerImage = "docker.io/curlimages/curl:latest"
+	bitwardenAPITestRunnerImage = "docker.io/curlimages/curl:8.7.1"
 	bitwardenCredMountPath      = "/etc/bitwarden-cred"
 )
🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed.

In `@test/utils/bitwarden_api_runner.go` around lines 35 - 38, The constant
bitwardenAPITestRunnerImage currently uses the mutable tag
"docker.io/curlimages/curl:latest" which can cause flaky tests; change the value
of bitwardenAPITestRunnerImage to a specific, immutable curlimages tag (e.g., a
known semver like :8.x.y) and add a short comment noting why the tag is pinned;
ensure any CI/test docs referencing this constant are updated when you
intentionally upgrade the pinned tag.
Makefile (2)

182-182: Add PHONY declaration for the targets on this line.

The static analysis tool flagged that targets listed here should be declared as .PHONY to ensure they're always executed regardless of whether a file with that name exists.

+.PHONY: fmt vet test test-unit test-e2e e2e-coverage-publish run update-vendor update-dep
 fmt vet test test-unit test-e2e e2e-coverage-publish run update-vendor update-dep: GOFLAGS=
🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed.

In `@Makefile` at line 182, The Makefile line defining the combined targets "fmt
vet test test-unit test-e2e e2e-coverage-publish run update-vendor update-dep:
GOFLAGS=" lacks a .PHONY declaration, so add a .PHONY rule listing these exact
target names (fmt vet test test-unit test-e2e e2e-coverage-publish run
update-vendor update-dep) elsewhere in the Makefile to ensure they always run
regardless of files with those names; update the .PHONY block to include all
these targets.

219-222: Codecov uploader pinning and SHA256 verification are correct.

The implementation correctly pins v0.7.6 and verifies the SHA256 checksum before execution. v0.7.6 is a valid release. However, v0.8.0 is now the latest available version. If this version is not pinned for specific compatibility reasons, consider upgrading to v0.8.0 to benefit from any bug fixes or security patches.

🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed.

In `@Makefile` around lines 219 - 222, Update the pinned Codecov uploader by
changing the CODECOV_VERSION variable from v0.7.6 to v0.8.0 and adjust the
derived URLs (CODECOV_URL and CODECOV_SHA_URL) will automatically follow; if
there's a deliberate compatibility constraint, add a brief comment next to
CODECOV_VERSION explaining why v0.7.6 must remain pinned instead of upgrading to
v0.8.0.
test/utils/bitwarden.go (2)

182-201: Consider validating organization and project IDs.

FetchBitwardenCredsFromSecret validates that token is non-empty but silently returns empty strings for orgID and projectID if they're missing. If these are required for Bitwarden API operations, consider validating them as well.

 	if v, ok := secret.Data[BitwardenCredKeyOrgID]; ok {
 		orgID = string(v)
+	} else {
+		return "", "", "", fmt.Errorf("bitwarden creds secret %s/%s missing required key %q", secretNamespace, secretName, BitwardenCredKeyOrgID)
 	}
🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed.

In `@test/utils/bitwarden.go` around lines 182 - 201,
FetchBitwardenCredsFromSecret currently only enforces token presence but
silently allows orgID and projectID to be empty; update the function to validate
BitwardenCredKeyOrgID and BitwardenCredKeyProjectID like TokenSecretKey and
return a descriptive error if either is missing or empty (use the same error
formatting pattern as the existing token check, referencing
secretNamespace/secretName and the missing key names) so callers receive a clear
failure when required organization or project IDs are absent.

326-341: Permissive error handling may mask connectivity issues.

The reachability check treats http_code=000 (connection refused/timeout/TLS failure) and empty logs as success to avoid failing the suite. While this is pragmatic, it could mask real connectivity problems that would cause subsequent tests to fail with less informative errors.

Consider at minimum logging a warning when these permissive paths are taken, so operators can investigate if tests fail later.

🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed.

In `@test/utils/bitwarden.go` around lines 326 - 341, In the PodFailed case of the
reachability check (the switch handling corev1.PodFailed), add a warning log
before returning true on the permissive paths so connectivity issues aren't
silently ignored: when getPodLogs(...) contains "http_code=000" or when logs
contain "error on the server" or are empty/whitespace, call the test/logger
warning function to record the pod/container identity (use
formatPodContainerStatus(p) and podName/BitwardenOperandNamespace) and the raw
logs, then return true,nil as before.
test/utils/dynamic_resources.go (1)

112-127: Input mutation may surprise callers.

getResourceInterface mutates the input unstructuredObj by calling SetNamespace (Line 121). This side effect could be unexpected for callers who pass an object they intend to reuse. Consider documenting this behavior or working with a copy.

 func (d DynamicResourceLoader) getResourceInterface(unstructuredObj *unstructured.Unstructured, overrideNamespace string) dynamic.ResourceInterface {
 	gvk := unstructuredObj.GroupVersionKind()
 	gr, err := restmapper.GetAPIGroupResources(d.KubeClient.Discovery())
 	require.NoError(d.t, err)
 	mapper := restmapper.NewDiscoveryRESTMapper(gr)
 	mapping, err := mapper.RESTMapping(gvk.GroupKind(), gvk.Version)
 	require.NoError(d.t, err)
 	if mapping.Scope.Name() == meta.RESTScopeNameNamespace {
 		if overrideNamespace != "" {
-			unstructuredObj.SetNamespace(overrideNamespace)
+			unstructuredObj.SetNamespace(overrideNamespace) // Note: mutates input object
 		}
 		require.NotEmpty(d.t, unstructuredObj.GetNamespace(), "Namespace can not be empty for namespaced resource")
 		return d.DynamicClient.Resource(mapping.Resource).Namespace(unstructuredObj.GetNamespace())
 	}
 	return d.DynamicClient.Resource(mapping.Resource)
 }
🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed.

In `@test/utils/dynamic_resources.go` around lines 112 - 127, The
getResourceInterface function currently mutates the caller's unstructuredObj by
calling SetNamespace on it when applying overrideNamespace; avoid surprising
side-effects by operating on a copy: create a deep copy of unstructuredObj
(e.g., via unstructuredObj.DeepCopy()), set the namespace on the copy rather
than the original, and use the copy's GetNamespace when selecting the Resource
Interface; alternatively, clearly document that getResourceInterface will mutate
unstructuredObj if overrideNamespace is provided. Ensure references to
SetNamespace, unstructuredObj, overrideNamespace and getResourceInterface are
updated accordingly.
test/utils/artifact_dump.go (1)

171-177: Consider pre-compiling the regex for sanitizeFilename.

The regex is compiled on every call. While acceptable for e2e test utilities where this runs infrequently, pre-compiling at package level would be slightly more efficient.

♻️ Optional: Pre-compile regex
+var sanitizeFilenameRe = regexp.MustCompile(`[^a-zA-Z0-9_-]`)
+
 func sanitizeFilename(s string) string {
-	s = regexp.MustCompile(`[^a-zA-Z0-9_-]`).ReplaceAllString(s, "_")
+	s = sanitizeFilenameRe.ReplaceAllString(s, "_")
 	if len(s) > 100 {
 		s = s[:100]
 	}
 	return s
 }
🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed.

In `@test/utils/artifact_dump.go` around lines 171 - 177, The sanitizeFilename
function currently compiles the regex on every call; move the
regexp.MustCompile(`[^a-zA-Z0-9_-]`) to a package-level variable (e.g.,
filenameSanitizeRegex) and update sanitizeFilename to use
filenameSanitizeRegex.ReplaceAllString(s, "_") so the regex is compiled once and
behavior (truncation to 100 chars) remains unchanged.
test/e2e/bitwarden_api_test.go (1)

139-144: Consider documenting why both 200 and 400 are acceptable.

The test accepts either HTTP 200 or 400 for empty ids list. Adding a brief comment explaining the expected API behavior variance would improve clarity.

🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed.

In `@test/e2e/bitwarden_api_test.go` around lines 139 - 144, Add a short comment
above the It("GET /rest/api/1/secrets-by-ids with empty ids should return 200 or
400", ...) test explaining why both HTTP 200 and 400 are accepted (e.g.,
different implementations may treat an empty ids list as a valid no-op returning
200 or as a client error returning 400), so future readers of the test (and
maintainers of runAPIJob/script and the API contract) understand the intended
variance; reference the test name, the script variable that posts '{"ids":[]}',
and runAPIJob to locate the exact spot to insert the comment.
🤖 Prompt for all review comments with AI agents
Verify each finding against the current code and only fix it if needed.

Nitpick comments:
In `@Makefile`:
- Line 182: The Makefile line defining the combined targets "fmt vet test
test-unit test-e2e e2e-coverage-publish run update-vendor update-dep: GOFLAGS="
lacks a .PHONY declaration, so add a .PHONY rule listing these exact target
names (fmt vet test test-unit test-e2e e2e-coverage-publish run update-vendor
update-dep) elsewhere in the Makefile to ensure they always run regardless of
files with those names; update the .PHONY block to include all these targets.
- Around line 219-222: Update the pinned Codecov uploader by changing the
CODECOV_VERSION variable from v0.7.6 to v0.8.0 and adjust the derived URLs
(CODECOV_URL and CODECOV_SHA_URL) will automatically follow; if there's a
deliberate compatibility constraint, add a brief comment next to CODECOV_VERSION
explaining why v0.7.6 must remain pinned instead of upgrading to v0.8.0.

In `@test/e2e/bitwarden_api_test.go`:
- Around line 139-144: Add a short comment above the It("GET
/rest/api/1/secrets-by-ids with empty ids should return 200 or 400", ...) test
explaining why both HTTP 200 and 400 are accepted (e.g., different
implementations may treat an empty ids list as a valid no-op returning 200 or as
a client error returning 400), so future readers of the test (and maintainers of
runAPIJob/script and the API contract) understand the intended variance;
reference the test name, the script variable that posts '{"ids":[]}', and
runAPIJob to locate the exact spot to insert the comment.

In `@test/utils/artifact_dump.go`:
- Around line 171-177: The sanitizeFilename function currently compiles the
regex on every call; move the regexp.MustCompile(`[^a-zA-Z0-9_-]`) to a
package-level variable (e.g., filenameSanitizeRegex) and update sanitizeFilename
to use filenameSanitizeRegex.ReplaceAllString(s, "_") so the regex is compiled
once and behavior (truncation to 100 chars) remains unchanged.

In `@test/utils/bitwarden_api_runner.go`:
- Around line 35-38: The constant bitwardenAPITestRunnerImage currently uses the
mutable tag "docker.io/curlimages/curl:latest" which can cause flaky tests;
change the value of bitwardenAPITestRunnerImage to a specific, immutable
curlimages tag (e.g., a known semver like :8.x.y) and add a short comment noting
why the tag is pinned; ensure any CI/test docs referencing this constant are
updated when you intentionally upgrade the pinned tag.

In `@test/utils/bitwarden_resources.go`:
- Around line 26-32: The constants externalSecretsAPIVersionV1 and
clusterSecretStoreKind are duplicated across bitwarden_resources.go and
aws_resources.go; extract these into a shared utils package constants file
(e.g., constants.go) and replace the local definitions in both
bitwarden_resources.go and aws_resources.go with references to the new shared
constants (externalSecretsAPIVersionV1, clusterSecretStoreKind), removing the
duplicate declarations so both files import/use the single source of truth.
- Around line 70-96: The SetNestedField calls in BitwardenExternalSecretByName
and BitwardenExternalSecretByUUID ignore returned errors; update both functions
to capture the error from unstructured.SetNestedField and handle it (e.g., if
err != nil { panic(fmt.Errorf("failed to set spec.data for %s: %w", name, err))
}) so failures are surfaced; reference the SetNestedField call sites in
BitwardenExternalSecretByName and BitwardenExternalSecretByUUID and include the
original error text in the panic/log message for diagnosability.

In `@test/utils/bitwarden.go`:
- Around line 182-201: FetchBitwardenCredsFromSecret currently only enforces
token presence but silently allows orgID and projectID to be empty; update the
function to validate BitwardenCredKeyOrgID and BitwardenCredKeyProjectID like
TokenSecretKey and return a descriptive error if either is missing or empty (use
the same error formatting pattern as the existing token check, referencing
secretNamespace/secretName and the missing key names) so callers receive a clear
failure when required organization or project IDs are absent.
- Around line 326-341: In the PodFailed case of the reachability check (the
switch handling corev1.PodFailed), add a warning log before returning true on
the permissive paths so connectivity issues aren't silently ignored: when
getPodLogs(...) contains "http_code=000" or when logs contain "error on the
server" or are empty/whitespace, call the test/logger warning function to record
the pod/container identity (use formatPodContainerStatus(p) and
podName/BitwardenOperandNamespace) and the raw logs, then return true,nil as
before.

In `@test/utils/dynamic_resources.go`:
- Around line 112-127: The getResourceInterface function currently mutates the
caller's unstructuredObj by calling SetNamespace on it when applying
overrideNamespace; avoid surprising side-effects by operating on a copy: create
a deep copy of unstructuredObj (e.g., via unstructuredObj.DeepCopy()), set the
namespace on the copy rather than the original, and use the copy's GetNamespace
when selecting the Resource Interface; alternatively, clearly document that
getResourceInterface will mutate unstructuredObj if overrideNamespace is
provided. Ensure references to SetNamespace, unstructuredObj, overrideNamespace
and getResourceInterface are updated accordingly.

ℹ️ Review info
⚙️ Run configuration

Configuration used: Repository: openshift/coderabbit/.coderabbit.yaml

Review profile: CHILL

Plan: Enterprise

Run ID: a9d37d64-68e3-464b-9b91-29402719f719

📥 Commits

Reviewing files that changed from the base of the PR and between 5670473 and 0c0adfd.

📒 Files selected for processing (20)
  • Makefile
  • README.md
  • hack/govulncheck.sh
  • images/ci/Dockerfile.codecoverage
  • test/README.md
  • test/apis/README.md
  • test/e2e/README.md
  • test/e2e/bitwarden_api_test.go
  • test/e2e/bitwarden_es_test.go
  • test/e2e/e2e_suite_test.go
  • test/e2e/e2e_test.go
  • test/go.mod
  • test/utils/artifact_dump.go
  • test/utils/aws_resources.go
  • test/utils/bitwarden.go
  • test/utils/bitwarden_api_runner.go
  • test/utils/bitwarden_resources.go
  • test/utils/cleanup.go
  • test/utils/conditions.go
  • test/utils/dynamic_resources.go
✅ Files skipped from review due to trivial changes (6)
  • README.md
  • test/README.md
  • test/apis/README.md
  • hack/govulncheck.sh
  • test/go.mod
  • test/e2e/README.md

@siddhibhor-56 siddhibhor-56 force-pushed the codecov branch 3 times, most recently from 3604751 to 8f666be Compare May 6, 2026 18:23
Copy link
Copy Markdown

@coderabbitai coderabbitai Bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 2

🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Inline comments:
In `@hack/e2e-coverage.sh`:
- Around line 97-118: The collect() function can leave the coverage-extractor
pod running if a later command fails; add a cleanup trap so the pod is always
removed on EXIT. Implement a small cleanup function (e.g., cleanup_extractor)
that runs oc delete pod coverage-extractor -n "${NAMESPACE}" --ignore-not-found
--wait=false 2>/dev/null || true, set trap 'cleanup_extractor' EXIT immediately
after creating the pod (right after the oc run/oc wait sequence), and
remove/reset the trap (trap - EXIT) once files are successfully copied and
before returning so normal completion doesn't trigger redundant cleanup.
- Around line 58-63: The CSV patch currently uses append ops to add env,
volumeMounts and volumes which will create duplicates if run twice; modify the
oc patch logic around oc patch "${csv}" to first check for an existing env entry
named GOCOVERDIR (or existing volume/volumeMount named coverage-data) and skip
the corresponding add ops if present, or alternatively replace the whole
container spec in
/spec/install/spec/deployments/0/spec/template/spec/containers/0 with a
sanitized container object that sets image to COVERAGE_IMAGE and sets env
GOCOVERDIR to GOCOVERDIR_PATH and the volumeMounts/volumes (using PVC_NAME),
ensuring you update the patch operations to use replace on the container array
or conditional logic to avoid appending duplicates.
🪄 Autofix (Beta)

Fix all unresolved CodeRabbit comments on this PR:

  • Push a commit to this branch (recommended)
  • Create a new PR with the fixes

ℹ️ Review info
⚙️ Run configuration

Configuration used: Repository: openshift/coderabbit/.coderabbit.yaml

Review profile: CHILL

Plan: Enterprise

Run ID: a6f2a1e6-94ca-4808-93c6-5d2746b62eb7

📥 Commits

Reviewing files that changed from the base of the PR and between 8f666be and a467b24.

📒 Files selected for processing (3)
  • Makefile
  • hack/e2e-coverage.sh
  • images/ci/Dockerfile.coverage

Comment thread hack/e2e-coverage.sh
Comment thread hack/e2e-coverage.sh Outdated
Copy link
Copy Markdown

@coderabbitai coderabbitai Bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 1

🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Inline comments:
In `@hack/govulncheck.sh`:
- Around line 27-29: The long-lived suppression in KNOWN_VULNS_PATTERN currently
hardcodes operator-impacting vuln IDs (e.g., GO-2026-4971, GO-2026-4918); change
this to be temporary by either removing these IDs entirely or replacing them
with an explicit temporary allowlist that includes an expiry date and an
owner/issue reference (e.g., add metadata comment or use an expirable mechanism
like ENV var or a separate TEMP_ALLOWLIST variable) so future reintroductions
aren’t silently ignored; locate the KNOWN_VULNS_PATTERN declaration and update
it to reference the temporary mechanism or remove the operator-impacting IDs and
add a comment noting the owner, expiry (YYYY-MM-DD), and tracking issue number.
🪄 Autofix (Beta)

Fix all unresolved CodeRabbit comments on this PR:

  • Push a commit to this branch (recommended)
  • Create a new PR with the fixes

ℹ️ Review info
⚙️ Run configuration

Configuration used: Repository: openshift/coderabbit/.coderabbit.yaml

Review profile: CHILL

Plan: Enterprise

Run ID: 17213dee-2264-4a7c-8ca0-61150ae6ce21

📥 Commits

Reviewing files that changed from the base of the PR and between a467b24 and b9a8973.

📒 Files selected for processing (2)
  • hack/govulncheck.sh
  • images/ci/Dockerfile.coverage
🚧 Files skipped from review as they are similar to previous changes (1)
  • images/ci/Dockerfile.coverage

Comment thread hack/govulncheck.sh
Comment on lines +27 to +29
# - https://pkg.go.dev/vuln/GO-2026-4971 - Dial and LookupPort panic on Windows with NUL input in net
# - https://pkg.go.dev/vuln/GO-2026-4918 - HTTP/2 infinite loop via SETTINGS_MAX_FRAME_SIZE of 0 in net/http, golang.org/x/net
KNOWN_VULNS_PATTERN="GO-2025-3521|GO-2025-3547|GO-2026-4601|GO-2026-4602|GO-2026-4971|GO-2026-4918"
Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

⚠️ Potential issue | 🟠 Major | ⚡ Quick win

Avoid permanent suppression of operator-impacting vuln IDs.

These IDs are documented as impacting operator code, but adding them directly to the long-lived allowlist means future reintroductions will stay silent in CI. Make these suppressions explicitly temporary (expiry + owner/issue), or remove them so the pipeline blocks until fixed downstream.

Suggested hardening (temporary suppression with expiry)
+# Temporary suppressions for downstream-unavailable fixes.
+# Must be revisited before this date.
+TEMP_IGNORE_UNTIL="2026-06-30"
+if [[ "$(date +%F)" > "${TEMP_IGNORE_UNTIL}" ]]; then
+    echo "-- ERROR -- Temporary govulncheck suppressions expired on ${TEMP_IGNORE_UNTIL}"
+    exit 1
+fi
+
 KNOWN_VULNS_PATTERN="GO-2025-3521|GO-2025-3547|GO-2026-4601|GO-2026-4602|GO-2026-4971|GO-2026-4918"
📝 Committable suggestion

‼️ IMPORTANT
Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation. Thoroughly test & benchmark the code to ensure it meets the requirements.

Suggested change
# - https://pkg.go.dev/vuln/GO-2026-4971 - Dial and LookupPort panic on Windows with NUL input in net
# - https://pkg.go.dev/vuln/GO-2026-4918 - HTTP/2 infinite loop via SETTINGS_MAX_FRAME_SIZE of 0 in net/http, golang.org/x/net
KNOWN_VULNS_PATTERN="GO-2025-3521|GO-2025-3547|GO-2026-4601|GO-2026-4602|GO-2026-4971|GO-2026-4918"
# - https://pkg.go.dev/vuln/GO-2026-4971 - Dial and LookupPort panic on Windows with NUL input in net
# - https://pkg.go.dev/vuln/GO-2026-4918 - HTTP/2 infinite loop via SETTINGS_MAX_FRAME_SIZE of 0 in net/http, golang.org/x/net
# Temporary suppressions for downstream-unavailable fixes.
# Must be revisited before this date.
TEMP_IGNORE_UNTIL="2026-06-30"
if [[ "$(date +%F)" > "${TEMP_IGNORE_UNTIL}" ]]; then
echo "-- ERROR -- Temporary govulncheck suppressions expired on ${TEMP_IGNORE_UNTIL}"
exit 1
fi
KNOWN_VULNS_PATTERN="GO-2025-3521|GO-2025-3547|GO-2026-4601|GO-2026-4602|GO-2026-4971|GO-2026-4918"
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@hack/govulncheck.sh` around lines 27 - 29, The long-lived suppression in
KNOWN_VULNS_PATTERN currently hardcodes operator-impacting vuln IDs (e.g.,
GO-2026-4971, GO-2026-4918); change this to be temporary by either removing
these IDs entirely or replacing them with an explicit temporary allowlist that
includes an expiry date and an owner/issue reference (e.g., add metadata comment
or use an expirable mechanism like ENV var or a separate TEMP_ALLOWLIST
variable) so future reintroductions aren’t silently ignored; locate the
KNOWN_VULNS_PATTERN declaration and update it to reference the temporary
mechanism or remove the operator-impacting IDs and add a comment noting the
owner, expiry (YYYY-MM-DD), and tracking issue number.

@siddhibhor-56
Copy link
Copy Markdown
Contributor Author

/test verify

@siddhibhor-56 siddhibhor-56 changed the title adds the e2e coverage collector for the operator using codecov OAPE-693:adds the e2e coverage collector for the operator using codecov May 11, 2026
@openshift-ci-robot openshift-ci-robot added the jira/valid-reference Indicates that this PR references a valid Jira ticket of any type. label May 11, 2026
@openshift-ci-robot
Copy link
Copy Markdown

openshift-ci-robot commented May 11, 2026

@siddhibhor-56: This pull request references OAPE-693 which is a valid jira issue.

Warning: The referenced jira issue has an invalid target version for the target branch this PR targets: expected the story to target the "5.0.0" version, but no target version was set.

Details

In response to this:

Adds the e2e coverage collector for the operator using codecov

Summary by CodeRabbit

  • Chores
  • Added end-to-end coverage support: builds and publishes a coverage-instrumented image and collects coverage data during E2E runs.
  • Added tooling to extract, convert, and optionally upload E2E coverage reports to Codecov (upload skipped if no token).
  • Improved CI/local workflows for gathering E2E coverage artifacts.
  • Expanded the set of ignored vulnerability IDs in the vulnerability scan tooling.

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the openshift-eng/jira-lifecycle-plugin repository.

@openshift-ci
Copy link
Copy Markdown

openshift-ci Bot commented May 14, 2026

@siddhibhor-56: all tests passed!

Full PR test history. Your PR dashboard.

Details

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository. I understand the commands that are listed here.

Comment thread hack/govulncheck.sh
KNOWN_VULNS_PATTERN="GO-2025-3521|GO-2025-3547|GO-2026-4601|GO-2026-4602"
# - https://pkg.go.dev/vuln/GO-2026-4971 - Dial and LookupPort panic on Windows with NUL input in net
# - https://pkg.go.dev/vuln/GO-2026-4918 - HTTP/2 infinite loop via SETTINGS_MAX_FRAME_SIZE of 0 in net/http, golang.org/x/net
KNOWN_VULNS_PATTERN="GO-2025-3521|GO-2025-3547|GO-2026-4601|GO-2026-4602|GO-2026-4971|GO-2026-4918"
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
KNOWN_VULNS_PATTERN="GO-2025-3521|GO-2025-3547|GO-2026-4601|GO-2026-4602|GO-2026-4971|GO-2026-4918"
KNOWN_VULNS_PATTERN="GO-2025-3521|GO-2025-3547|GO-2026-4971|GO-2026-4918"

Comment thread Makefile
##
## Targets for building a coverage-instrumented operator image, collecting
## coverage data written during E2E tests, and uploading the report to Codecov.
## Uses emptyDir (no PVC): the collect step sends SIGTERM to the operator
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Does the comment need an update regarding PVC?

Comment thread hack/e2e-coverage.sh
if [[ -n "${CODECOV_TOKEN:-}" ]]; then
echo "Uploading to Codecov..."
local codecov_bin="${artifact_dir}/codecov"
curl -sS -o "${codecov_bin}" https://uploader.codecov.io/latest/linux/codecov
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We should pin and download a particular version, which will help with debugging.

Comment thread hack/e2e-coverage.sh
]"

echo "Waiting for operator rollout with coverage image..."
sleep 5
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We should poll or instead use oc wait, this could cause flakiness.

Comment thread hack/e2e-coverage.sh
Comment on lines +84 to +85
sleep 10
oc wait pod/"${pod}" --for=condition=Ready -n "${NAMESPACE}" --timeout=120s
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Do we need both sleep and wait?

Comment thread hack/e2e-coverage.sh
--file="${coverage_profile}"
--flags=e2e
--name="E2E Coverage"
--verbose
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Do we really need it to be verbose, if so, please make sure, token is not leaked in the logs.

-cover -covermode=count -coverpkg=./... \
-o external-secrets-operator cmd/external-secrets-operator/main.go

FROM registry.access.redhat.com/ubi9-minimal:9.4
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
FROM registry.access.redhat.com/ubi9-minimal:9.4
FROM registry.access.redhat.com/ubi9/ubi-minimal:latest

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

jira/valid-reference Indicates that this PR references a valid Jira ticket of any type.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants