Skip to content

OLS-3218 - Migrating prow integration tests to konflux and preparation of konflux pipelines#70

Open
JoaoFula wants to merge 3 commits into
openshift:mainfrom
JoaoFula:implement-ols-3218
Open

OLS-3218 - Migrating prow integration tests to konflux and preparation of konflux pipelines#70
JoaoFula wants to merge 3 commits into
openshift:mainfrom
JoaoFula:implement-ols-3218

Conversation

@JoaoFula

@JoaoFula JoaoFula commented Jun 5, 2026

Copy link
Copy Markdown

No description provided.

@coderabbitai

coderabbitai Bot commented Jun 5, 2026

Copy link
Copy Markdown

Note

Reviews paused

It looks like this branch is under active development. To avoid overwhelming you with review comments due to an influx of new commits, CodeRabbit has automatically paused this review. You can configure this behavior by changing the reviews.auto_review.auto_pause_after_reviewed_commits setting.

Use the following commands to manage reviews:

  • @coderabbitai resume to resume automatic reviews.
  • @coderabbitai review to trigger a single review.

Use the checkboxes below for quick actions:

  • ▶️ Resume reviews
  • 🔍 Trigger review
📝 Walkthrough

Walkthrough

This PR adds sandbox integration test infrastructure for three LLM providers. IntegrationTestScenario resources register test configurations, a Tekton Pipeline orchestrates execution by parsing SNAPSHOT revisions and mounting credentials, and a Bash script handles provider-specific credential setup and test invocation. A minor fix to container runtime detection improves error handling when neither podman nor docker is available.

Changes

Sandbox Integration Test Infrastructure

Layer / File(s) Summary
IntegrationTestScenario registration
.tekton/integration-tests/integration-test-scenarios/sandbox-integration-test-scenarios.yaml
Three IntegrationTestScenario resources (sandbox-integration-claude, sandbox-integration-gemini, sandbox-integration-openai) register optional test scenarios targeting the ols application with provider-specific provider and credential-secret parameters pointing to the shared sandbox pipeline.
Tekton pipeline orchestration
.tekton/integration-tests/pipelines/sandbox-integration-test-pipeline.yaml
Pipeline accepts SNAPSHOT, provider, and credential-secret parameters; parses SNAPSHOT JSON to extract the sandbox repo revision; mounts the credential secret at /var/run/credentials; clones the repository at that exact revision; and invokes run-sandbox-integration-tests.sh with the selected provider.
Test execution script with credential setup
.tekton/integration-tests/scripts/run-sandbox-integration-tests.sh
Script validates the provider argument (claude, gemini, or openai) and mounted credential file; configures provider-specific environment variables (Google ADC JSON path for Claude/Gemini with project-id parsing for Claude; OpenAI API key); installs pinned uv==0.11.19 with bounded retry logic; and invokes scripts/e2e-containers.sh in host mode with JUnit output and artifact directory configuration.
Container runtime detection and credential handling safety
scripts/e2e-containers.sh
Appends || true to the RUNTIME detection command to safely fallback to empty value when neither podman nor docker is available under set -e; adds conditional check for GOOGLE_APPLICATION_CREDENTIALS unset before copying ADC credentials in host mode to preserve existing credential configuration.
🚥 Pre-merge checks | ✅ 3 | ❌ 2

❌ Failed checks (2 warnings)

Check name Status Explanation Resolution
Description check ⚠️ Warning No pull request description was provided by the author. Add a pull request description explaining the changes, migration rationale, and any notable implementation details.
Docstring Coverage ⚠️ Warning Docstring coverage is 50.00% which is insufficient. The required threshold is 80.00%. Write docstrings for the functions missing them to satisfy the coverage threshold.
✅ Passed checks (3 passed)
Check name Status Explanation
Title check ✅ Passed The title directly references OLS-3218 and accurately describes the main objectives: migrating prow integration tests to Konflux and preparing Konflux pipelines.
Linked Issues check ✅ Passed Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check ✅ Passed Check skipped because no linked issues were found for this pull request.

✏️ Tip: You can configure your own custom pre-merge checks in the settings.


Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands and usage tips.

@openshift-ci openshift-ci Bot requested review from joshuawilson and xrajesh June 5, 2026 14:24
@openshift-ci

openshift-ci Bot commented Jun 5, 2026

Copy link
Copy Markdown

[APPROVALNOTIFIER] This PR is NOT APPROVED

This pull-request has been approved by:
Once this PR has been reviewed and has the lgtm label, please assign xrajesh for approval. For more information see the Code Review Process.

The full list of commands accepted by this bot can be found here.

Details Needs approval from an approver in each of these files:

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 2

🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Inline comments:
In @.tekton/integration-tests/pipelines/sandbox-integration-test-pipeline.yaml:
- Around line 27-40: The embedded taskSpec is missing the credential param used
by the volume secret, so add a matching param entry to taskSpec.params named
exactly "credential-secret" (type: string) so the substitution
$(params.credential-secret) inside taskSpec.volumes[].secret.secretName
resolves; update the taskSpec.params list (alongside SNAPSHOT and provider) to
include - name: credential-secret and ensure the param name matches the
pipeline-level param declaration.

In @.tekton/integration-tests/scripts/run-sandbox-integration-tests.sh:
- Around line 51-53: The script currently runs a dynamic, unpinned install via
the line "pip install --quiet uv"; change that to install a pinned, reproducible
version (e.g., replace with an exact spec like "uv==<exact-version>") and wrap
the install in a retry/backoff loop (limited attempts, exponential or fixed
backoff and fail fast with a clear error) so transient network failures don't
make CI flaky; update the install invocation in the install-check block that
tests for the "uv" command accordingly.
🪄 Autofix (Beta)

Fix all unresolved CodeRabbit comments on this PR:

  • Push a commit to this branch (recommended)
  • Create a new PR with the fixes

ℹ️ Review info
⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: CHILL

Plan: Enterprise

Run ID: 3bc8da03-5e86-4343-923a-60e5e81cd2e6

📥 Commits

Reviewing files that changed from the base of the PR and between b522f46 and babff72.

📒 Files selected for processing (3)
  • .tekton/integration-tests/integration-test-scenarios/sandbox-integration-test-scenarios.yaml
  • .tekton/integration-tests/pipelines/sandbox-integration-test-pipeline.yaml
  • .tekton/integration-tests/scripts/run-sandbox-integration-tests.sh

Comment thread .tekton/integration-tests/scripts/run-sandbox-integration-tests.sh
@JoaoFula

JoaoFula commented Jun 5, 2026

Copy link
Copy Markdown
Author

/retest

@JoaoFula

JoaoFula commented Jun 8, 2026

Copy link
Copy Markdown
Author

/retest

2 similar comments
@JoaoFula

JoaoFula commented Jun 8, 2026

Copy link
Copy Markdown
Author

/retest

@JoaoFula

Copy link
Copy Markdown
Author

/retest

@JoaoFula JoaoFula force-pushed the implement-ols-3218 branch from f468dcc to aa9c743 Compare June 15, 2026 13:54
@JoaoFula

Copy link
Copy Markdown
Author

/retest

@JoaoFula JoaoFula force-pushed the implement-ols-3218 branch from aa9c743 to 3b25578 Compare June 15, 2026 14:32
@JoaoFula

Copy link
Copy Markdown
Author

/retest

1 similar comment
@JoaoFula

Copy link
Copy Markdown
Author

/retest

@JoaoFula JoaoFula force-pushed the implement-ols-3218 branch 3 times, most recently from 3e2f33a to fe4f954 Compare June 16, 2026 13:09

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Caution

Some comments are outside the diff and can’t be posted inline due to platform limitations.

⚠️ Outside diff range comments (1)
.tekton/integration-tests/scripts/run-sandbox-integration-tests.sh (1)

40-42: ⚠️ Potential issue | 🟡 Minor | ⚡ Quick win

Trailing whitespace/newline may corrupt the API key.

cat preserves trailing newlines from the file. If the secret has a trailing newline (common with echo or editors), the API key will fail silently.

Suggested fix
-    OPENAI_API_KEY=$(cat "${CRED_PATH}")
+    OPENAI_API_KEY=$(tr -d '[:space:]' < "${CRED_PATH}")
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In @.tekton/integration-tests/scripts/run-sandbox-integration-tests.sh around
lines 40 - 42, The OPENAI_API_KEY assignment reads from the file using cat,
which preserves trailing newlines and whitespace that will corrupt the API key
value. In the openai case block where OPENAI_API_KEY is assigned from cat
"${CRED_PATH}", strip trailing whitespace and newlines from the file content
(using a tool like tr to remove newlines or a similar approach) to ensure the
API key is clean before exporting it.
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Outside diff comments:
In @.tekton/integration-tests/scripts/run-sandbox-integration-tests.sh:
- Around line 40-42: The OPENAI_API_KEY assignment reads from the file using
cat, which preserves trailing newlines and whitespace that will corrupt the API
key value. In the openai case block where OPENAI_API_KEY is assigned from cat
"${CRED_PATH}", strip trailing whitespace and newlines from the file content
(using a tool like tr to remove newlines or a similar approach) to ensure the
API key is clean before exporting it.

ℹ️ Review info
⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: CHILL

Plan: Enterprise

Run ID: 94bb1461-9774-413d-bece-b59954e7e7b5

📥 Commits

Reviewing files that changed from the base of the PR and between 1794610 and fe4f954.

📒 Files selected for processing (4)
  • .tekton/integration-tests/integration-test-scenarios/sandbox-integration-test-scenarios.yaml
  • .tekton/integration-tests/pipelines/sandbox-integration-test-pipeline.yaml
  • .tekton/integration-tests/scripts/run-sandbox-integration-tests.sh
  • scripts/e2e-containers.sh
🔗 Linked repositories identified

CodeRabbit considers these linked repositories for cross-repo context during reviews:

  • openshift/lightspeed-agentic-operator (manual)
🚧 Files skipped from review as they are similar to previous changes (2)
  • .tekton/integration-tests/integration-test-scenarios/sandbox-integration-test-scenarios.yaml
  • .tekton/integration-tests/pipelines/sandbox-integration-test-pipeline.yaml

@JoaoFula JoaoFula force-pushed the implement-ols-3218 branch from fe4f954 to a4b2727 Compare June 16, 2026 14:26
@openshift-ci openshift-ci Bot added the needs-rebase Indicates a PR cannot be merged because it has merge conflicts with HEAD. label Jun 16, 2026
JoaoFula added 2 commits June 16, 2026 16:30
Sets up provider credentials from mounted Konflux workspace secrets
and runs BDD tests in host mode via e2e-containers.sh.
Parameterized pipeline that clones the sandbox repo at the SNAPSHOT
revision and delegates to run-sandbox-integration-tests.sh.
Accepts provider and credential-secret params from ITS CRs.
@JoaoFula JoaoFula force-pushed the implement-ols-3218 branch from a4b2727 to cc40675 Compare June 16, 2026 14:31
@openshift-ci openshift-ci Bot removed the needs-rebase Indicates a PR cannot be merged because it has merge conflicts with HEAD. label Jun 16, 2026
@JoaoFula

Copy link
Copy Markdown
Author

/retest

Three ITS CRs (claude, gemini, openai) trigger on sandbox component
builds. Each passes provider and credential-secret params to the
shared sandbox-integration-test-pipeline.

removing e2e presubmits

adding except for oauth and reforcing logic around filtering for events

adding except for oauth and reforcing logic around filtering for events

adding except for oauth and reforcing logic around filtering for events

adding except for oauth and reforcing logic around filtering for events

adding except for oauth and reforcing logic around filtering for events

adding except for oauth and reforcing logic around filtering for events

adding except for oauth and reforcing logic around filtering for events
@JoaoFula JoaoFula force-pushed the implement-ols-3218 branch from cc40675 to 7d3f18d Compare June 17, 2026 08:49
@openshift-ci

openshift-ci Bot commented Jun 17, 2026

Copy link
Copy Markdown

@JoaoFula: all tests passed!

Full PR test history. Your PR dashboard.

Details

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository. I understand the commands that are listed here.

@JoaoFula

Copy link
Copy Markdown
Author

/retest

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant