product-lifecycle: Add plc_lookup.py CLI, migrate to v2 API by harche · Pull Request #13 · openshift/agentic-skills

harche · 2026-04-21T22:26:32Z

Summary

Replaces inline curl/python one-liners in product-lifecycle SKILL.md with a standalone Python CLI (plc_lookup.py) wrapping the Red Hat Product Life Cycle API v2
Addresses review feedback from PR OTA-1963: skills: Add cluster-update skills (update-advisor, product-lifecycle) #6: v1→v2 migration, all 5 support status types, drop PLCC acronym, proper error handling
Adds pagination (--limit, --offset) for broad queries
Includes 46 tests (unit with mocked API + integration against live API)
Updates update-advisor SKILL.md cross-references

Builds on top of #6 (that PR stays open).

CLI usage

python3 cluster-update/product-lifecycle/scripts/plc_lookup.py products "logging for Red Hat OpenShift"
python3 cluster-update/product-lifecycle/scripts/plc_lookup.py products "OpenShift" --ocp 4.21 --limit 5
python3 cluster-update/product-lifecycle/scripts/plc_lookup.py olm-check --ocp 4.21 --operators '[{"package":"cluster-logging"}]'

Test plan

All 46 tests pass locally (python3 -m unittest — 19 unit + 17 integration against live API + 10 pagination)
Verify plc_lookup.py -h shows clean help output
Verify script works from repo root path
Verify SKILL.md paths resolve correctly

🤖 Generated with Claude Code

openshift-ci · 2026-04-21T22:26:49Z

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: harche

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Details

Needs approval from an approver in each of these files:

~~OWNERS~~ [harche]

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

harche · 2026-04-22T20:47:17Z

/hold

will look into this after #6

harche · 2026-05-29T15:28:22Z

Testing Results

Eval Framework (Agent-in-the-loop)

Ran 12 skill evals via the agentic-skills eval framework using Claude on Vertex AI. Test queries use the exact prompt format CVO proposals send to the agent, with real OLM operator data and readiness JSON.

product-lifecycle (7 evals)

Test Case	Query	Expected	Result
`plc_proposal_olm_batch_check`	5-operator batch olm-check from proposal context	5 checked, 3 unavailable	✅
`plc_cluster_logging_supported`	cluster-logging v6.5 lifecycle + OCP 4.21 compat	supported, compatible	✅
`plc_web_terminal_compat_check`	web-terminal supported version for OCP 4.21	no supported version for 4.21	✅
`plc_compliance_operator_status`	compliance-operator v1.9 lifecycle status	found, supported	✅
`plc_ocp_platform_status`	OCP 4.21 platform lifecycle	supported	✅
`plc_ocp_old_version_extended`	OCP 4.14 lifecycle phase	extended	✅
`plc_batch_known_operators_only`	2-operator batch check (cluster-logging + web-terminal)	both found, neither EOL	✅

update-advisor (5 evals)

Each test sends a complete readiness JSON (same format CVO produces) and verifies the agent's upgrade decision:

Test Case	Scenario	Expected Decision	Result
`advisor_healthy_cluster_recommend`	All 9 checks pass, no issues	recommend	✅
`advisor_degraded_operator_caution`	1 degraded operator (authentication)	caution	✅
`advisor_api_deprecation_block`	Removed API with 1250 active requests	block	✅
`advisor_etcd_unhealthy_block`	1 of 3 etcd members not ready	block	✅
`advisor_errored_checks_escalate`	7 of 9 readiness checks errored (API unreachable)	escalate	✅

All 12 agent evals + 46 unit/integration tests passed.

End-to-End with CVO Readiness Data

Tested the full proposal → readiness → skill pipeline using a custom CVO build from openshift/cluster-version-operator#1395 deployed to a live OCP 4.21.5 cluster on GCP (6 nodes).

Installed OLM operators for testing:

cluster-logging v6.5.1 (channel: stable-6.5)
compliance-operator v1.9.0 (channel: stable)
openshift-pipelines-operator-rh v1.22.0 (channel: latest)
web-terminal v1.16.0 (channel: fast)
devworkspace-operator v0.41.0 (channel: fast)

Flow verified:

CVO creates proposals with readiness JSON containing olm_operator_lifecycle data for all 5 operators
The update-advisor skill analyzes readiness data and classifies findings per its decision policy
The update-advisor calls product-lifecycle to cross-check operators against the PLC API
plc_lookup.py olm-check correctly returns lifecycle data for cluster-logging and web-terminal, reports "unavailable" for compliance-operator, openshift-pipelines, and devworkspace-operator
plc_lookup.py products finds compliance-operator by product name even when olm-check can't match by package name

Cluster readiness summary (from proposal ota-4-21-5-to-4-21-16):

9/9 readiness checks passed in 2.56s
34 ClusterOperators healthy, 6 nodes ready, 3/3 etcd members
207 CRDs (0 version issues), 21 PDBs (0 blocking)
OVNKubernetes, TLS Intermediate, no deprecated APIs
5 OLM operators detected with versions, channels, and install status

Ground truth verification

All expected values in evals were verified against the live PLC v2 API:

$ plc_lookup.py products "Red Hat OpenShift Container Platform" --ocp 4.21  →  4.21: supported
$ plc_lookup.py products "Red Hat OpenShift Container Platform" --ocp 4.14  →  4.14: extended
$ plc_lookup.py products "logging for Red Hat OpenShift" --ocp 4.21         →  6.5: supported, ocp_compatible=true
$ plc_lookup.py products "compliance operator"                              →  v1.9: supported
$ plc_lookup.py olm-check --ocp 4.21 --operators '[5 operators]'           →  2 found, 3 unavailable

🤖 Generated with Claude Code

harche · 2026-05-29T15:33:50Z

/cc @wking

harche · 2026-05-29T15:48:32Z

/hold improving evals with better data.

harche · 2026-05-29T17:46:30Z

Converting to draft, I need to make a few more changes before it is ready for the review.

harche · 2026-05-29T18:38:35Z

Converting to draft, I need to make a few more changes before it is ready for the review.

back to open.

harche · 2026-06-02T14:19:13Z

Cross-PR testing

Tested end-to-end with:

openshift/cluster-version-operator#1395 — CVO readiness checks + proposal wiring + typed analysis schema
openshift/cluster-update-console-plugin#8 — Console UI with dropdown selector, expandable analysis panels, typed component rendering

All three deployed together on OCP 4.21.5 cluster. The product-lifecycle skill (plc_lookup.py) successfully runs inside the agent sandbox, querying the PLCC API and populating operator lifecycle data in the ota_olm_operator_status component.

The skill path fix (cluster-update/... → product-lifecycle/...) is required for the sandbox mount paths to resolve correctly.

harche · 2026-06-02T14:19:31Z

/override ci/prow/eval

openshift-ci · 2026-06-02T14:20:53Z

@harche: Overrode contexts on behalf of harche: ci/prow/eval

Details

In response to this:

/override ci/prow/eval

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository.

harche · 2026-06-03T20:22:28Z

/hold check if we can enhance this based on https://github.com/openshift-kni/ai-sandbox/blob/main/rds-policy/rds_agent/skills/rds-policy-update/SKILL.md

harche · 2026-06-04T14:42:57Z

OTA-1972: Update API types to match current agentic.openshift.io CRDs openshift/cluster-update-console-plugin#8

/hold cancel

will take it up as follow up.

Cali0707 · 2026-06-12T17:45:41Z

@@ -0,0 +1,132 @@
+# product-lifecycle eval


@harche maybe we want to mirror the OWNERS from the main skill into here somehow so that the same group of people can own the skill + the evals?

For this PR to move forwarded I have added the symlink to the OWNERS file evals/skills/product-lifecycle/OWNERS → ../../../cluster-update/OWNERS

harche · 2026-06-15T18:47:30Z

/override ci/prow/eval

openshift-ci · 2026-06-15T18:49:16Z

@harche: Overrode contexts on behalf of harche: ci/prow/eval

Details

In response to this:

/override ci/prow/eval

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository.

jrangelramos · 2026-06-15T20:43:43Z

/lgtm

wking · 2026-06-15T20:57:58Z

@@ -1,9 +1,9 @@
-# Product Life Cycle API Reference
+# Product Life Cycle API Reference (v2)


nit: as far as the skill goes, it doesn't need to care about v1 vs. v2, right? Actually, we don't even need this reference at all now that the API is wrapped in plc_lookup.py? This api-details.md should pivot to just being about "how to use plc_lookup.py"?

Good call — api-details.md was orphaned (nothing in the repo linked to it), and SKILL.md already fully documents the plc_lookup.py CLI. Rather than pivoting it to a "how to use plc_lookup.py" doc that would duplicate SKILL.md, I removed the reference file entirely.

wking · 2026-06-15T21:03:23Z

+When the readiness data includes olm_operator_lifecycle results, use
+the product-lifecycle skill to cross-reference each operator's package
+name against the Red Hat Product Life Cycle API. Report support phase,
+EOL dates, and OCP compatibility from PLCC alongside the OLM data.


nit: I it looks like evals are the only place in this repo we use the PLCC acronym as of this pull:

$ git grep -i plcc origin/pr/13 | grep -v png origin/pr/13:evals/skills/product-lifecycle/system_prompt.md:EOL dates, and OCP compatibility from PLCC alongside the OLM data. origin/pr/13:evals/skills/update-advisor/system_prompt.md:EOL dates, and OCP compatibility from PLCC alongside the OLM data.

I think we should use "Product Life Cycle" or "Red Hat Product Life Cycle" or some other string drawn from cluster-update/product-lifecycle/SKILL.md to make it easy for AI to make the connection to that skill.

Agreed. Replaced PLCC with "the product-lifecycle skill" in both eval system prompts (product-lifecycle and update-advisor) so the wording is drawn from the skill and easier for the model to connect. No PLCC left anywhere in the repo.

Add standalone Python CLI (`plc_lookup.py`) wrapping the Red Hat Product Life Cycle API v2 for querying product support status, EOL dates, and OCP version compatibility. Replaces inline curl/python one-liners. Commands: - `products` — query by product name with optional OCP compat check - `olm-check` — batch check OLM operators against a target OCP version Includes 46 tests (unit with mocked API + integration against live API), 3 agent eval test cases (all passing on Claude via Vertex AI), and OWNERS updates. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

openshift-ci · 2026-06-16T12:47:57Z

New changes are detected. LGTM label has been removed.

harche · 2026-06-16T13:02:23Z

/override ci/prow/eval

openshift-ci · 2026-06-16T13:03:03Z

@harche: Overrode contexts on behalf of harche: ci/prow/eval

Details

In response to this:

/override ci/prow/eval

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository.

openshift-ci · 2026-06-16T13:03:06Z

@harche: all tests passed!

Full PR test history. Your PR dashboard.

Details

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository. I understand the commands that are listed here.

openshift-ci Bot requested review from DavidHurta and mrunalp April 21, 2026 22:26

openshift-ci Bot added the approved Indicates a PR has been approved by an approver from all required OWNERS files. label Apr 21, 2026

harche mentioned this pull request Apr 22, 2026

OTA-1963: skills: Add cluster-update skills (update-advisor, product-lifecycle) #6

Merged

4 tasks

openshift-ci Bot added the do-not-merge/hold Indicates that a PR should not merge because someone has issued a /hold command. label Apr 22, 2026

harche force-pushed the skills/cluster-update-plc-script branch from ce82a3e to b0a3f09 Compare May 29, 2026 15:24

harche mentioned this pull request May 29, 2026

pkg/readiness: Add readiness checks and wire into proposal controller openshift/cluster-version-operator#1395

Open

5 tasks

openshift-ci Bot requested a review from wking May 29, 2026 15:33

harche force-pushed the skills/cluster-update-plc-script branch from 7e43631 to eace3b2 Compare May 29, 2026 15:41

harche force-pushed the skills/cluster-update-plc-script branch 4 times, most recently from f01f289 to 81bf078 Compare May 29, 2026 16:38

harche marked this pull request as draft May 29, 2026 17:46

openshift-ci Bot added the do-not-merge/work-in-progress Indicates that a PR should not merge because it is a work in progress. label May 29, 2026

harche force-pushed the skills/cluster-update-plc-script branch from 81bf078 to 0544f37 Compare May 29, 2026 17:53

harche marked this pull request as ready for review May 29, 2026 18:38

openshift-ci Bot removed the do-not-merge/work-in-progress Indicates that a PR should not merge because it is a work in progress. label May 29, 2026

openshift-ci Bot requested review from Cali0707 and PratikMahajan May 29, 2026 18:38

harche force-pushed the skills/cluster-update-plc-script branch from 0544f37 to ef3bf25 Compare June 1, 2026 20:38

harche mentioned this pull request Jun 2, 2026

OTA-1972: Update API types to match current agentic.openshift.io CRDs openshift/cluster-update-console-plugin#8

Merged

3 tasks

openshift-ci Bot removed the do-not-merge/hold Indicates that a PR should not merge because someone has issued a /hold command. label Jun 4, 2026

Cali0707 reviewed Jun 12, 2026

View reviewed changes

harche force-pushed the skills/cluster-update-plc-script branch from ef3bf25 to 94bab65 Compare June 15, 2026 18:37

wking reviewed Jun 15, 2026

View reviewed changes

openshift-ci Bot assigned jrangelramos Jun 15, 2026

openshift-ci Bot added the lgtm Indicates that a PR is ready to be merged. label Jun 15, 2026

harche force-pushed the skills/cluster-update-plc-script branch from 94bab65 to 27b285a Compare June 16, 2026 12:47

openshift-ci Bot removed the lgtm Indicates that a PR is ready to be merged. label Jun 16, 2026

		@@ -1,9 +1,9 @@
		# Product Life Cycle API Reference
		# Product Life Cycle API Reference (v2)

Conversation

harche commented Apr 21, 2026

Summary

CLI usage

Test plan

Uh oh!

openshift-ci Bot commented Apr 21, 2026

Uh oh!

harche commented Apr 22, 2026

Uh oh!

harche commented May 29, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Testing Results

Eval Framework (Agent-in-the-loop)

product-lifecycle (7 evals)

update-advisor (5 evals)

End-to-End with CVO Readiness Data

Ground truth verification

Uh oh!

harche commented May 29, 2026

Uh oh!

harche commented May 29, 2026

Uh oh!

harche commented May 29, 2026

Uh oh!

harche commented May 29, 2026

Uh oh!

harche commented Jun 2, 2026

Cross-PR testing

Uh oh!

harche commented Jun 2, 2026

Uh oh!

openshift-ci Bot commented Jun 2, 2026

Uh oh!

harche commented Jun 3, 2026

Uh oh!

harche commented Jun 4, 2026 • edited by openshift-ci Bot Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Cali0707 Jun 12, 2026

Choose a reason for hiding this comment

Uh oh!

harche Jun 15, 2026

Choose a reason for hiding this comment

Uh oh!

harche commented Jun 15, 2026

Uh oh!

openshift-ci Bot commented Jun 15, 2026

Uh oh!

jrangelramos commented Jun 15, 2026

Uh oh!

wking Jun 15, 2026

Choose a reason for hiding this comment

Uh oh!

harche Jun 16, 2026

Choose a reason for hiding this comment

Uh oh!

wking Jun 15, 2026

Choose a reason for hiding this comment

Uh oh!

harche Jun 16, 2026

Choose a reason for hiding this comment

Uh oh!

openshift-ci Bot commented Jun 16, 2026

Uh oh!

harche commented Jun 16, 2026

Uh oh!

openshift-ci Bot commented Jun 16, 2026

Uh oh!

openshift-ci Bot commented Jun 16, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

harche commented May 29, 2026 •

edited

Loading

harche commented Jun 4, 2026 •

edited by openshift-ci Bot

Loading