Repository navigation
feat(automations): add the GitHub CVE Remediation Factory template and skill - #948
Conversation
… Factory branch Points @openhands/extensions at a tarball of OpenHands/extensions#948 so the preview image carries the new template. Replace with the released version before merging. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Points canvas-server at the agent-canvas PR image (OpenHands/OpenHands#19700) and sandboxes at the extensions branch (OpenHands/extensions#948) through EXTENSIONS_REF. Becomes the agent-canvas version bump once both are released. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Points canvas-server at the agent-canvas PR image (OpenHands/OpenHands#19700) and sandboxes at the extensions branch (OpenHands/extensions#948) through EXTENSIONS_REF. Do not merge: at release, replace the image tag with the agent-canvas release and remove EXTENSIONS_REF.
…d skill Adds a featured "GitHub CVE Remediation Factory" catalog card with an assisted setup form, a shield-check icon, and the cve-remediation-factory skill. The skill scripts a short conversation (plan, go-ahead, done), builds an inventory of shipped images and source repositories, reuses pull-only CI logins for private registries, detects existing factories and remediators, and seeds a private findings repository with a daily Trivy + Grype scan, a per-tool results report and a weekday remediation automation.
88b9937 to
b82d5a9
Compare
…the CVE Remediation Factory Assisted templates now start an automation-setup conversation: the seed is sent as the first message, the conversation is tagged as automation setup, and the draft editor opens beside it. Maps the shield-check manifest icon and updates the card-order tests for the CVE Remediation Factory, now the first featured template. Temporarily builds against a tarball of OpenHands/extensions#948; replace it with the released version before merging.
Do not merge yet. Until the releases exist, canvas-server uses the agent-canvas image built for OpenHands/OpenHands#19700 and sandboxes load skills from the OpenHands/extensions#948 branch through EXTENSIONS_REF. At release, set the agent-canvas tag to the release that includes #19700 and remove EXTENSIONS_REF.
all-hands-bot
left a comment
There was a problem hiding this comment.
This review was posted by an AI agent (OpenHands).
Scope: this repository owns the automations catalog and the skills registry, so an automation template plus its skill belongs here; no product or architecture decision is needed.
Verification on head b82d5a9 (re-fetched today): all 13 GitHub Actions checks pass on this exact SHA (CodeQL, test, check, sync-extensions, sync-sdk-skill, PR description check on the latest run). Locally I ran 414 tests across test_automation_setup, test_catalogs, test_catalog_schema, test_skills_catalog, test_skills_have_readme, test_interface_manifest, test_plugin_manifest and test_check_pr_description - all pass. The manifest validates against automations/catalog.schema.json (including the new shield-check icon enum in both schemas), and npm run build:skills / build:automations reproduce every generated file byte-identically; sync_extensions.py --check is clean beyond pre-existing non-blocking coverage warnings.
Scripts exercised, not just read: template/scripts/scan.py plan/collect/history end to end on synthetic Trivy/Grype reports - GHSA-to-CVE mapping via match-level relatedVulnerabilities, fixed_version = higher of the two scanners, severity normalization, per-tool counts, the results README and trend SVG all render correctly, including failed targets and single-day history. seed.py registers all 11 documented subcommands, parse_ref handles Docker Hub shorthand/library/, digests, ECR, ACR and localhost correctly, and install_logins injects a reused pull-only login with continue-on-error, strips id:/if: keys, adds id-token: write only for OIDC logins, and produces a workflow that still parses. fill_factory placeholder substitution verified.
Security posture is sound: the token travels only in Authorization headers (never in URLs, arguments, or output; whoami returns credential names, not values); the remediator prompt defaults to propose, forbids merging, force-pushes, ignore files, VEX entries and scanner-config tampering; find-access only reuses logins from jobs whose steps do not publish or deploy; the scan workflow pins actions/checkout by SHA and uses least-privilege permissions with a concurrency guard.
One note on the existing CodeQL alert at seed.py:1016 ("clear-text logging of sensitive information"): false positive. Line 1016 is main's final print(json.dumps(result, ...)); what flows to stdout there is credential names (token_env, TOKEN_ENVS entries, reused-login secret names from find-access), never values, matching the module docstring's "Never prints a token" contract.
One finding needs a decision before merge - the defaultEnabled: true flag on this skill's marketplace entry contradicts AGENTS.md, which says the default-enabled set stays small and provider-agnostic, and that language-, vendor- or workflow-specific skills "should start off and be opted into from the catalog UI". This skill is GitHub-only and is a specific workflow (CVE remediation), yet it now ships in DEFAULT_ENABLED_SKILL_NAMES for every new workspace. Drop the flag, or record a maintainer exception if being default-on is intentional for a featured security template.
🔄 CHANGES REQUESTED
| "source": "./skills/cve-remediation-factory", | ||
| "description": "Set up a private CVE remediation factory on GitHub: inventory and scan the container images you ship, then open small upgrade pull requests for fixable CRITICAL and HIGH vulnerabilities.", | ||
| "category": "automations", | ||
| "defaultEnabled": true, |
There was a problem hiding this comment.
AGENTS.md sets the policy for this flag: the default-enabled set should stay small and provider-agnostic, and anything language-, vendor- or workflow-specific "should start off and be opted into from the catalog UI". This skill is GitHub-only (SKILL.md: "GitHub only") and is a specific workflow (CVE remediation), so it meets both exclusions, yet defaultEnabled: true ships it in DEFAULT_ENABLED_SKILL_NAMES for every new workspace. Please drop the flag so users opt in from the catalog UI, or note the maintainer-approved exception if default-on is intentional for a featured security template.
| RUN_URL: ${{ github.server_url }}/${{ github.repository }}/actions/runs/${{ github.run_id }} | ||
| REGISTRY_HOST: ${{ vars.REGISTRY_HOST || 'ghcr.io' }} | ||
| REGISTRY_USER: ${{ vars.REGISTRY_USER || github.actor }} | ||
| REGISTRY_PASSWORD: ${{ secrets.REGISTRY_PASSWORD || secrets.GITHUB_TOKEN }} |
There was a problem hiding this comment.
This fallback collides with the docs. registries.md, "Other registries" tells people to save REGISTRY_PASSWORD for their own registry and add a login step, but here REGISTRY_HOST still defaults to ghcr.io. So once that secret exists, run_scan.sh writes it under auths["ghcr.io"] (run_scan.sh#L44-L53).
I ran that snippet against a config that already had the other registry's login and got {'registry.acme.io': 'acme-user:ACME-PW', 'ghcr.io': 'github-actor:ACME-PW'}. The GITHUB_TOKEN login for ghcr is gone, so private ghcr images start failing with unauthorized, and the other registry's password gets sent to ghcr.io. seed --access can land in the same spot when it copies a login that reads secrets.REGISTRY_PASSWORD.
Easiest fix is probably giving the docs' secrets different names, or only using REGISTRY_PASSWORD here when vars.REGISTRY_HOST is set too.
There was a problem hiding this comment.
Addressed, thanks!
| def seed(args, token: str) -> dict: | ||
| groups = json.loads(Path(args.access).read_text()).get("groups", []) if args.access else None | ||
| info, created = ensure_repo(token, args.repo) | ||
| progress(f"{'Created' if created else 'Found'} repository {args.repo} ({'private' if info.get('private') else 'PUBLIC'})") |
There was a problem hiding this comment.
If the findings repo already exists and is public, this only prints PUBLIC in a progress line and keeps going. I mocked ensure_repo to return private: False and seed went all the way to the push. From then on every scan force-pushes latest/findings/*.jsonl to scan-results (run_scan.sh#L139-L144), so the CVE list for the org's private images is world-readable. Easy to get into if someone switches visibility on the prefilled create page, or reuses an old public repo.
I'd fail here with a message pointing at the visibility setting, unless the user explicitly opts in.
There was a problem hiding this comment.
Addressed, thanks!
| if aws is not None: | ||
| role = str(aws.get("role-to-assume") or "") | ||
| account = re.match(r"arn:aws[\w-]*:iam::(\d{12}):", role) | ||
| job_aws = {"region": str(aws.get("aws-region") or ""), "role": role, "account": account[1] if account else None} |
There was a problem hiding this comment.
aws-region is stored as written, so a pretty common setup like aws-region: ${{ env.AWS_REGION }} (with AWS_REGION in the workflow env) gives the host 123456789012.dkr.ecr.${{ env.AWS_REGION }}.amazonaws.com. That never matches the image, the group comes back no_path/none_found, and the ECR images move to suggested even though a usable pull login exists. Same result with ${{ vars.AWS_REGION }}, and with registry: ${{ vars.REGISTRY }} on docker/login-action (L483).
Running the region through the same env substitution _hosts_for already does would cover the env. case, and anything still holding ${{ could fall back to * like an empty region does today.
There was a problem hiding this comment.
Addressed, thanks!
| host, path = first, rest | ||
| else: | ||
| host, path = "registry-1.docker.io", ref | ||
| if "/" not in path.split("@")[0].split(":")[0]: |
There was a problem hiding this comment.
library/ is only added when the ref has no registry, so docker.io/redis:7 gets looked up at /v2/redis/manifests/7. Checked live: redis:7 and docker.io/library/redis:7 come back public, docker.io/redis:7 comes back private. The inventory uses a literal ref as found ("Use it as is" in inventory.md), so a manifest that says docker.io/redis:7 gets an official image reported as private and moved to suggested as "no pull access found".
Doing the library/ check after docker.io is normalised to registry-1.docker.io would fix it.
There was a problem hiding this comment.
Addressed, thanks!
| schedule: | ||
| - cron: '0 6 * * *' | ||
| push: | ||
| branches: [main] |
There was a problem hiding this comment.
This is an edge case but pointing it out:
seed pushes to the repo's default_branch (seed.py#L143, #L195), but this only triggers on main. For an org whose default branch for new repos is master or trunk, the setup push doesn't start a scan, so the "first scan is running now" link in Done shows nothing until the 06:00 cron. The blob/main links in scan.py#L338, #L428 and SKILL.md#L254 would 404 there too.
There was a problem hiding this comment.
Will address post-launch, captured in PLTF-4110
|
|
||
|
|
||
| def registry_check(args, token: str | None) -> dict: | ||
| refs = sorted(set(re.findall(r'^\s*-?\s*image:\s*["\']?([^"\'\s#]+)', Path(args.factory).read_text(), re.M))) |
There was a problem hiding this comment.
This regex takes every image: line in the file, including targets.suggested, which inventory.md has the agent write as image entries. With images: [ghcr.io/a/b:1] and suggested: [ghcr.io/a/c:latest], registry-check checked both and reported 2 private. That inflates the private count in the ACCESS LINE, adds fix_urls for packages that won't be scanned, and lets seed --access copy logins for them.
PyYAML is already available through load_yaml(), so reading targets.images directly would be simpler and exact.
There was a problem hiding this comment.
Addressed, thanks!
| continue | ||
| for term in terms: | ||
| q = urllib.parse.quote(f"{term} org:{org} path:.github/workflows") | ||
| status, data = api(token, "GET", f"/search/code?q={q}&per_page=50") |
There was a problem hiding this comment.
Code search allows 10 requests a minute (gh api rate_limit shows code_search.limit: 10), and this sends one request per term per org with no pause. ECR + GAR + Docker Hub is 6 terms, so two --org values means 12 back-to-back requests. In my run the 11th got a 403, the second org was recorded as unavailable: HTTP 403 with nothing saying why, its workflows were never searched, and every group came back none_found.
Waiting and retrying on 403/429 (the response has Retry-After / x-ratelimit-reset) would keep it under the limit. Saying "rate limited" in the output would also stop the agent from reading it as "no login exists".
There was a problem hiding this comment.
Addressed, thanks!
…ess and the findings repo
- Use REGISTRY_PASSWORD only when REGISTRY_HOST is set, so another registry's secret never replaces the ghcr.io login.
- Refuse to seed a public findings repository.
- Look up docker.io/<name> images under library/.
- registry-check reads only targets.images.
- Resolve aws-region written as ${{ env.X }}; treat other expressions as any region.
- Wait and retry once when GitHub code search is rate limited, and say so in the output.
|
🚀 Released in v0.43.0. |
…the CVE Remediation Factory Assisted templates now start an automation-setup conversation: the seed is sent as the first message, the conversation is tagged as automation setup, and the draft editor opens beside it. Maps the shield-check manifest icon and updates the card-order tests for the CVE Remediation Factory, now the first featured template. Temporarily builds against a tarball of OpenHands/extensions#948; replace it with the released version before merging.
HUMAN:
Adding an automation for the the provisioning of a CVE Factory which will help scan for security vulnerabilities in a users repositories.
Additionally help configure an automation for daily remediation PR's based on scanned findings
AGENT:
Ready for review: CI passes. Merge this first; OpenHands/OpenHands#19700 then takes the release that contains it.
Why
Teams need a standing way to find fixable CVEs in what they actually ship and get small upgrade PRs for them, without building the pipeline themselves. This template sets that up from a short form and a two-message conversation.
Summary
Catalog card
automations/catalog/cve-remediation-factory/: "GitHub CVE Remediation Factory", featured first among the suggested templates. Its assisted form asks for repositories, what you ship, a starting repo and notes, and an optional Slack channel. Adds a newshield-checkicon.Skill
skills/cve-remediation-factory/.SKILL.mdscripts the conversation:scripts/seed.pydoes the predictable work:fetch,find-images(images and lockfiles).registry-check, plusfind-access, which reuses pull-only CI logins for private registries.seed, which is additive and never writes into a non-factory repo.remediator-checkasks the automation API whether a remediator already exists before suggesting a new one;find-factoriespowers "did you mean" when a named factory doesn't exist.Findings-repository template:
factory.yamlfor configuration;remediator.md, which the weekday remediation automation follows;docs/registries.mdwith pull-access steps;automation-worklogbranch.Issue Number
Fixes #951
Tracked internally as Linear PLTF-4080.
How to Test
uv run --group test pytest tests/: 1585 passed, 33 skipped. The sync, plugin-resource and SDK-skill checks also pass, as does CodeQL.End to end: I ran the template many times on a local OpenHands Enterprise compose stack, with sandboxes loading this branch's skill:
remediator-checkfound the existing remediator through the stack's automation API.To try it without building anything, use the preview branch in OpenHands/enterprise#1368:
docker compose up, then Automations → Templates.Video/Screenshots
A 48-second recording from a running OpenHands Enterprise stack with this build: the GitHub CVE Remediation Factory card first under Suggested, the setup form and review, then the two-panel setup view. The first message arrives with the form's answers, the agent fills the automation's schedule (weekdays, 09:00 UTC), and progress lines show each step as it starts.
cve-remediation-factory-template.webm
The two agent messages, from the template's script (
SKILL.md), with example values:Plan
Done
Notes
mainonce it's merged.@openhands/extensionsrelease, followed by a Canvas bump (feat(automations): open assisted templates in the setup view and add the CVE Remediation Factory icon OpenHands#19700).