Skip to content

feat(automations): add the GitHub CVE Remediation Factory template and skill - #948

Merged
dylan-openhands merged 2 commits into
mainfrom
dj/cve-factory
Nov 16, 2026
Merged

dylan-openhands merged 2 commits into
mainfrom
dj/cve-factory

Conversation

@dylan-openhands

@dylan-openhands dylan-openhands commented Nov 12, 2026 •

Copy link
Copy Markdown
Contributor

HUMAN:
Adding an automation for the the provisioning of a CVE Factory which will help scan for security vulnerabilities in a users repositories.

Additionally help configure an automation for daily remediation PR's based on scanned findings


AGENT:

Ready for review: CI passes. Merge this first; OpenHands/OpenHands#19700 then takes the release that contains it.

Why

Teams need a standing way to find fixable CVEs in what they actually ship and get small upgrade PRs for them, without building the pipeline themselves. This template sets that up from a short form and a two-message conversation.

Summary

  • Catalog card automations/catalog/cve-remediation-factory/: "GitHub CVE Remediation Factory", featured first among the suggested templates. Its assisted form asks for repositories, what you ship, a starting repo and notes, and an optional Slack channel. Adds a new shield-check icon.

  • Skill skills/cve-remediation-factory/. SKILL.md scripts the conversation:

    • a numbered first message;
    • a progress line as each step starts;
    • a Plan with a go-ahead;
    • a Done message that leads with the one action and lists links in click order.

    scripts/seed.py does the predictable work:

    • Inventory: fetch, find-images (images and lockfiles).
    • Access: registry-check, plus find-access, which reuses pull-only CI logins for private registries.
    • Setup: seed, which is additive and never writes into a non-factory repo.
    • Existing factories: remediator-check asks the automation API whether a remediator already exists before suggesting a new one; find-factories powers "did you mean" when a named factory doesn't exist.
  • Findings-repository template:

    • a daily Trivy + Grype scan of images and source repositories;
    • a results README showing each tool's own counts, with a per-tool trend chart;
    • factory.yaml for configuration;
    • remediator.md, which the weekday remediation automation follows;
    • docs/registries.md with pull-access steps;
    • a README work log on the automation-worklog branch.

Issue Number

Fixes #951

Tracked internally as Linear PLTF-4080.

How to Test

uv run --group test pytest tests/: 1585 passed, 33 skipped. The sync, plugin-resource and SDK-skill checks also pass, as does CodeQL.

End to end: I ran the template many times on a local OpenHands Enterprise compose stack, with sandboxes loading this branch's skill:

  • Fresh setups: each created a private findings repo, started the first scan, and filled in the remediator form.
  • Existing factory: a re-run that named one updated it instead of creating a new one.
  • Duplicate check: remediator-check found the existing remediator through the stack's automation API.

To try it without building anything, use the preview branch in OpenHands/enterprise#1368: docker compose up, then Automations → Templates.

Video/Screenshots

A 48-second recording from a running OpenHands Enterprise stack with this build: the GitHub CVE Remediation Factory card first under Suggested, the setup form and review, then the two-panel setup view. The first message arrives with the form's answers, the agent fills the automation's schedule (weekdays, 09:00 UTC), and progress lines show each step as it starts.

cve-remediation-factory-template.webm

The two agent messages, from the template's script (SKILL.md), with example values:

Plan

Alright, I am now ready to set up your CVE Remediation Factory

  • I will be acting as @JDoe: commits and pull requests show as yours.
  • I will create 1 new private repository, called acme/cve-remediation-factory, with workflows that scan for vulnerabilities daily at 06:00 UTC.
  • I'll scan 18 images and 1 repository from acme/deploy, acme/api, acme/web and 4 more. 3 more are saved for later.
  • 3 private images in GitHub Packages (acme) need an access change before I can scan them; I'll give you the exact steps when I'm done.
  • The automation form is on the right: when we are done, I will ask you to create this automation. It will interpret the scan results every weekday, then open pull requests addressing CVEs for you to review, at most 3 open at a time.
  • Posts a daily status to #security-alerts.

Green light? Reply go ahead, or tell me what to change.

Done

Last step: press Create Automation on the right. Once the first scan finishes, press Run now on it to see it open pull requests.

Overview and links to review in order

Later: come back to this conversation or run /cve-remediation-factory again and point me at acme/cve-remediation-factory to troubleshoot issues (agents can make mistakes), change what's scanned, or adjust the remediator's prompt.

Notes

@github-actions github-actions Bot added the type: feat A new feature label Nov 12, 2026
dylan-openhands added a commit to OpenHands/OpenHands that referenced this pull request Nov 13, 2026
… Factory branch

Points @openhands/extensions at a tarball of OpenHands/extensions#948 so the
preview image carries the new template. Replace with the released version
before merging.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
dylan-openhands added a commit to OpenHands/enterprise that referenced this pull request Nov 13, 2026
Points canvas-server at the agent-canvas PR image (OpenHands/OpenHands#19700)
and sandboxes at the extensions branch (OpenHands/extensions#948) through
EXTENSIONS_REF. Becomes the agent-canvas version bump once both are released.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@dylan-openhands
dylan-openhands marked this pull request as ready for review November 13, 2026 22:11
dylan-openhands added a commit to OpenHands/enterprise that referenced this pull request Nov 13, 2026
Points canvas-server at the agent-canvas PR image (OpenHands/OpenHands#19700)
and sandboxes at the extensions branch (OpenHands/extensions#948) through
EXTENSIONS_REF. Do not merge: at release, replace the image tag with the
agent-canvas release and remove EXTENSIONS_REF.
…d skill

Adds a featured "GitHub CVE Remediation Factory" catalog card with an
assisted setup form, a shield-check icon, and the cve-remediation-factory
skill. The skill scripts a short conversation (plan, go-ahead, done), builds
an inventory of shipped images and source repositories, reuses pull-only CI
logins for private registries, detects existing factories and remediators,
and seeds a private findings repository with a daily Trivy + Grype scan, a
per-tool results report and a weekday remediation automation.
dylan-openhands added a commit to OpenHands/OpenHands that referenced this pull request Nov 13, 2026
…the CVE Remediation Factory

Assisted templates now start an automation-setup conversation: the seed is
sent as the first message, the conversation is tagged as automation setup,
and the draft editor opens beside it. Maps the shield-check manifest icon and
updates the card-order tests for the CVE Remediation Factory, now the first
featured template. Temporarily builds against a tarball of
OpenHands/extensions#948; replace it with the released version before merging.
dylan-openhands added a commit to OpenHands/enterprise that referenced this pull request Nov 13, 2026
Do not merge yet. Until the releases exist, canvas-server uses the agent-canvas
image built for OpenHands/OpenHands#19700 and sandboxes load skills from the
OpenHands/extensions#948 branch through EXTENSIONS_REF. At release, set the
agent-canvas tag to the release that includes #19700 and remove EXTENSIONS_REF.

@all-hands-bot all-hands-bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This review was posted by an AI agent (OpenHands).

Scope: this repository owns the automations catalog and the skills registry, so an automation template plus its skill belongs here; no product or architecture decision is needed.

Verification on head b82d5a9 (re-fetched today): all 13 GitHub Actions checks pass on this exact SHA (CodeQL, test, check, sync-extensions, sync-sdk-skill, PR description check on the latest run). Locally I ran 414 tests across test_automation_setup, test_catalogs, test_catalog_schema, test_skills_catalog, test_skills_have_readme, test_interface_manifest, test_plugin_manifest and test_check_pr_description - all pass. The manifest validates against automations/catalog.schema.json (including the new shield-check icon enum in both schemas), and npm run build:skills / build:automations reproduce every generated file byte-identically; sync_extensions.py --check is clean beyond pre-existing non-blocking coverage warnings.

Scripts exercised, not just read: template/scripts/scan.py plan/collect/history end to end on synthetic Trivy/Grype reports - GHSA-to-CVE mapping via match-level relatedVulnerabilities, fixed_version = higher of the two scanners, severity normalization, per-tool counts, the results README and trend SVG all render correctly, including failed targets and single-day history. seed.py registers all 11 documented subcommands, parse_ref handles Docker Hub shorthand/library/, digests, ECR, ACR and localhost correctly, and install_logins injects a reused pull-only login with continue-on-error, strips id:/if: keys, adds id-token: write only for OIDC logins, and produces a workflow that still parses. fill_factory placeholder substitution verified.

Security posture is sound: the token travels only in Authorization headers (never in URLs, arguments, or output; whoami returns credential names, not values); the remediator prompt defaults to propose, forbids merging, force-pushes, ignore files, VEX entries and scanner-config tampering; find-access only reuses logins from jobs whose steps do not publish or deploy; the scan workflow pins actions/checkout by SHA and uses least-privilege permissions with a concurrency guard.

One note on the existing CodeQL alert at seed.py:1016 ("clear-text logging of sensitive information"): false positive. Line 1016 is main's final print(json.dumps(result, ...)); what flows to stdout there is credential names (token_env, TOKEN_ENVS entries, reused-login secret names from find-access), never values, matching the module docstring's "Never prints a token" contract.

One finding needs a decision before merge - the defaultEnabled: true flag on this skill's marketplace entry contradicts AGENTS.md, which says the default-enabled set stays small and provider-agnostic, and that language-, vendor- or workflow-specific skills "should start off and be opted into from the catalog UI". This skill is GitHub-only and is a specific workflow (CVE remediation), yet it now ships in DEFAULT_ENABLED_SKILL_NAMES for every new workspace. Drop the flag, or record a maintainer exception if being default-on is intentional for a featured security template.

🔄 CHANGES REQUESTED

"source": "./skills/cve-remediation-factory",
"description": "Set up a private CVE remediation factory on GitHub: inventory and scan the container images you ship, then open small upgrade pull requests for fixable CRITICAL and HIGH vulnerabilities.",
"category": "automations",
"defaultEnabled": true,

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

AGENTS.md sets the policy for this flag: the default-enabled set should stay small and provider-agnostic, and anything language-, vendor- or workflow-specific "should start off and be opted into from the catalog UI". This skill is GitHub-only (SKILL.md: "GitHub only") and is a specific workflow (CVE remediation), so it meets both exclusions, yet defaultEnabled: true ships it in DEFAULT_ENABLED_SKILL_NAMES for every new workspace. Please drop the flag so users opt in from the catalog UI, or note the maintainer-approved exception if default-on is intentional for a featured security template.

RUN_URL: ${{ github.server_url }}/${{ github.repository }}/actions/runs/${{ github.run_id }}
REGISTRY_HOST: ${{ vars.REGISTRY_HOST || 'ghcr.io' }}
REGISTRY_USER: ${{ vars.REGISTRY_USER || github.actor }}
REGISTRY_PASSWORD: ${{ secrets.REGISTRY_PASSWORD || secrets.GITHUB_TOKEN }}

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This fallback collides with the docs. registries.md, "Other registries" tells people to save REGISTRY_PASSWORD for their own registry and add a login step, but here REGISTRY_HOST still defaults to ghcr.io. So once that secret exists, run_scan.sh writes it under auths["ghcr.io"] (run_scan.sh#L44-L53).

I ran that snippet against a config that already had the other registry's login and got {'registry.acme.io': 'acme-user:ACME-PW', 'ghcr.io': 'github-actor:ACME-PW'}. The GITHUB_TOKEN login for ghcr is gone, so private ghcr images start failing with unauthorized, and the other registry's password gets sent to ghcr.io. seed --access can land in the same spot when it copies a login that reads secrets.REGISTRY_PASSWORD.

Easiest fix is probably giving the docs' secrets different names, or only using REGISTRY_PASSWORD here when vars.REGISTRY_HOST is set too.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Addressed, thanks!

def seed(args, token: str) -> dict:
groups = json.loads(Path(args.access).read_text()).get("groups", []) if args.access else None
info, created = ensure_repo(token, args.repo)
progress(f"{'Created' if created else 'Found'} repository {args.repo} ({'private' if info.get('private') else 'PUBLIC'})")

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If the findings repo already exists and is public, this only prints PUBLIC in a progress line and keeps going. I mocked ensure_repo to return private: False and seed went all the way to the push. From then on every scan force-pushes latest/findings/*.jsonl to scan-results (run_scan.sh#L139-L144), so the CVE list for the org's private images is world-readable. Easy to get into if someone switches visibility on the prefilled create page, or reuses an old public repo.

I'd fail here with a message pointing at the visibility setting, unless the user explicitly opts in.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Addressed, thanks!

if aws is not None:
role = str(aws.get("role-to-assume") or "")
account = re.match(r"arn:aws[\w-]*:iam::(\d{12}):", role)
job_aws = {"region": str(aws.get("aws-region") or ""), "role": role, "account": account[1] if account else None}

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

aws-region is stored as written, so a pretty common setup like aws-region: ${{ env.AWS_REGION }} (with AWS_REGION in the workflow env) gives the host 123456789012.dkr.ecr.${{ env.AWS_REGION }}.amazonaws.com. That never matches the image, the group comes back no_path/none_found, and the ECR images move to suggested even though a usable pull login exists. Same result with ${{ vars.AWS_REGION }}, and with registry: ${{ vars.REGISTRY }} on docker/login-action (L483).

Running the region through the same env substitution _hosts_for already does would cover the env. case, and anything still holding ${{ could fall back to * like an empty region does today.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Addressed, thanks!

host, path = first, rest
else:
host, path = "registry-1.docker.io", ref
if "/" not in path.split("@")[0].split(":")[0]:

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

library/ is only added when the ref has no registry, so docker.io/redis:7 gets looked up at /v2/redis/manifests/7. Checked live: redis:7 and docker.io/library/redis:7 come back public, docker.io/redis:7 comes back private. The inventory uses a literal ref as found ("Use it as is" in inventory.md), so a manifest that says docker.io/redis:7 gets an official image reported as private and moved to suggested as "no pull access found".

Doing the library/ check after docker.io is normalised to registry-1.docker.io would fix it.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Addressed, thanks!

schedule:
- cron: '0 6 * * *'
push:
branches: [main]

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is an edge case but pointing it out:

seed pushes to the repo's default_branch (seed.py#L143, #L195), but this only triggers on main. For an org whose default branch for new repos is master or trunk, the setup push doesn't start a scan, so the "first scan is running now" link in Done shows nothing until the 06:00 cron. The blob/main links in scan.py#L338, #L428 and SKILL.md#L254 would 404 there too.

@dylan-openhands dylan-openhands Nov 16, 2026 •

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Will address post-launch, captured in PLTF-4110



def registry_check(args, token: str | None) -> dict:
refs = sorted(set(re.findall(r'^\s*-?\s*image:\s*["\']?([^"\'\s#]+)', Path(args.factory).read_text(), re.M)))

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This regex takes every image: line in the file, including targets.suggested, which inventory.md has the agent write as image entries. With images: [ghcr.io/a/b:1] and suggested: [ghcr.io/a/c:latest], registry-check checked both and reported 2 private. That inflates the private count in the ACCESS LINE, adds fix_urls for packages that won't be scanned, and lets seed --access copy logins for them.

PyYAML is already available through load_yaml(), so reading targets.images directly would be simpler and exact.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Addressed, thanks!

continue
for term in terms:
q = urllib.parse.quote(f"{term} org:{org} path:.github/workflows")
status, data = api(token, "GET", f"/search/code?q={q}&per_page=50")

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code search allows 10 requests a minute (gh api rate_limit shows code_search.limit: 10), and this sends one request per term per org with no pause. ECR + GAR + Docker Hub is 6 terms, so two --org values means 12 back-to-back requests. In my run the 11th got a 403, the second org was recorded as unavailable: HTTP 403 with nothing saying why, its workflows were never searched, and every group came back none_found.

Waiting and retrying on 403/429 (the response has Retry-After / x-ratelimit-reset) would keep it under the limit. Saying "rate limited" in the output would also stop the agent from reading it as "no login exists".

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Addressed, thanks!

…ess and the findings repo

- Use REGISTRY_PASSWORD only when REGISTRY_HOST is set, so another registry's secret never replaces the ghcr.io login.
- Refuse to seed a public findings repository.
- Look up docker.io/<name> images under library/.
- registry-check reads only targets.images.
- Resolve aws-region written as ${{ env.X }}; treat other expressions as any region.
- Wait and retry once when GitHub code search is rate limited, and say so in the output.
@dylan-openhands
dylan-openhands merged commit 294f6be into main Nov 16, 2026
13 checks passed
@dylan-openhands
dylan-openhands deleted the dj/cve-factory branch November 16, 2026 18:44
@openhands-release-bot openhands-release-bot Bot added the released: v0.43.0 Shipped in v0.43.0 label Nov 16, 2026
@openhands-release-bot

Copy link
Copy Markdown
Contributor

🚀 Released in v0.43.0.

dylan-openhands added a commit to OpenHands/OpenHands that referenced this pull request Nov 16, 2026
…the CVE Remediation Factory

Assisted templates now start an automation-setup conversation: the seed is
sent as the first message, the conversation is tagged as automation setup,
and the draft editor opens beside it. Maps the shield-check manifest icon and
updates the card-order tests for the CVE Remediation Factory, now the first
featured template. Temporarily builds against a tarball of
OpenHands/extensions#948; replace it with the released version before merging.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

released: v0.43.0 Shipped in v0.43.0 type: feat A new feature

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Add a GitHub CVE Remediation Factory automation template

5 participants