Skip to content

fix(validate): respect output flags for runtime validation - #1187

Merged
burtenshaw merged 6 commits into
huggingface:mainfrom
mugenkyou:main
Sep 17, 2026
Merged

burtenshaw merged 6 commits into
huggingface:mainfrom
mugenkyou:main

Conversation

@mugenkyou

@mugenkyou mugenkyou commented Sep 16, 2026

Copy link
Copy Markdown
Contributor

Summary

Fixes runtime openenv validate to respect --json and --output when validating a running server URL.

Type of Change

  • Bug fix
  • New feature
  • Breaking change
  • Documentation
  • New environment
  • Refactoring

Alignment Checklist

  • I have read .claude/docs/PRINCIPLES.md and this PR aligns with our principles
  • I have checked .claude/docs/INVARIANTS.md and no invariants are violated
  • I have run /pre-submit-pr (or bash .claude/hooks/lint.sh and tests) and addressed all issues

RFC Status

  • Not required (bug fix, docs, minor refactoring)
  • RFC exists: #___
  • RFC needed (will create before merge)

Changes

  • Route runtime validation reports through write_report().
  • Respect --json for runtime validation output.
  • Support human-readable output when --json is not provided.
  • Handle runtime --output write failures consistently with local validation.
  • Add regression tests for runtime output and error-handling cases.

Test Plan

  • PYTHONPATH=src:envs pytest tests/test_cli/test_validate.py -q — 15 passed
  • ruff check src/openenv/cli/commands/validate.py src/openenv/validation/report.py tests/test_cli/test_validate.py
  • ruff format --check src/openenv/cli/commands/validate.py src/openenv/validation/report.py tests/test_cli/test_validate.py
  • usort check src/openenv/cli/commands/validate.py src/openenv/validation/report.py tests/test_cli/test_validate.py
  • git diff --check

Claude Code Review

N/A

Closes #1186


Note

Low Risk
CLI-only output behavior for URL runtime validation; no changes to validation logic or local package paths beyond shared report rendering.

Overview
Runtime openenv validate (URL / --url) no longer always prints raw JSON. It now mirrors local validation: --json prints the report as JSON, --output writes that JSON to a file (still showing human-readable text on stdout when --json is off), and the default is a formatted text report.

_render_report accepts runtime report dicts and prints profile, standard version, per-criterion PASS/FAIL (with details, expected, and actual on failures). _write_runtime_report handles JSON serialization and optional file writes; write failures exit with code 3 like other internal errors.

Tests cover human-readable output, failed criteria formatting, --output / --output+--json, and output write failures; existing JSON runtime test now passes --json explicitly.

Reviewed by Cursor Bugbot for commit d34dce0. Bugbot is set up for automated code reviews on this repo. Configure here.

@cursor cursor Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Cursor Bugbot has reviewed your changes using default effort and found 1 potential issue.

Fix All in Cursor

❌ Bugbot Autofix is OFF. To automatically fix reported issues with cloud agents, enable autofix in the Cursor dashboard.

Reviewed by Cursor Bugbot for commit 5289f72. Configure here.

Comment thread src/openenv/cli/commands/validate.py
@mugenkyou

Copy link
Copy Markdown
Contributor Author

Hey @burtenshaw, could you take a look at this PR when you get a chance? Would appreciate a review. Thanks!

@cursor cursor Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The runtime CLI behavior is now covered and the focused suite passes locally (16/16); changed-file usort, Ruff format/check, and git diff --check are clean. One contract issue remains before merge, and exact-head repository workflows are still awaiting maintainer approval.

Open in Web View Automation 

Sent by Cursor Automation: Release

Comment thread src/openenv/validation/report.py Outdated
@cursor
cursor Bot requested a review from burtenshaw September 16, 2026 16:01
@cursor cursor Bot mentioned this pull request Sep 17, 2026
22 tasks

@cursor cursor Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Re-reviewed at head 98c9532e. My earlier change request is resolved — I am no longer blocking this.

The contract concern was that widening openenv.validation.write_report() to accept arbitrary legacy dictionaries would make the exported RFC 008 serializer's schema-versioned promise false. d07abd89 fixes that the right way: the diff now touches only src/openenv/cli/commands/validate.py and tests/test_cli/test_validate.py, the public serializer stays typed to ValidationReport, and the runtime-probe dictionary is serialized by the private _write_runtime_report helper inside the CLI. _render_report accepting a dict is additive and leaves the typed path unchanged.

Verified at this exact head on Python 3.12: tests/test_cli/test_validate.py 17 passed, tests/test_validation 103 passed, and usort / ruff format --check / ruff check clean on both changed files. Cursor Bugbot passes.

What still gates the merge is not code: repository CI (test on 3.11 and 3.12, lint, Package CI, docs) has never run on this PR because the workflows need a maintainer to approve the run for an external contributor. Once those are green this is merge-ready from my side.

This is not in the 0.5.0 candidate (#1190) since it has no repository CI yet, so it is not holding the release.

View PR

Open in Web View Automation 

Sent by Cursor Automation: Release

@bot-ci-comment

Copy link
Copy Markdown

The docs for this PR live here. All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.

@burtenshaw
burtenshaw merged commit 32b782a into huggingface:main Sep 17, 2026
10 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

CLI: openenv validate ignores --output and --json for runtime URL targets

2 participants