Summary
Agents treat -f / --format json / --json / --stdin as one Webcmd grammar. On v0.7.4 that grammar still breaks on several authoring commands, and --json is rejected even on commands that already accept --format json.
This is the same class of failure as #338 / #171 / #175, still reproducing after those landed.
What the eval actually did (v0.7.4, local task eval)
| Invocation |
Result |
webcmd hackernews top --json |
unknown option '--json' — valid flags include -f, --format |
webcmd rest-countries top --json |
same |
webcmd site fixture get quotes-toscrape/list -f json |
unknown option '-f' — valid flags: --output |
webcmd site memory show --kind endpoints -f json |
unknown option '-f' |
webcmd site fixture put --stdin |
unknown option '--stdin' |
Two of those cost a wasted turn on an otherwise 5.0 scenario (engineering-news-brief, heal-country-cli). The site-memory ones burned turns on create-quotes-cli.
--json is not a wild guess. It is the dominant CLI convention, and Webcmd already prints valid flags: --limit, -f, --format. The agent then has to invent the mapping --json → --format json.
Why this is still open
#338 added -f to several list commands (profile list now works). It did not:
- Accept
--json as an alias of --format json on commands that already have --format
- Wire
-f through site fixture get, site memory show, and the other site-memory reads
- Accept
--stdin on site fixture put, even though browser run --stdin is the pattern the same agent just used
An inconsistent flag does not read as "this command is different." It reads as "I got the invocation wrong," so the agent retries with a temp file or drops into the filesystem.
Fix
- Accept
--json as an alias of --format json on every command that already has --format. Unknown-flag errors that list --format should also list --json, or accept it.
- Add
-f, --format to site fixture get, site memory show, site memory list, site note list, site endpoint list — the enumerations an agent parses mid-authoring.
- Accept
--stdin / - on site fixture put (and any other put that currently requires a path), matching browser run --stdin.
Check
webcmd hackernews top --json --limit 1 and webcmd hackernews top --format json --limit 1 produce the same payload.
webcmd site fixture get <site/cmd> -f json exits 0 or a structured empty, never unknown option '-f'.
printf '{}' | webcmd site fixture put <site/cmd> --stdin is accepted (or the error names the valid input shape).
Summary
Agents treat
-f/--format json/--json/--stdinas one Webcmd grammar. On v0.7.4 that grammar still breaks on several authoring commands, and--jsonis rejected even on commands that already accept--format json.This is the same class of failure as #338 / #171 / #175, still reproducing after those landed.
What the eval actually did (v0.7.4, local task eval)
webcmd hackernews top --jsonunknown option '--json'— valid flags include-f, --formatwebcmd rest-countries top --jsonwebcmd site fixture get quotes-toscrape/list -f jsonunknown option '-f'— valid flags:--outputwebcmd site memory show --kind endpoints -f jsonunknown option '-f'webcmd site fixture put --stdinunknown option '--stdin'Two of those cost a wasted turn on an otherwise 5.0 scenario (
engineering-news-brief,heal-country-cli). The site-memory ones burned turns oncreate-quotes-cli.--jsonis not a wild guess. It is the dominant CLI convention, and Webcmd already printsvalid flags: --limit, -f, --format. The agent then has to invent the mapping--json→--format json.Why this is still open
#338 added
-fto several list commands (profile listnow works). It did not:--jsonas an alias of--format jsonon commands that already have--format-fthroughsite fixture get,site memory show, and the other site-memory reads--stdinonsite fixture put, even thoughbrowser run --stdinis the pattern the same agent just usedAn inconsistent flag does not read as "this command is different." It reads as "I got the invocation wrong," so the agent retries with a temp file or drops into the filesystem.
Fix
--jsonas an alias of--format jsonon every command that already has--format. Unknown-flag errors that list--formatshould also list--json, or accept it.-f, --formattosite fixture get,site memory show,site memory list,site note list,site endpoint list— the enumerations an agent parses mid-authoring.--stdin/-onsite fixture put(and any otherputthat currently requires a path), matchingbrowser run --stdin.Check
webcmd hackernews top --json --limit 1andwebcmd hackernews top --format json --limit 1produce the same payload.webcmd site fixture get <site/cmd> -f jsonexits 0 or a structured empty, neverunknown option '-f'.printf '{}' | webcmd site fixture put <site/cmd> --stdinis accepted (or the error names the valid input shape).