Repository navigation
feat: simplify harness authoring and preview native output - #215
Conversation
Support single-file harness profiles with optional context and IR 0.9 across readers. Add check, preview and build commands with native prompt provenance, strict diagnostics and guarded failed-output cleanup. Update authoring contracts, tutorials, fixtures and regression coverage.
Separate prompt guidance from enforceable target and host requirements. Defer new workflow DSL and runner state, retain existing runtime contracts, and document evidence required to reopen scope. Align roadmap, composition and direct-session support descriptions.
goldmedal
left a comment
There was a problem hiding this comment.
Review
Verdict: approve with two small fixes requested. CI is green on head 15ebf9d (rust ×2, ts ×2, bird-eval ×2, doc, docs-site, publish-check, fixture-contract, workflows). I rebuilt the CLI at this head, ran the two new author-flow test files (16 passed), and probed the new paths by hand; notes below.
P2 — fix before merge
-
compileparse errors lost their file-path prefix.project_needs_hub(cli/src/lib.rs) runs before compilation inrun_compileandcompile_project_to_irand returns the bare serde string. Repro:components: 3now printserror: profile components must be a list; a YAML syntax error printserror: found unexpected end of stream at line 6 column 1…with no file named. Before this PR the same path producedfailed to parse <profile.yml>: ….check/previeware unaffected because they validate first. Suggest wrapping the error inproject_needs_hub(or at its two call sites) with the profile path. -
Context-free agent files carry five blank lines between the frontmatter and
## respond. Indispatcher/claude-code-cli/src/emit/agent.rsthe emptypreambleand emptyprompt_section()are still pushed together with their separators. Cosmetic, butpreviewadvertises exact bytes and this is the first output the new tutorial shows. Suggest skipping empty parts when joining.
P3 — nits / follow-ups
- vercel rejects context-free IR only via the raw serde error (
invalid type: null, expected struct ContextBinding at line 5 column 29), andcontext_free_unsupported_shapes_and_targets_refuse_before_outputpins that string. Every other target emits a purposeful "context-free IR is not supported by …" message. An explicit check indispatcher/vercel/src/ir.rsplus a test update would make the targets consistent. - Shorthand without
descriptionsynthesizesanalytical skill that renders no render blocks (outcome: none).as the subagent description. Claude Code uses that field for delegation, and it is misleading for a text helper. Consider requiringdescriptionin the shorthand, or deriving it from the prompt's first line. - Shorthand
tieris free text.tier: mediumcompiles and only fails at dispatch.parse_componentcould validate againststrong|cheap|orchestratorand give a source-located error. - The context-free check treats every
params[].sourceas a context requirement, butauthoring.mddefinessource: runtime-injectedas runtime-supplied, not context. The resulting message ("requires context for parameter … source 'runtime-injected'") would be wrong. Masked today because the emitter also rejects such params, so this is wording only. - Check-order nit: a context-free profile mounting
generate_dashboardreports the missinganswer_querycomposition mount before the context requirement. Accurate, just not the most useful first error.
Compatibility notes (confirm intent; not defects)
ProfileComponentMountandProfileContextare nowdeny_unknown_fields. External profiles with stray mount or context keys that were silently ignored now fail compile (verified withfoo: baron a mount).authoring.mddocuments this; the PR body only mentionsconfig. Worth one line in the PR body since the squash commit feeds the changelog.- Downstream readers that hard-code
warble_ir_version: "0.8"or pin@warble/ir-spec 0.8.x(e.g. the WrenAI genbi app) will need a coordinated bump when they move to this release. Nothing breaks until they do. packages/ir-specis excluded from release-please, so the manual 0.9.0 bump matches how 0.7 → 0.8 was done.
Verification performed
cargo build --bin warble;cargo test -p warble-cli --test author_commands --test authoring_tutorial→ 16 passed.- Probes with the built binary: compile
first-harness(null root and component bindings, IR 0.9); plaindispatchto claude-code:headless, codex:interactive and vercel;manifest;check/preview/buildonfirst-harness,code-reviewand the bounddemo-agent; a mixed bound + inline-shorthand profile;buildrefusals on an existing file, an existing directory and a nested new path; empty profile, duplicate ids,use+prompt,{{project}}without context, hub mount without context, unknown mount/context field, non-null mountconfig. - Not exercised:
produce-sessionon context-free IR, the interactive target's launch pins, live model runs.
Harness authors can now start with one profile containing an inline prompt, without creating a placeholder semantic context or managing an intermediate IR file.
warble check,preview, andbuildcarry that profile through validation, inspection, and native-file output for plain Claude Code headless and interactive targets. The native CLI is started separately by the user.Changes
Compatibility
The IR moves from 0.8 to 0.9 to represent an absent context binding. Compiler, readers, package declarations, and fixtures are updated together. Regenerate stored IR and use matching readers; 0.8 artifacts are not accepted by 0.9 readers. The new author commands support only plain Claude Code file targets; other targets and explicit host contracts continue through
compile/dispatch. Prompt instructions are not substitutes for tool permissions or execution isolation.Profile component mounts and profile context declarations now reject unknown fields that were previously ignored. Remove stray fields; unsupported non-null mount
configis also rejected. Custom tier names remain supported through dispatch-time model mappings.Validation
Local verification covers Rust lint/test/build/release and fresh rustdoc (877 passing, 2 existing ignored), SDK (375 passing), Codex (259 passing), BIRD (661 passing, 8 expected external-fixture skips), documentation generation/drift/release checks and production build, publish checks, fixture/workflow checks, and the release plan. Author-flow tests compare preview with emitted files and cover missing context, diagnostics, output collisions, and cleanup failures. A regression demonstrates that pi rejects context-free IR before writing output; all 11 pi dispatch tests pass. GitHub checks on
59750a5completed with 12 passing checks and 4 expected release-only skips; no failures or pending checks.Two bounded real-model authoring exercises were completed separately from the deterministic gates. They support the usability observations, not a general review-accuracy or runtime-safety certification. No release or merge is part of this PR.