Skip to content

fix(skills): drive verify-yaml-deployment without host Python or Node - #14196

Merged
cv merged 2 commits into
v1from
fix/14182-verify-host-tools
Nov 5, 2026
Merged

cv merged 2 commits into
v1from
fix/14182-verify-host-tools

Conversation

@kaofelix

@kaofelix kaofelix commented Nov 4, 2026 •

Copy link
Copy Markdown
Collaborator

Failure

verify-yaml-deployment ran Python and Node on the host, which AGENTS.md rules out (#14182):

  • features/deepagents-code-terminal.md wrote and ran a 47-line terminal.py to drive dcode.
  • agents/deepagents-code.md, features/exposure-change.md, and targets/dgx-spark.md edited the YAML with python3 and PyYAML, which also dropped the example's comments.
  • features/openclaw-dashboard.md installed an unpinned playwright from npm on the host and ran it with Node.

Decision

  • Terminal: shell functions around util-linux script replace terminal.py. They type into a FIFO and read the screen from script's output file, using the same mechanism as pseudo_terminal.rs in the interfaces_live test. They keep the steps, the pass lines (ui ok, reply ok, exit 0), and the evidence files. sed and grep run under LC_ALL=C: in en_US.UTF-8, the range [@-~] misses letters such as K, which leaves control sequences in the text.
    After review, the driver checks for Interrupted by user after Ctrl-D, when the UI reports it. It stops typing once the session has ended, so a launcher that exits at once can't kill the shell with SIGPIPE. It checks once more after a wait ends. It also strips DCS, APC, and colon-form SGR sequences, and turns carriage returns into line breaks.
  • YAML edits: hand edits replace the three PyYAML snippets. SKILL.md already allowed them, and they are what a configuration author does. Each edit shows the target YAML and a grep check, and the launch saves yaml-edits.diff against the example. A Rust helper would duplicate authoring for three small edits.
  • Browser: DB5 keeps Playwright, as a narrow exception recorded in the feature's Gotchas. Its Node and Chromium now run only inside mcr.microsoft.com/playwright v1.64.1, pinned by digest, on the host network so the origin stays loopback. The token goes in the container's environment, not its arguments. Driving the WebSocket instead would drop the only proof of browser sign-in and the loopback-origin check (feat: share OpenClaw configuration across native interfaces NeMo-Fabric#573). A Rust CDP or WebDriver driver would add a dependency and a host Chromium for one skill.
    Whether create-verification-skill/SKILL.md's Helpers rule should name this exception is left to review; this PR changes only the verify skill.
  • The live run found one piece of map drift, which is fixed here: plan and apply now say "…conversation history, except on mounted volumes."

verify-fabric-boundary/features/native-interfaces.md reuses both drivers. Its references stay accurate, but this PR doesn't rerun that skill.

Validation

The changed procedures ran live on the DGX Spark target at b5489e5768 plus this change, on 2026-11-04. Every command came from the edited files, extracted from their code blocks. Each agent got a full pass with its own UID, gateway port and subnet, against an existing vLLM endpoint serving nemotron-3.5-super-vl:

  • deepagents-code: DT1–DT4 passed through the new terminal driver. dcode answered the prompt on its own line, with openai:nemotron-3.5-super-vl in the status line and exit status 0. After the replacement, the old launcher reported No such file or directory and the new one passed again.
  • openclaw: DB1–DB5 passed. DB5 printed reply from Chromium inside the pinned image, and its screenshot shows the signed-in Control UI with the prompt and the agent's reply. exposure-change ran its STRIP branch.
  • hermes: HI1–HI4 passed. exposure-change ran its probe branch.
  • All three:
    • The hand-edit checks printed their documented counts.
    • Plan and apply succeeded on the edited YAML.
    • deploy-and-answer returned READY_OK, and CHANGED_OK under the replacement runtime.
    • export-reapply planned no changes.
    • The credential scans found nothing.
    • Destroy and the UID-scoped removal left no resources.
  • No host Python or Node: stub python3, python, node, npm, and npx commands that log any call were first on PATH for every run, and none was called.
  • Before the live run: the terminal functions were tested against a stand-in TUI on the same host, including a nonzero exit status.
  • Review fixes: against a stand-in TUI, the driver taken from the previous commit reported reply ok for an interrupted answer. Its shell also died with status 141 when the launcher exited at once, or exited right after answering. The fixed driver reported reply FAIL, ui FAIL with exit 1, and reply ok, and left no escape bytes in any transcript. A second live deepagents-code pass then ran, using the fixed files, with a new UID, port, and subnet. It covered DA1, DT1–DT4, export-reapply, and exposure-change; both terminal sessions printed ui ok, reply ok, exit 0. Their transcripts hold the word on its own line and openai:nemotron-3.5-super-vl, with no Interrupted by user. That pass reused the first pass's hand-edited YAML, changing only its identity, gateway port, subnet, and image. Destroy and removal again left no resources, and no host Python or Node ran.

The documentation check (fern check) covers docs/, not skills, so the local checks were a reread of the diff and git diff --check. No code changed, so I didn't run cargo ci.

Drive dcode's terminal UI with shell functions around util-linux script,
replace the PyYAML snippets with hand edits and checks, and run the
dashboard's Playwright driver inside its pinned image. Proven live on
DGX Spark for deepagents-code, openclaw, and hermes.
@kaofelix

kaofelix commented Nov 4, 2026

Copy link
Copy Markdown
Collaborator Author
Authoring session — opening
Field Status Value Evidence or limit
Harness recorded Claude Code 2.1.318 Session record metadata, one version throughout
Skills recorded maintain-verification-skill (complete read); verify-yaml-deployment (complete read); nemoclaw-contributor-create-pr (injected at the cutoff) Reads and injections in the session record; create-verification-skill was read only in part and isn't counted
Instruction files recorded AGENTS.md (injected); user global instruction files (injected); WRITING.md (complete read); PULL_REQUEST_TEMPLATE.md (complete read); SESSION_REPORT_TEMPLATE.md (complete read) Startup injections and reads in the session record
Tool calls partial Bash: 119; Edit: 18; Read: 9; Write: 2; ToolSearch: 2; TaskStop: 1 Wrapper calls to the cutoff, including failures. 59 Bash calls ran hc exec on the DGX Spark host, and the live steps ran as scripts in a remote shell; their nested commands aren't counted
Session window recorded 2026-11-04T16:15Z to 2026-11-04T17:39Z; 84 min First record event to the PR skill's invocation, including idle time and waits on live runs
Compactions recorded 0 Complete record to the cutoff
Models recorded claude-sonnet-5-5; claude-opus-5-5 Assistant turns, in order: 2 Sonnet turns before a switch to Opus
Reasoning levels recorded medium Effort recorded on every turn; no changes

@github-actions

github-actions Bot commented Nov 4, 2026

Copy link
Copy Markdown
Contributor

@wscurran wscurran added area: e2e End-to-end tests, nightly failures, or validation infrastructure area: security Security controls, permissions, secrets, or hardening area: skills Skills, agent behaviors, prompts, or skill packaging chore Build, CI, dependency, or tooling maintenance integration: dcode LangChain Deep Code integration behavior integration: openclaw OpenClaw integration behavior platform: container Affects Docker, containerd, Podman, or images v1 NemoClaw v1 branch labels Nov 4, 2026
Check for an interrupted answer after Ctrl-D, stop typing into an ended
session, check once more after a wait ends, and strip more control
sequences. Add the probe exposure to an existing map.
@cv
cv enabled auto-merge (squash) November 4, 2026 23:46
@cv
cv merged commit 2fed945 into v1 Nov 5, 2026
53 of 55 checks passed
@cv
cv deleted the fix/14182-verify-host-tools branch November 5, 2026 00:02
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

area: e2e End-to-end tests, nightly failures, or validation infrastructure area: security Security controls, permissions, secrets, or hardening area: skills Skills, agent behaviors, prompts, or skill packaging chore Build, CI, dependency, or tooling maintenance integration: dcode LangChain Deep Code integration behavior integration: openclaw OpenClaw integration behavior platform: container Affects Docker, containerd, Podman, or images v1 NemoClaw v1 branch

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants