Skip to content

feat(driver): stream external runtime exec and logs - #1388

Merged
skevetter merged 8 commits into
mainfrom
codex/external-runtime-streams
Oct 6, 2026
Merged

skevetter merged 8 commits into
mainfrom
codex/external-runtime-streams

Conversation

@skevetter

@skevetter skevetter commented Oct 6, 2026 •

Copy link
Copy Markdown
Contributor

The external runtime host can now execute commands and retrieve logs through Runtime Protocol v1. Exec preserves literal argv, streams bounded binary stdin/stdout/stderr frames concurrently, and requires a terminal exit followed by clean stream completion. Nonzero and signal exits return a typed command error; transport and backend failures retain the existing sanitized runtime error handling.

Cancellation closes interruptible stdin, joins the input pump, reaps the supervised plugin session, and preserves caller deadline errors after cleanup. Read-error recording and owned input closure are serialized: failures observed before host-owned Close survive cancellation and command completion; errors from unblocking a pending Read during owned Close preserve a successful early exit regardless of their type. Readers without Close retain their failures independently of stream cancellation. Text command output uses chunk-safe redaction, including overlapping or repeated secrets at stream end. The host retains per-workspace RunImage environment values across later operations, including prior and failed-request values, using cached immutable operation snapshots and workspace-specific rebuild locks. An immutable byte trie avoids comparing every output byte with every retained secret. RawStdout and argv execution preserve stdout bytes for agent injection. Logs capability is checked before launch and the protocol's merged binary log stream goes to stdout. Documentation spells out stream ownership and the limits of arbitrary blocking readers/writers.

This completes the planned C3 host streaming stage. Workspace factory registration and image-backend integration remain the next C4 stage.

Validation:

  • Real-plugin race tests cover large duplex binary traffic, exact argv and shell/user mapping, split-secret and workspace-environment redaction, exit status, malformed streams, input/output failures (including an input error arriving at the cleanup boundary), cancellation, child cleanup, and logs.
  • Focused local secrets and subprocess race tests cover every split of repeated/overlapping secrets, bounded streaming progress, workspace isolation, retained values, and concurrent redaction snapshots.
  • Info timeout and Exec deadline/crash race regressions passed 20 repetitions. Error normalization runs after all owned cleanup, so unary and streaming calls preserve caller deadlines consistently. A retained-value benchmark covers 1, 100, and 1,000 values; the 1,000-value case improved from roughly 210 ms to 2 ms per 32 KiB chunk locally.
  • Go vet and Linux/Windows test compilation passed.
  • Documentation production build (Webpack, 112 pages) and link checks passed.
  • CLI CI lint and applicable pre-commit checks passed.
  • Local CodeRabbit reviews covered the implementation and follow-ups. The concurrent Read/Close ordering suggestion was evaluated against the documented observation boundary and deterministic ownership tests; no valid actionable finding remains.
  • The cleanup-boundary input-error regressions passed 20 race-enabled repetitions.
  • The owned-closure regressions cover os.ErrClosed, unexpected EOF, custom reader errors, real os.File input, and both input-failure cancellation orderings; race-enabled boundary and early-exit tests passed 20 repetitions.
  • macOS unit CI allows 45 minutes for race compilation and the existing crypto suite after two 30-minute runner cancellations; Linux remains at 30 minutes and Go retains its default 10-minute timeout per test binary. Workflow lint, applicable prek, CLI lint, and local CodeRabbit review passed.
  • Final-head CI passed all 72 jobs on c0d692de2, including Linux/macOS race suites, all platform builds, lint, pre-commit, and integration suites. One Podman job needed a targeted retry after its daemon stopped responding during cleanup; that retry passed. Applicable external checks passed.
  • Greptile reviewed all 19 files on the final head with confidence 4/5 and zero new inline comments. Its summary concern about late stdin failures is dispositioned against the explicit owned-closure boundary and regression evidence in the PR discussion.
  • CodeRabbit completed a full review of all 19 files at the final head with no actionable comments and no architecture-level findings. Its docstring-coverage warning is a nonblocking metric suggestion; exported contracts and non-obvious ownership rules are documented, without adding comments that merely restate private helpers or tests.
  • All actionable review threads are resolved; all eight commits have valid GitHub signatures.

@netlify

netlify Bot commented Oct 6, 2026 •

Copy link
Copy Markdown

✅ Deploy Preview for devsydev canceled.

Name Link
🔨 Latest commit c0d692d
🔍 Latest deploy log https://app.netlify.com/projects/devsydev/deploys/6ac4cf1c58c65e00071ea5ec

@coderabbitai

coderabbitai Bot commented Oct 6, 2026 •

Copy link
Copy Markdown

Review in Change Stack →

No actionable comments were generated in the recent review. 🎉

ℹ️ Recent review info
⚙️ Run configuration
  • Configuration used: Organization UI
  • Review profile: CHILL
  • Plan: Advanced
  • Run ID: c65f171b-ace5-4d29-8c45-682fa6f08dfd
📥 Commits

Reviewing files that changed from the base of the PR and between 48de1a3 and c0d692d.

📒 Files selected for processing (19)
  • .github/workflows/pr-ci.yml
  • pkg/driver/external/exec.go
  • pkg/driver/external/exec_exchange_test.go
  • pkg/driver/external/exec_input.go
  • pkg/driver/external/exec_test.go
  • pkg/driver/external/host.go
  • pkg/driver/external/internal/testfixture/main.go
  • pkg/driver/external/internal/testfixture/stream.go
  • pkg/driver/external/lifecycle.go
  • pkg/driver/external/logs.go
  • pkg/driver/external/logs_test.go
  • pkg/driver/external/redaction.go
  • pkg/driver/external/redaction_test.go
  • pkg/driver/external/session.go
  • pkg/driver/external/stream.go
  • pkg/secrets/redact.go
  • pkg/secrets/redact_boundary_test.go
  • pkg/secrets/redact_matcher.go
  • sites/docs-devsy-sh/content/docs/developing-providers/driver.mdx

Included review availability: This review used your included allowance. Your plan provides up to 2 included reviews per hour; 1 remain after this review.


📝 Walkthrough

Walkthrough

The external host adds command execution and log streaming through runtime RPCs. It retains environment values per workspace for output redaction and adjusts streaming redaction boundaries to avoid splitting complete secret matches.

Changes

External Runtime Streaming

Layer / File(s) Summary
Workspace redaction and lifecycle integration
pkg/driver/external/host.go, pkg/driver/external/redaction.go, pkg/driver/external/lifecycle.go, pkg/driver/external/redaction_test.go, pkg/secrets/redact.go, pkg/secrets/redact_matcher.go, pkg/secrets/redact_boundary_test.go
The host retains nonempty environment values per workspace and uses workspace redactors in lifecycle operations. Streaming redaction adjusts chunk boundaries to avoid splitting complete matches. Tests cover workspace isolation, retained values, snapshots, and chunk boundaries.
Exec command streaming
pkg/driver/external/exec.go, pkg/driver/external/exec_input.go, pkg/driver/external/stream.go, pkg/driver/external/session.go, pkg/driver/external/internal/testfixture/*, pkg/driver/external/exec*test.go, .github/workflows/pr-ci.yml, sites/docs-devsy-sh/content/docs/developing-providers/driver.mdx
The host adds shell and literal-argv execution through the Exec RPC. It streams stdin and stdout/stderr, validates output and terminal frames, closes closable input, and returns command exit errors separately from RPC or output errors. Tests cover streaming, validation, cancellation, exit outcomes, and I/O failures. The macOS unit-test timeout is increased to 45 minutes.
Runtime Logs streaming
pkg/driver/external/logs.go, pkg/driver/external/internal/testfixture/stream.go, pkg/driver/external/logs_test.go, sites/docs-devsy-sh/content/docs/developing-providers/driver.mdx
The host streams Logs chunks to stdout and returns stream or writer errors. Tests cover binary output, validation, protocol errors, and cancellation. Provider documentation describes Exec and Logs support.

Priority: ➖ Normal

Estimated code review effort: 4 (Complex) | ~45 minutes

Change: Feature

Sequence Diagram(s)

sequenceDiagram
  participant Caller
  participant Host
  participant RuntimeExec
  participant InputReader
  participant OutputWriters
  Caller->>Host: Run shell or literal-argv command
  Host->>RuntimeExec: Start Exec request
  Host->>InputReader: Read stdin
  InputReader-->>RuntimeExec: Send stdin chunks and close stdin
  RuntimeExec-->>Host: Send output frames and terminal exit
  Host->>OutputWriters: Write stdout and stderr
  Host-->>Caller: Return stream result or command exit error
Loading

Merge Risk: ⚪ Minimal · up to c0d69

No actionable merge-blocking issue was identified; the change is ready to merge after normal checks.

Security Architecture Review

Security architecture risk: 🔵 Low · up to c0d69

Command and log streaming adds sensitive execution and output paths. Text masking, bounded frames, and cancellation cleanup constrain the change. Binary output intentionally remains unmasked, and secret-retention ownership during future lifecycle integration still needs definition.

Retained concerns
No architecture-level concerns identified.

Security review details

Security Blast Radius

  • inferred — The new command surface reaches the caller-selected workspace, including root execution through the literal-argv capability. The runtime executable is already trusted with inherited host environment values; workspace-specific masking is not a sandbox or a tenant-authorization boundary.

Trust Boundaries and Controls

  • observed — Inspected generic text-command callers do not request RawStdout. Binary agent delivery and protocol forwarding explicitly request byte-preserving output, while stderr remains redacted. Logs similarly assigns responsibility for its raw binary output to its caller.
  • observed — Caller-visible runtime errors and backend debug messages pass through the operation's redactor. Plugin diagnostic output also uses streaming redaction, extending retained workspace values to later diagnostic paths within the same Host.

Resilience and Maintainability Implications

  • observed — Output frames are restricted to 1–32768 bytes, short writes fail the operation, and frames after terminal exit are rejected. Session cleanup retains process-kill and reaping hooks, with error normalization deferred until owned cleanup completes.

Hardening Proposals

  • proposed — Before production factory integration, define the redaction owner's lifecycle, recovery and expiry policy. Preserve or securely reconstruct required values before replacement Hosts emit text diagnostics, rather than assuming retention survives Host recreation.
🚥 Pre-merge checks | ✅ 4 | ❌ 1

❌ Failed checks (1 warning)

Check name Status Explanation Resolution
Docstring Coverage ⚠️ Warning Docstring coverage is 4.76% which is insufficient. The required threshold is 80.00%. Docstring coverage is scoped to functions touched by this diff. Analyzed 21 functions across 17 files. (2 skipped: … Write docstrings for the functions missing them to satisfy the coverage threshold.
✅ Passed checks (4 passed)
Check name Status Explanation
Linked Issues check ✅ Passed Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check ✅ Passed Check skipped because no linked issues were found for this pull request.
Description Check ✅ Passed Check skipped - CodeRabbit’s high-level summary is enabled.
Title check ✅ Passed The title clearly summarizes the main changes: streaming external-runtime command execution and log retrieval.
Full details: Docstring Coverage

Explanation

Docstring coverage is 4.76% which is insufficient. The required threshold is 80.00%. Docstring coverage is scoped to functions touched by this diff. Analyzed 21 functions across 17 files. (2 skipped: 2 unsupported.)

  • Fix all pre-merge checks with AI
✨ Finishing Touches 💡 1
🛠️ Fix failing CI checks 💡
  • Commit to this branch
  • Create a new PR
✨ Simplify code
  • Commit to this branch
  • Create a new PR
  • Autopilot · Keep fixing CodeRabbit findings and required CI, and resolving merge conflicts

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands.

@netlify

netlify Bot commented Oct 6, 2026 •

Copy link
Copy Markdown

✅ Deploy Preview for images-devsy-sh canceled.

Name Link
🔨 Latest commit c0d692d
🔍 Latest deploy log https://app.netlify.com/projects/images-devsy-sh/deploys/6ac4cf1c977ae200089e079f

@skevetter

Copy link
Copy Markdown
Contributor Author

@greptileai review

@greptile-apps

greptile-apps Bot commented Oct 6, 2026 •

Copy link
Copy Markdown

RetriggerConfidence Score: 4/5

[Medium risk] Adds streaming execution and log retrieval to the external runtime driver.

The PR is not yet safe to merge because a late stdin failure can still look like a successful command.

What we checked:

  • Successful calls stay successful: operationError returns nil immediately for a successful call, even if the caller’s deadline expires during cleanup.

Summary

The external runtime host now streams commands and logs through Runtime Protocol v1, with binary input and output and clear command exit results. It also keeps workspace environment values available for later redaction, so secrets remain masked across operations.

  • External runtimes now stream commands and report their exit status.
  • External runtime logs arrive as one binary stream on stdout.
  • Workspace environment values stay masked in later runtime output.

Reviews (6) · Last reviewed commit: "ci: allow macOS race tests to finish wit..."

Comment thread pkg/driver/external/exec.go
Comment thread pkg/driver/external/exec.go
@skevetter

Copy link
Copy Markdown
Contributor Author

@greptileai review

Comment thread pkg/secrets/redact.go
Comment thread pkg/driver/external/redaction.go Outdated
@skevetter

Copy link
Copy Markdown
Contributor Author

@greptileai review

@skevetter

Copy link
Copy Markdown
Contributor Author

@coderabbitai full review

@coderabbitai

coderabbitai Bot commented Oct 6, 2026 •

Copy link
Copy Markdown
✅ Action performed

Full review finished.

@skevetter
skevetter force-pushed the codex/external-runtime-streams branch from f95745d to 096a1d7 Compare October 6, 2026 06:46
@skevetter

Copy link
Copy Markdown
Contributor Author

@greptileai review

Comment thread pkg/driver/external/exec.go
@skevetter

Copy link
Copy Markdown
Contributor Author

@greptileai review

Comment thread pkg/driver/external/exec.go Outdated
@skevetter

Copy link
Copy Markdown
Contributor Author

@greptileai review

@skevetter

Copy link
Copy Markdown
Contributor Author

Final-head review disposition for c0d692de29148e193182e1969ce6d33a8ef79b58: Greptile reviewed 19 files with confidence 4/5 and added zero inline comments. I checked the summary concern that a late stdin failure can look successful against the final execInput implementation and its tests.

Read-error recording and host-owned Close are serialized by the same mutex. A non-EOF input failure recorded before owned closure survives successful terminal completion even when completion wins stream cancellation. Readers without Close never acquire the owned-closure marker, so their late failures remain errors. For a closable reader whose pending Read is interrupted by host-owned Close after a valid terminal exit and clean EOF, the cleanup error intentionally preserves that successful exit, regardless of the error type.

Concurrent Read and Close use this explicit observation boundary. An arbitrary io.Reader provides no separate provenance signal that would distinguish its underlying return immediately before Close from a return caused by Close. Holding the mutex across a blocking Read would prevent Close from unblocking it. Error-name allowlists cannot establish that ordering either.

The deterministic tests cover a read failure before buffered success, both cancellation orderings without owned Close, and six error types released by owned Close. A real os.File early-exit test verifies the successful result and input closure. These boundary and early-exit regressions passed 20 race-enabled repetitions locally, and the final Linux/macOS race suites passed in CI. No additional actionable defect was supplied by this review; the summary concern is dispositioned against the documented ownership contract.

@skevetter

Copy link
Copy Markdown
Contributor Author

@coderabbitai full review

@coderabbitai

coderabbitai Bot commented Oct 6, 2026 •

Copy link
Copy Markdown
✅ Action performed

Full review finished.

@skevetter
skevetter marked this pull request as ready for review October 6, 2026 15:04
@mergify

mergify Bot commented Oct 6, 2026 •

Copy link
Copy Markdown

This pull request does not currently match the merge queue conditions, so it cannot be queued from here. The box comes back if it matches again.

@skevetter
skevetter merged commit 6e9defe into main Oct 6, 2026
164 of 166 checks passed
@skevetter
skevetter deleted the codex/external-runtime-streams branch October 6, 2026 21:44
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant