ax ssh <task> -- <cmd> does not forward local stdin to the remote process (silent failure, exit 0 on both ends)

[karimad]

Summary

Piping data into ax ssh <task> -- <cmd> via a subprocess's stdin does not reach the remote command. The remote process exits 0, the local ax ssh invocation exits 0, and no error is produced anywhere — the data is just silently dropped.

Repro

import subprocess
p = subprocess.Popen(
    ["ax", "ssh", "my-task", "--", "sh", "-c", "cat > /workspace/out.json"],
    stdin=subprocess.PIPE,
)
p.communicate(input=b'{"hello": "world"}')
print(p.returncode)  # 0

$ ax ssh my-task -- cat /workspace/out.json
$ ax ssh my-task -- wc -c /workspace/out.json
0 /workspace/out.json

out.json exists and is 0 bytes, despite the write command returning success on both the local and remote sides.

Impact

Any workflow that assumes ax ssh's local stdin reaches the remote process (the natural expectation given the ssh-shaped interface) will silently produce empty files or truncated input, with no error signal to catch it — this is what makes it worth flagging over a normal bug: there's genuinely nothing to check for on the caller's side.

Workaround

Base64-encode the payload into the remote command's own argv instead of stdin:

subprocess.run([
    "ax", "ssh", "my-task", "--", "sh", "-c",
    f"echo {base64_payload} | base64 -d > /workspace/out.json",
])

This reliably delivers the data.

Environment

Hit against a live kind cluster running ax + Agent Substrate, ax built from a recent main checkout this session.