Subprocess Pipe Deadlock in `shells.py` on Non-UTF-8 Output

[Madhumasa84]

Summary

subprocess.Popen is instantiated with text=True but omits errors="replace". When a command outputs non-UTF-8 bytes, _reader encounters a UnicodeDecodeError, exits its reading loop, and blocks in wait(). The unconsumed stdout fills the 64KB OS pipe buffer, causing a permanent deadlock between the child process and the reader thread.

Root Cause

In openhack/shells.py:91-122:

proc = subprocess.Popen(command, shell=True, stdout=subprocess.PIPE, stderr=subprocess.STDOUT, text=True)
...
def _reader(self, sh: BackgroundShell) -> None:
    try:
        for line in sh.proc.stdout:
            sh.append(line.rstrip("\n"))
    except Exception:
        pass
    finally:
        sh.returncode = sh.proc.wait()  # DEADLOCK: child write() is blocked

Reproduction

manager.spawn("python3 -c 'import sys; sys.stdout.buffer.write(b\"\\xff\" * 100000)'")
# The reader crashes on decode, stops draining the pipe, and hangs in sh.proc.wait() forever.

Impact

Background shell tasks and security scanners (strings, fuzzers, compiled binary tests) freeze the CLI indefinitely.

Proposed Fix

Add errors="replace" to text-mode Popen calls:

proc = subprocess.Popen(..., text=True, errors="replace")