ingest: piped stdin is silently discarded (always on Windows, on any slow producer elsewhere)

[abhay-codes07]

CLI Version

hydradb-cli 0.2.0

Python Version

Python 3.10.11 (Windows) / Python 3.12.3 (Ubuntu)

Operating System

Windows 11 (fully broken) / Ubuntu 24.04 (intermittent)

Bug Description

Piping text into hydradb ingest silently discards the input instead of ingesting it.

The README documents the pipe form:

echo "piped note" | hydradb ingest

On Windows this never works, and on Linux/macOS it stops working as soon as the command upstream of the pipe takes longer than 100ms to produce output. In both cases the text is dropped and the CLI reports No text provided, so the note is lost rather than stored.

The cause is in read_stdin_safe() (src/hydradb_cli/utils/common.py):

if sys.stdin.isatty():
    return None

import select

try:
    ready, _, _ = select.select([sys.stdin], [], [], 0.1)
    if ready:
        data = sys.stdin.read().strip()
        return data if data else None
except (OSError, ValueError):
    pass
return None

Two independent failures:

1. Windows — always fails. select on Windows accepts socket handles only. Passing sys.stdin raises OSError (WinError 10038 / 10093) immediately. The bare except (OSError, ValueError): pass converts that into "no input", so every piped ingest fails on the platform.

   >>> import select, sys; select.select([sys.stdin], [], [], 0.1)
   OSError: [WinError 10093] Either the application has not called WSAStartup, or WSAStartup failed
   

2. Linux/macOS — races the producer. The 100ms timeout is a readiness poll, not a read. If the upstream process has not written within that window, select reports nothing ready and the input is discarded. Anything that has to do work first loses its payload:

   curl -s https://example.com/notes.txt | hydradb ingest    # drops the note
   pdftotext report.pdf - | hydradb ingest --kind knowledge  # drops the text

Both paths fail silently — no warning that content arrived after the poll and was thrown away.

This affects hydradb ingest, hydradb ingest --text -, and the deprecated memories add / knowledge upload-text aliases, since all of them resolve text through _resolve_text_input() → read_stdin_safe().

For an "agent-friendly command line interface", the pipe is the primary programmatic entry point, so this makes the documented scripting workflow unusable on Windows and unreliable elsewhere.

Steps to Reproduce

Windows (always reproduces):

1. hydradb login --api-key <key> --database <db>
2. echo "piped note" | hydradb ingest
3. See ✗ Error: No text provided. — the note is never sent.

Linux/macOS (reproduces with any slow producer):

1. hydradb login --api-key <key> --database <db>
2. (sleep 0.5; echo "slow note") | hydradb ingest
3. See ✗ Error: No text provided. — while echo "fast note" | hydradb ingest on the same machine succeeds.

The platform-independent core can be reproduced without credentials by calling the helper directly:

(sleep 0.5; echo "slow note") | python -c \
  "from hydradb_cli.utils.common import read_stdin_safe; print(repr(read_stdin_safe()))"
# -> None      (expected: 'slow note')

CLI Output

# Windows 11, hydradb-cli 0.2.0
$ echo "piped note from stdin" | hydradb ingest
  ✗ Error: No text provided. Use --text 'your text', pipe via stdin, or use --text - for interactive input.

$ echo "piped note" | hydradb ingest --text -
  ✗ Error: No input received from stdin.

# Ubuntu 24.04, hydradb-cli 0.2.0
$ echo "fast note" | hydradb ingest
  ✓ Memory added (1 success, 0 failed)

$ (sleep 0.5; echo "slow note") | hydradb ingest
  ✗ Error: No text provided. Use --text 'your text', pipe via stdin, or use --text - for interactive input.

Additional Context

Expected behaviour: a non-tty stdin is always a redirect — a pipe, a file, or a closed handle — so it is guaranteed to reach EOF and can simply be read to completion. The readiness poll guards against a hang that cannot occur, at the cost of discarding real input.

There is currently no test coverage for the stdin path, which is why this went unnoticed.

Happy to send a PR for this — I have a fix and regression tests ready (verified on Python 3.10/Windows and 3.12/Ubuntu). Will link it here.

[abhay-codes07]

Opened #22 with the fix and regression tests.