Repository navigation
S3: bind data inputs with -i NAME=[FORMAT:]LOCATION, stdin included #7
Description
Part of #17. Design: docs/design.md §3.2.
Scope
-i/--input NAME=[FORMAT:]LOCATION(repeatable):- Formats:
csv,tsv,json,ndjson. - Without a format, the extension decides; stdin (
-) must name one. - An
http(s)URL needs--remote.
- Formats:
- Only one input may be stdin, and then the script must come from
-eor a file. Anything else is a usage error. - Schema: inferred, with
--infer-rows=N, or given by--schema NAME='{ … }'. - Shadowing: a bound input shadows a catalog relation of the same name, so a catalog view can run against a sample file.
- Unbounded stdin: the input can be declared unbounded so
tail -f … | relixstreams, and a blocking operator over it is refused at plan time.
Dependencies
- Blocked on Bind a relation to an input stream (CSV/TSV/JSON/NDJSON) with the connectors' parsers and schema inference relix-core#80 (stream-backed inputs with the connectors' parsers and schema inference), then an engine pin bump.
- S1 (S1: command skeleton — script sources, exit codes, diagnostics, profiles, engine options #4).
- Shadowing of catalog names needs S4 (S4: catalog discovery — .relix/ directories up the hierarchy, nearest-wins overlay #8).
Acceptance
- Each format, from a file and from stdin, end-to-end through
Main. - Shadowing of a catalog relation.
- The usage errors above.
- A streaming NDJSON stdin emits a row before end-of-input.
Activity
Bind data inputs with -i NAME=[FORMAT:]LOCATION, standard input included
david-Darkcollective commentedon Oct 8, 2026
Shadowing a catalog relation landed with #35, which added the catalog (#8). An input bound with
-iis declared after the catalog's survivors, so it replaces a catalog relation of the same name, and a catalog view reads the bound input.CatalogDiscoveryTestcovers this ("an input bound with -i replaces the catalog's relation of that name").What keeps this open is the engine refusing a blocking operator over an unbounded input at plan time: DarkCollective/relix-core#98. Once that lands and the engine pin picks it up, the
@Disabledtest from #34 can be enabled and this issue closed.david-Darkcollective commentedon Oct 8, 2026
Done. The last open piece was the engine refusing a blocking operator over an unbounded input (DarkCollective/relix-core#98, released in 1.0.0-rc8). #37 re-enabled
InputTest.Unbounded.blockingRefused, and all ofInputTestpasses ondevelopwith nothing skipped. Formats, stdin, usage errors and streaming came with #34; shadowing a catalog relation came with #35.Bind data inputs with -i NAME=[FORMAT:]LOCATION, standard input included