specify args multiple() times through the combinator on options

[jbolda]

Motivation

We want to support cli --config inputs1.yml --config inputs2.yml and be able to merge them.

Approach

• combinator sets multiple
• read takes that as an arg so all read logic is still contained there

[pkg-pr-new]

Open in StackBlitz

npm i https://pkg.pr.new/configliere@32

commit: ba05db5

[cowboyd]

robo assisted feedback:

multiple() belongs on Param: cardinality is part of the parameter definition and affects its model type. Passing multiple into ReadCLI, however, makes individual source readers responsible for parameter lifecycle. In particular, this is unsafe:

through: param.multiple ? undefined : horizon?.index

Consider a child introduced after a dynamic phase:

app --config root.yml auth0 --config child.yml

Before resume(), auth0 is an unresolved horizon. The root phase may claim --config root.yml, but it must leave the suffix hidden until route discovery can determine whether auth0 is a child selector. Giving the repeated reader an unbounded view lets the root claim both configuration options; after /auth0 is discovered, its input is already gone.

The division of responsibility should be:

• multiple() remains declarative cardinality metadata on the parameter.
• param.cli.read(view) remains singular and receives the ordinary bounded view.
• bindPhase() accumulates successful captures for a repeated parameter and keeps it pending.
• After each committed claim, the scheduler recomputes the horizon and arbitrates again.
• At the CLI fixed point, the accumulated values are decoded and validated as one array. If nothing was captured, normal Env -> Values -> undefined fallback continues.

That preserves dynamic route boundaries, earliest-claim arbitration, and automatic support for custom readers without requiring every reader to understand a multiple boolean.

Two other concerns surfaced:

1. decodeMany() eagerly constructs the Cartesian product of decoder candidates. Because scalar() can offer both number and string interpretations, n numeric-looking occurrences can create 2^n candidate arrays before validation. Candidate traversal needs to be lazy/short-circuiting, or collection decoding needs a different abstraction.
2. MultipleParam<K, T> currently forces the schema result back into T[]. That loses valid schema outputs such as string[] | undefined, nullable arrays, or a schema that transforms the captured array into another model type. Cardinality should describe captured input while the Standard Schema output remains authoritative.

Regression coverage was pushed in 1b7d8f5. It currently exposes three failures:

• a repeated parent option crosses a dynamically discovered child selector;
• a custom singular reader cannot participate in multiple();
• an optional array schema loses undefined from its inferred model.

The added malformed-occurrence, source-precedence, default-schema, and numeric-looking-string cases already pass.

[cowboyd]

I had a pretty extensive conversation with the robot about how to fix the combinatoric explosion potential with multiple decoders (lazily iterating candidates with a generator helps, but it is still susceptible to the cases where you iterate all the way to the end of the huge list)

I think I ended up with a way to both make the decoding system simpler, and also eliminate it as a possibility so that the complexity of decoding is linear with the number of options to decode. The TLDR is that by changing the primitive to be less lossy, we can make the combinators on top of it more powerful, while still conforming to the thin Decoder api.

Most of the implementation you have here stays, it's that how decoding is represented internally to be more combinatorially robust.

A transcript of the entire conversation is here for context: https://gist.github.com/cowboyd/eb8ce14699d03a1f933f62b928f1d4ce

Here is the recommendation we arrived at:

Multiple decoding

Status

Proposed design for repeated CLI parameter representations.

Motivation

A parameter marked with multiple() may capture several CLI representations:

--config one.yml --config two.yml

Those representations must be collected without looking through unresolved route
selectors. Once collection is complete, they must be decoded consistently and
validated as one model value.

The decoder currently removes failed alternatives and returns only successful
candidates:

decode("abc"); // ["abc"]
decode("123"); // [123, "123"]

This loses the correspondence between a candidate and the decoder alternative
that produced it. Decoding repeated representations independently therefore
requires a Cartesian product:

[1, 2, 3][1, 2, "003"][1, "002", 3] // ...
  ["001", "002", "003"];

With d alternatives and k representations, that produces up to d^k
candidates. It also permits one parameter occurrence to use a different
interpretation from another.

Decision

Decoder candidate positions are stable. A decoder represents a failed
alternative with Nothing instead of removing its slot:

decode("abc"); // [Nothing, Just("abc")]
decode("123"); // [Just(123), Just("123")]

Candidate slot i always corresponds to decoder alternative i.

multiple() lifts a scalar decoder into a decoder over a collection of
representations:

Decoder<A, B>
    ↓ multiple
Decoder<A[], B[]>

Each decoder alternative is applied consistently to the complete collection.
Given:

decode("001"); // [Just(1), Just("001")]
decode("002"); // [Just(2), Just("002")]

the lifted decoder returns:

multiple(decode)(["001", "002"]);
// [
//   Just([1, 2]),
//   Just(["001", "002"]),
// ]

If an alternative cannot decode every representation, its lifted slot is
Nothing:

multiple(decode)(["abc", "123"]);
// [
//   Nothing,
//   Just(["abc", "123"]),
// ]

The number of aggregate candidates is therefore bounded by the number of decoder
alternatives, not exponentiated by the number of representations.

Decoder representation

The decoder output becomes a fixed-width array of Maybe candidates:

type Decoder<T = unknown> = (
  value: string,
) => readonly Maybe<T>[];

Every invocation of a decoder must return the same number of slots in the same
semantic order.

The built-in decoders become:

const number: Decoder<number> = (value) => {
  if (!numeric.test(value)) {
    return [Nothing()];
  }

  let decoded = Number(value);

  return Number.isFinite(decoded) ? [Just(decoded)] : [Nothing()];
};

const text: Decoder<string> = (value) => [Just(value)];

const scalar: Decoder = (value) => [
  ...number(value),
  ...text(value),
];

const boolean: Decoder<boolean> = (value) => [
  value === "true" ? Just(true) : value === "false" ? Just(false) : Nothing(),
];

No public decoder identifier or brand is required. Positional correspondence is
the only identity retained.

Decoder lifting

The multiple decoder transposes scalar candidate slots and sequences each slot:

function multiple<T>(
  decode: Decoder<T>,
): (values: readonly string[]) => readonly Maybe<readonly T[]>[] {
  return (values) => {
    let rows = values.map(decode);
    let width = rows[0]?.length ?? 0;

    return Array.from({ length: width }, (_, index) => {
      let column = rows.map((row) => row[index]);

      if (column.every(present)) {
        return Just(column.map((candidate) => candidate.value));
      }

      return Nothing();
    });
  };
}

The implementation must treat unequal row widths as a violated decoder invariant
rather than silently aligning unrelated alternatives.

An empty representation collection is absence, not a candidate array. It does
not reach the multiple decoder. Source precedence proceeds to Env, Values, and
finally schema validation with undefined.

Binding lifecycle

Capture is incremental; interpretation happens during finalization.

route search with the currently known graph
                    ↓
capture CLI representations inside the safe segment
                    ↓
CLI binding reaches a fixed point
                    ↓
decode complete representation sets
                    ↓
validate decoded candidates
                    ↓
try lower-priority sources only for absent CLI parameters

A multiple parameter cannot be decoded, validated, or settled until its current
binding scope is closed and every safely visible representation has been
captured.

"Closed" refers to the current phase's CLI binding scope, not necessarily the
final application route. Consider:

--config one --config two auth0 --port 9000
                          ^
                       horizon

The current phase may capture one and two. It must stop at auth0, finalize
the config parameter, and expose an increment if required. It must not search
past auth0 for more --config occurrences because a later phase may introduce
auth0 as a child route.

Phase binding

bindPhase() should keep capture and interpretation separate for CLI sources.
It maintains capture state such as:

interface Capture {
  readonly values: unknown[];
  readonly issues: Issue[];
  readonly exists: boolean;
  readonly failed: boolean;
}

During CLI arbitration:

• Readers propose one physical occurrence at a time.
• The earliest claim wins.
• Committing a claim advances the horizon.
• No decoder or schema is invoked.
• A singular parameter stops offering after its first existing read.
• A multiple parameter continues offering until the fixed point.
• Successful and failed read diagnostics accumulate.
• Any existing CLI capture, including a failed capture, blocks lower-priority
  sources.

After the CLI fixed point:

for (let param of params) {
  let capture = captures.get(param.name);

  if (!capture?.exists) {
    // Try Env, Values, and finally undefined.
  } else if (capture.failed) {
    // Return all accumulated capture issues.
  } else if (param.multiple) {
    // Decode the complete representation collection.
  } else {
    // Decode the single representation.
  }
}

If one repeated occurrence fails after earlier successful occurrences, all
capture diagnostics are preserved. Whether collection continues after a failed
occurrence must be explicit. Continuing until the fixed point gives the most
complete diagnostics and fulfills the rule that all safely visible
representations are collected.

Candidate validation

Singular and multiple decoders both return fixed-width Maybe candidates.
Validation skips Nothing without compacting or reordering slots:

function decode<T>(
  param: Param<string, T>,
  representation: unknown,
  candidates: Iterable<Maybe<unknown>>,
  path: string[],
): Result<T> {
  let issues: readonly Issue[] | undefined;
  let found = false;

  for (let candidate of candidates) {
    if (!candidate.exists) {
      continue;
    }

    found = true;

    let result = validate(param, candidate.value, path);
    if (result.ok) {
      return result;
    }

    issues = issues ?? result.issues;
  }

  return found
    ? { ok: false, issues: issues ?? [] }
    : unableToDecode(representation, path);
}

The first candidate accepted by the schema wins, preserving existing decoder
precedence.

Source semantics

multiple() changes CLI capture and decoding cardinality. It does not change
source precedence:

CLI fixed point
Env
Values
undefined

One or more CLI occurrences make the CLI source present. Invalid CLI capture,
decoding, or validation blocks Env and Values.

Values remain already-interpreted JavaScript values and go directly to schema
validation. Environment representations retain their source-specific capture and
decoding semantics. If environment repetition is desired later, it requires an
explicit Env representation design rather than inheriting CLI behavior
implicitly.

For non-text CLI representations, the policy must be explicit. The simplest
identity rule is:

// Singular already-interpreted representation
[Just(value)] // Multiple already-interpreted representations
  [Just(values)];

Alternatively, multiple() may be restricted to valued text options. The
implementation must choose and test one policy.

Model typing

The schema output remains authoritative. multiple() changes capture and
decoding cardinality; it does not mechanically turn the model type into an
array.

option(
  name("config"),
  multiple(),
  schema(type("string[]")),
);
// model.config: string[]

A transforming schema may accept an array candidate and return another model
type:

option(
  name("config"),
  multiple(),
  schema(arrayToSet),
);
// model.config: Set<string>

Implementation changes

lib/decode.ts

• Change Decoder to return fixed-width Maybe candidates.
• Preserve failed built-in decoder slots with Nothing.
• Add the lifted multiple decoder.
• Enforce equal candidate widths across a representation collection.

lib/bind.ts

• Retain the one-occurrence reader loop and horizon arbitration.
• Collect CLI representations and diagnostics until the fixed point.
• Defer decoding and validation until capture finalization.
• Replace the Cartesian candidates() generator with the lifted multiple
  decoder.
• Preserve earlier capture diagnostics when a later occurrence fails.
• Ensure an existing failed CLI source blocks Env and Values.

lib/param.ts

• Keep multiple as parameter cardinality metadata or store the lifted decoder
  directly. If kept as metadata, lifting occurs during binding finalization.
• Preserve schema output as the parameter's model type.
• Ensure decoder combinators compose consistently whether declared before or
  after multiple().

docs/binding.md

Add these invariants:

• Decoder candidate positions are stable. Failed interpretations remain as
  Nothing slots.
• Repeated capture lifts each decoder alternative across the complete
  representation collection.
• Repeated decoding never forms mixed Cartesian combinations.
• CLI representations are captured until the current binding scope reaches a
  fixed point; decoding and validation happen during finalization.

Tests

Decoder slots

expect(scalar("abc")).toEqual([
  Nothing(),
  Just("abc"),
]);

expect(scalar("123")).toEqual([
  Just(123),
  Just("123"),
]);

Multiple lifting

expect(multiple(scalar)(["001", "002"])).toEqual([
  Just([1, 2]),
  Just(["001", "002"]),
]);

expect(multiple(scalar)(["abc", "123"])).toEqual([
  Nothing(),
  Just(["abc", "123"]),
]);

Observable parsing

• Repeated numeric-looking strings remain strings under a string[] schema.
• Repeated numerals become numbers under a number[] schema.
• A custom singular reader participates in repeated collection.
• An incomplete occurrence after successful occurrences reports its capture
  issue and blocks lower-priority sources.
• Successful-read diagnostics survive a later failed occurrence.
• Repeated parent options do not capture occurrences across a child-route
  selector.
• Dynamic phases finalize only the representations safely visible before their
  unresolved horizon.
• Optional and defaulting schemas receive undefined when every source is
  absent.

The decoder tests should directly assert that candidate count is bounded by
decoder width. That expresses the safety contract without testing resolver call
counts or other parser implementation mechanics.

Summary

The design rests on two rules:

1. Candidate slots are stable across decoder invocations.
2. Capture is incremental, while interpretation happens after the current
   binding scope closes.

Together they make multiple() an ordinary decoder transformation, eliminate
Cartesian candidate growth, enforce consistent interpretation across repeated
representations, preserve route boundaries, and keep schemas authoritative over
the final model value.

Gist

Conversation with GPT about configliere multiple()

Conversation with GPT about configliere multiple() - configliere-conversation.md