diff --git a/CHANGELOG.md b/CHANGELOG.md new file mode 100644 index 0000000..cbcb72f --- /dev/null +++ b/CHANGELOG.md @@ -0,0 +1,8 @@ +# Changelog + +## v0.0.50 + +### Compatibility + +- A direct array element whose literal value is `$super` or `$super*` is now interpreted as an array-composition marker. + Write `raw:$super` or `raw:$super*` when either literal string must remain data. diff --git a/openspec/changes/array-composition-on-inheritance/design.md b/openspec/changes/array-composition-on-inheritance/design.md index 81fef26..1017b05 100644 --- a/openspec/changes/array-composition-on-inheritance/design.md +++ b/openspec/changes/array-composition-on-inheritance/design.md @@ -198,7 +198,7 @@ Both `"z"` and `"d"` are atoms, so no cross-kind check fires and nothing is repo **A literal `$super` array element changes meaning.** Documents that use that exact string as data are affected. -→ Mitigation: `raw:$super`, a minor version bump, and a changelog entry. Recorded in the proposal rather than described as a change with no compatibility effect. +→ Mitigation: `raw:$super` and a changelog entry backfilled under `v0.0.50`, the release that introduced marker recognition. Recorded in the proposal rather than described as a change with no compatibility effect. **There is no escape hatch for a composition the markers cannot express.** Evaluation cannot reach the inherited value. Parking it under a second key that nobody overrides does make it reachable by `ref`, but that key then appears in the output: `$local` is removed before processing, but it holds node definitions for `$extends` and `$includes` by name, not values an expression can address, and there is no other way to hide a key. @@ -225,7 +225,7 @@ The last three are their own pains and belong in their own issues rather than in ## Migration Plan -An unmarked array composes exactly as it does today, so no configuration changes meaning through composition. The one exception is the literal `$super` array element described above, which is why this ships as a minor version bump with a changelog entry pointing at `raw:` rather than as a change with no compatibility effect. +An unmarked array composes exactly as it does today, so no configuration changes meaning through composition. The one exception is the literal `$super` array element described above, whose marker recognition shipped in `v0.0.50` and is recorded there with a changelog entry pointing at `raw:` rather than as a change with no compatibility effect. ## Open Questions diff --git a/openspec/changes/array-composition-on-inheritance/proposal.md b/openspec/changes/array-composition-on-inheritance/proposal.md index 9a15bc1..ba31041 100644 --- a/openspec/changes/array-composition-on-inheritance/proposal.md +++ b/openspec/changes/array-composition-on-inheritance/proposal.md @@ -26,7 +26,7 @@ Supporting rules: ### Compatibility -No configuration changes meaning through composition: an unmarked array behaves exactly as before. There is one narrow break — **a document that contains the literal string `$super` as an array element**, which is data today and a marker afterwards. The remedy is `raw:$super`. This warrants a minor version bump and a changelog entry rather than being described as a change with no effect on existing documents. +No configuration changes meaning through composition: an unmarked array behaves exactly as before. There is one narrow break — **a document that contains the literal string `$super` as an array element**, which is data today and a marker afterwards. The remedy is `raw:$super`. Marker recognition introduced that break in `v0.0.50`, so the changelog records it under that release rather than describing the feature as having no effect on existing documents. ### Deliberately not included diff --git a/openspec/changes/array-composition-on-inheritance/tasks.md b/openspec/changes/array-composition-on-inheritance/tasks.md index fd4ebe8..2a766c8 100644 --- a/openspec/changes/array-composition-on-inheritance/tasks.md +++ b/openspec/changes/array-composition-on-inheritance/tasks.md @@ -13,18 +13,18 @@ ## 2. The pairing marker -- [ ] 2.1 Implement `$super*` pairing — literal prefix before the marker, index-wise pairing after it, unpaired elements surviving on both sides — recovering the index-wise recursion and keep-extras logic from the reverted commit `80908b2` where it fits; verify one autotest case per behaviour -- [ ] 2.2 Implement the pair rule by kind: object with object deep-merges, array with array takes the queue element, atom with atom takes the queue element, and `{}` keeps the inherited element unchanged; verify one autotest case per rule -- [ ] 2.3 Reject a pair whose sides are of different kinds, counting null as an atom; verify negative autotest cases for object-with-atom and array-with-object -- [ ] 2.4 Confirm kind agreement is required of the pair and not of keys inside a paired object merge, so a key whose values differ in kind is an override; verify an autotest case -- [ ] 2.5 Reject composing a `$super*` delta with a marked array on either side; verify negative autotest cases for a splice delta over a pairing delta and for a pairing delta over a splice delta -- [ ] 2.6 Confirm composition does not depend on file grouping; verify autotest cases for the layered layout and the flat layout rendering identically, and that both match the spec delta's expected result -- [ ] 2.7 Run `make test` and `tools/bin/autotest`; verify both pass and that every scenario in the spec delta is traceable to at least one autotest case +- [x] 2.1 Implement `$super*` pairing — literal prefix before the marker, index-wise pairing after it, unpaired elements surviving on both sides — recovering the index-wise recursion and keep-extras logic from the reverted commit `80908b2` where it fits; verify one autotest case per behaviour +- [x] 2.2 Implement the pair rule by kind: object with object deep-merges, array with array takes the queue element, atom with atom takes the queue element, and `{}` keeps the inherited element unchanged; verify one autotest case per rule +- [x] 2.3 Reject a pair whose sides are of different kinds, counting null as an atom; verify negative autotest cases for object-with-atom and array-with-object +- [x] 2.4 Confirm kind agreement is required of the pair and not of keys inside a paired object merge, so a key whose values differ in kind is an override; verify an autotest case +- [x] 2.5 Reject composing a `$super*` delta with a marked array on either side; verify negative autotest cases for a splice delta over a pairing delta and for a pairing delta over a splice delta +- [x] 2.6 Confirm composition does not depend on file grouping; verify autotest cases for the layered layout and the flat layout rendering identically, and that both match the spec delta's expected result +- [x] 2.7 Run `make test` and `tools/bin/autotest`; verify both pass and that every scenario in the spec delta is traceable to at least one autotest case ## 3. Documentation and release -- [ ] 3.1 Add the terms this change introduces — marker, marked array, inherited array, delta, splice, prefix, queue, pairing, kind, grounding — to `tools/etc/docs/concepts/terminology.adoc`, each defined once; verify no synonym for any of them appears in the documentation added by this change -- [ ] 3.2 Update the merge-semantics table and the future-consideration note in `tools/etc/docs/concepts/evaluation-model.adoc` left by #73 / #75, which this change makes out of date; verify the generated documentation builds and the note no longer describes array composition as future work -- [ ] 3.3 Document both markers in the reference documentation, including the reserved namespace and the two divergences from jq's `*`; verify the examples shown match autotest cases -- [ ] 3.4 Document that override-then-append is not expressible, and that restating inherited elements as padding silently becomes an override when the ancestor grows; verify the boundary and its example appear in the shipped documentation rather than only in `design.md` -- [ ] 3.5 Record the compatibility break for a literal `$super` array element, with `raw:` as the remedy, in the changelog, and bump the minor version; verify the changelog entry and version bump are present +- [x] 3.1 Add the terms this change introduces — marker, marked array, inherited array, delta, splice, prefix, queue, pairing, kind, grounding — to `tools/etc/docs/concepts/terminology.adoc`, each defined once; verify no synonym for any of them appears in the documentation added by this change +- [x] 3.2 Update the merge-semantics table and the future-consideration note in `tools/etc/docs/concepts/evaluation-model.adoc` left by #73 / #75, which this change makes out of date; verify the generated documentation builds and the note no longer describes array composition as future work +- [x] 3.3 Document both markers in the reference documentation, including the reserved namespace and the two divergences from jq's `*`; verify the examples shown match autotest cases +- [x] 3.4 Document that override-then-append is not expressible, and that restating inherited elements as padding silently becomes an override when the ancestor grows; verify the boundary and its example appear in the shipped documentation rather than only in `design.md` +- [x] 3.5 Backfill the compatibility break for literal `$super` and `$super*` array elements, with `raw:` as the remedy, under the `v0.0.50` changelog entry that corresponds to marker recognition; verify the changelog entry is present diff --git a/tools/etc/docs/concepts/evaluation-model.adoc b/tools/etc/docs/concepts/evaluation-model.adoc index a6f2b41..ead33e2 100644 --- a/tools/etc/docs/concepts/evaluation-model.adoc +++ b/tools/etc/docs/concepts/evaluation-model.adoc @@ -115,22 +115,18 @@ How each key is combined depends on the type of its value: |Recursive merge — parent and child objects are combined key by key using this same rule. |Array -|Replaced entirely — the child array overrides the parent; arrays are never merged element-wise. +|An unmarked child array replaces the inherited array. +A marked child array uses `$super` to splice inherited elements at the marker or `$super*` to pair elements after the marker by index. |Scalar (string, number, boolean, null) |Replaced entirely — the child value overrides the parent value. |=== -"Recursive" therefore applies only to nested objects; arrays and scalars are always replaced wholesale, never merged. -In jq{plus}{plus} terminology these wholesale replacements are _overrides_: the only _merge_ the engine performs is between two objects. +"Recursive" therefore applies only to nested objects and to object pairs created by `$super*` array composition. +In jq{plus}{plus} terminology, a child value replacing an inherited value is an _override_. This is the link:design-principles.html[same-kind merge principle]. -[NOTE] -==== -Array replacement is the current behavior. -We are considering enhancing the syntax so that a child can explicitly *append*, *prepend*, or *merge* an array with the one inherited from its ancestors, instead of always replacing it. -This is a future consideration and not yet supported; see https://github.com/dakusui/jqplusplus/issues/74[issue #74] for the tracking discussion. -==== +See the link:../reference/syntax-reference.html#_array_composition[Syntax Reference] for the marker syntax and its constraints. Given `$extends: [A, B]`, A is merged over B first, then the current object is merged over that result, giving the priority order: current object → A → B. diff --git a/tools/etc/docs/concepts/terminology.adoc b/tools/etc/docs/concepts/terminology.adoc index df3704c..9bad926 100644 --- a/tools/etc/docs/concepts/terminology.adoc +++ b/tools/etc/docs/concepts/terminology.adoc @@ -45,6 +45,30 @@ When multiple parents are listed, earlier entries in the `$extends` array take p Priority from highest to lowest: current object → first parent → … → last parent. +=== Array Composition + +*Array composition* is an explicit way for a child array to combine with the array it inherits during structural composition. + +*Marker*:: A marker is the direct array-element string `$super` or `$super*` that selects array composition. + +*Marked array*:: A marked array is an array containing exactly one marker. + +*Inherited array*:: An inherited array is the array supplied by the lower-precedence side of a structural composition. + +*Delta*:: A delta is a marked array carried forward until an inherited array is available to resolve it. + +*Splice*:: A splice is the `$super` operation that substitutes every inherited-array element at the marker's position. + +*Prefix*:: A prefix is the literal sequence before `$super*` in a marked array. + +*Queue*:: A queue is the sequence after `$super*` whose elements pair with inherited-array elements by index. + +*Pairing*:: Pairing is the `$super*` operation that combines corresponding queue and inherited-array elements. + +*Kind*:: A kind is one of object, array, or atom, where strings, numbers, booleans, and null are atoms. + +*Grounding*:: Grounding is the final check that reports a marker which no inherited array resolved. + === Inclusion (`$includes`) *Inclusion* is the mechanism by which an object pulls in one or more external fragments that are applied on top of the including object. diff --git a/tools/etc/docs/reference/syntax-reference.adoc b/tools/etc/docs/reference/syntax-reference.adoc index fcd1385..6dcfa23 100644 --- a/tools/etc/docs/reference/syntax-reference.adoc +++ b/tools/etc/docs/reference/syntax-reference.adoc @@ -37,6 +37,13 @@ This is the opposite precedence direction from `$extends`. Priority is: last fragment -> ... -> first fragment -> current object. If both `$extends` and `$includes` are present, `$includes` wins over the combined result. +|`$super` and `$super*` in an array +|Array composition +|A marked child array combines with its inherited array instead of replacing it. +`$super` splices inherited elements, while `$super*` pairs elements after the marker by index. +|A marker must be a direct array element. +An unmarked child array still replaces the inherited array. + |`$local` |Local node definitions |Defines reusable named objects inside the current file. @@ -193,6 +200,65 @@ Switching environments is a matter of changing `JF_PATH`—the configuration fil This pattern lets teams publish versioned "standard" fragments that any project can `$includes` without copying content. The result is a small but coherent DSL: every configuration file speaks the same vocabulary of shared fragments, and individual files only spell out what makes them different. +==== Array Composition + +A child array normally overrides the inherited array. +Place exactly one marker as a direct element of the child array to compose instead. + +`$super` splices all inherited elements at the marker's position. +It can therefore append, prepend, or wrap them. + +`P.json`: + +[source,json] +---- +{ "m": ["a", "b"] } +---- + +`input.json`: + +[source,json] +---- +{ "$extends": ["P.json"], "m": ["$super", "z"] } +---- + +The result is `{ "m": ["a", "b", "z"] }`. + +`$super*` keeps the elements before it as a literal prefix and pairs each element after it with the inherited element at the same index. +Object pairs merge recursively, while array and atom pairs take the child element. +Unpaired elements on either side remain in their original order. +Each pair must have the same kind: object, array, or atom. +Strings, numbers, booleans, and null are atoms. +A pair of different kinds is an error. + +`P.json`: + +[source,json] +---- +{ "m": [{"a": 1, "b": 2}] } +---- + +`input.json`: + +[source,json] +---- +{ "$extends": ["P.json"], "m": ["$super*", {"b": 20}] } +---- + +The result is `{ "m": [{"a": 1, "b": 20}] }`. + +Markers use the reserved `$super` namespace only in direct array-element position. +An unrecognized spelling in that position, or more than one marker in the same array, is an error. +Use `raw:$super` or `raw:$super*` when either literal string is required as an array element. + +`$super*` resembles jq's recursive object-merge operator, but differs in two ways. +It pairs arrays by index even though jq rejects the corresponding operation on two arrays, and it takes the child value for atom pairs even though jq applies arithmetic to two atoms. + +An array containing `$super*` cannot compose with an array containing either marker. +As a result, an override followed by an append is not expressible in one array composition. +Do not restate inherited elements as padding to simulate it: if the inherited array later grows, the intended appended element silently overrides the new inherited element instead. +Use `$super` when the goal is to append without overriding, or restructure the configuration so that the values have stable object keys. + ==== $local Defines named local objects that can be referenced within the same file.