From a36241e22f182825b2e0424a7dd36728c140bd2a Mon Sep 17 00:00:00 2001 From: "Jonathan D.A. Jewell" <6759885+hyperpolymath@users.noreply.github.com> Date: Mon, 3 Aug 2026 19:21:36 +0100 Subject: [PATCH 1/2] docs(a2ml): record the dialect architecture and what blocks the IANA submission MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit A design record, not a change to anything validated. It exists so the next person starts from the architecture rather than rediscovering it from 1,543 files. == The architecture is already real; the spec does not say so SPEC.adoc §3.1 already makes the Idris2 model in a2ml/src/A2ML/ authoritative and the prose shape informative. Eleven modules implement it, and the presence of Surface.idr AND Translator.idr AND Converters.idr is itself the evidence that the design always assumed several surfaces mapping onto one core. So A2ML is a typed core with surface projections, not a markup format with variants. A surface is legitimate exactly when it defines a translation into the core. That rule makes new surfaces additive rather than fragmenting, and it is why leaving one unspecified is the actual problem. == Census, re-measured 2026-08-03 1,543 .a2ml files, classified by first non-comment non-blank line: manifest (--- frontmatter) 913 specified by NOTHING record ([section]) 376 RECORD-DIALECT-SPEC.adoc markup (@directive) 139 SPEC.adoc §2 unclassified 115 see below The manifest dialect is 59% of the corpus and has no grammar. `grep -niE 'frontmatter|yaml'` across both specs returns zero hits, and SPEC.adoc §2 has no key-value production at all. An earlier census reported 906/456/154. Totals agree at 1,543 and the ordering is unchanged; the difference is that this pass separates the unclassified group. Both are recorded so neither figure looks authoritative on its own. New finding: 27 of the 115 unclassified files open with `;` or `;;` — Scheme/Lisp comment syntax. A small s-expression-shaped population exists that neither written spec counts. That strengthens the family-type design rather than complicating it: surfaces accrete whether or not a spec permits them. == Why it blocks IANA The registration (Revision 2, 2026-04-03) declares application/vnd.a2ml with `charset` as the only optional parameter. Submitting it would point IANA at a specification that does not describe 59% of its own corpus, and give a recipient no way to know which grammar a document follows. Worse, three OTHER media-type forms are already written down across the repo — application/vnd.a2ml+text (2 files), text/a2ml (3), application/vnd.{format}+a2ml (2) — none matching the registration. Reconciling those is part of the same pass. == The fix, which is mostly already decided RECORD-DIALECT-SPEC.adoc §14 recommends a `dialect=` parameter on one family type rather than separate types, and explicitly declines to impose it alone. This document takes that up: 1. Write MANIFEST-DIALECT-SPEC.adoc — the only substantial new writing. 2. Promote SPEC.adoc to family root: name the core canonical, name the dialects, define `dialect=`, state the translation rule. 3. Add dialect= to the registration and reconcile the three stale forms. 4. Then submit. K9 supplies the precedent: it already registers an optional parameter (`security-level`, default `kennel`, advisory on the wire and authoritative in the document). dialect= should follow that shape exactly. == K9 has an independent blocker, found while checking the precedent k9-svc/IANA-MEDIA-TYPE-APPLICATION.md says it "registers application/vnd.k9 without a structured syntax suffix". k9-svc/SPEC.adoc line 34 still declares `application/vnd.k9+nickel`. The spec and the application disagree on the media type, on current main. That is a live contradiction, not a design question, and it must be settled before K9 submits. Note also that the mandatory `K9!` magic line is not valid Nickel, so a .k9.ncl file is not parseable by a stock Nickel toolchain — which argues for dropping `+nickel`, since the suffix implies exactly that parseability. == Verified, not asserted Every path cited exists; the validator exemption is quoted from a2ml/actions/validate/validate-a2ml.sh lines 20-26 and 151-156; the "435 files ... conform to a grammar that existed only by imitation" quote is RECORD-DIALECT-SPEC.adoc lines 23-24; the three stale media-type strings were counted, not remembered. Deliberately NOT done: writing the manifest grammar. That needs the corpus surveyed for the YAML constructs actually used first — a permissive spec written blind is harder to narrow than a narrow one is to widen. --- a2ml/docs/DIALECT-ARCHITECTURE.adoc | 307 ++++++++++++++++++++++++++++ 1 file changed, 307 insertions(+) create mode 100644 a2ml/docs/DIALECT-ARCHITECTURE.adoc diff --git a/a2ml/docs/DIALECT-ARCHITECTURE.adoc b/a2ml/docs/DIALECT-ARCHITECTURE.adoc new file mode 100644 index 00000000..1cd54703 --- /dev/null +++ b/a2ml/docs/DIALECT-ARCHITECTURE.adoc @@ -0,0 +1,307 @@ +// SPDX-License-Identifier: CC-BY-SA-4.0 +// SPDX-FileCopyrightText: 2026 Jonathan D.A. Jewell += A2ML dialect architecture — canonical core, generated surfaces +:toc: left +:icons: font +:sectnums: +:status: PROPOSAL +:date: 2026-08-03 + +[abstract] +== Abstract + +A2ML has one canonical form and three surface dialects. The canonical form is +already normative and already exists; the surfaces are not all specified, and the +IANA registration describes only one of them. This document states the +architecture explicitly, identifies the single missing piece, and sets out the +sequence to an honest submission. + +It also covers K9, which has the same shape and an independent blocker of its own. + +[IMPORTANT] +==== +**Nothing here proposes a new design.** The architecture below is what the estate +already does. The problem is that `SPEC.adoc` does not say so, so two of the three +surfaces exist only by imitation and the media-type registration cannot describe +its own corpus. +==== + +== 1. The organising principle + +A2ML is *not* a markup format with variants. It is a **typed core with surface +projections**: + +.... + surface dialects canonical form + ──────────────── ────────────── + markup (@directive) ─┐ + record ([section]) ─┼──▶ Idris2 typed core ──▶ proofs, attestation + manifest (--- YAML) ─┘ a2ml/src/A2ML/ +.... + +`SPEC.adoc` §3.1 already states this and already makes the core authoritative: + +[quote, 'a2ml/SPEC.adoc §3.1 "Core Types (Idris2)"'] +____ +The normative Idris2 model lives in `src/A2ML/`. The shape below is informative; +the source files are authoritative. +____ + +Eleven Idris2 modules implement it — `Surface.idr`, `Parser.idr`, `Translator.idr`, +`Converters.idr`, `Profiles.idr`, `BaseVocab.idr`, `Proofs.idr` and their tests. +The presence of `Surface.idr` *and* `Translator.idr` *and* `Converters.idr` is +itself the evidence: the design has always assumed more than one surface mapping +onto one core. + +**Consequence.** A surface dialect is legitimate exactly when it has a defined +translation into the core. It does not need its own semantics, only its own +grammar and mapping. That is why adding YAML- or TOML-shaped surfaces is cheap +and safe, and why leaving one *unspecified* is not. + +== 2. The three surfaces, measured + +Census of all **1,543** `.a2ml` files in `standards`, classified by first +non-comment, non-blank line (2026-08-03): + +[cols="1,3,1,2"] +|=== +| Dialect | Surface | Files | Specified by + +| `manifest` +| `---` frontmatter, `key: value` (YAML-family) +| **913** +| **nothing** + +| `record` +| `[section]` / `[[array]]`, `key = value` (TOML-family) +| 376 +| `a2ml/RECORD-DIALECT-SPEC.adoc` + +| `markup` +| `@directive:` … `@end`, `#` headings, lists +| 139 +| `a2ml/SPEC.adoc` §2 + +| *(unclassified)* +| see §2.2 +| 115 +| — +|=== + +`grep -niE 'frontmatter\|yaml'` across *both* specs returns **zero** hits, and +`SPEC.adoc` §2 contains no key-value or mapping production at all — only Heading, +DirectiveBlock, InlineDirective, List, CodeBlock and Paragraph. + +So the largest surface in the corpus — **59%**, more than twice the next — is +described by no document. + +[NOTE] +==== +An earlier census using a different classifier reported 906 / 456 / 154. The +totals agree (1,543) and the ordering is unchanged; the per-dialect figures shift +because that pass did not separate the unclassified group below. Either way the +conclusion is the same and the manifest dialect is the largest by a wide margin. +==== + +`RECORD-DIALECT-SPEC.adoc` was written to fix precisely this for the record +dialect. Its own abstract concedes that *"435 files across the estate conform to a +grammar that existed only by imitation."* The manifest dialect is the same +condition, unfixed and twice the size. + +=== 2.1 A fourth, unwritten surface + +115 files match none of the three. Of those, **27 open with `;` or `;;`** — +Scheme/Lisp comment syntax. There is a small s-expression-shaped population in +the corpus, counted by neither written spec. + +The remainder are prose (`**THIS…`, `This…`), stray HTML comments, a bare +`version` line, and files whose only content is an SPDX header — i.e. files that +are arguably not A2ML documents at all and should be triaged separately. + +The s-expression group matters for the design: it is evidence that surfaces +accrete whether or not the spec permits them. A family type with a `dialect=` +parameter accommodates that; a single type with a single grammar cannot. + +=== 2.2 The toolchain already works around it + +`a2ml/actions/validate/validate-a2ml.sh` special-cases `AI-MANIFEST*.a2ml` and `AI.a2ml` as "markdown +prose" and skips identity checks on them. That exemption is correct as a stopgap +and **must not be read as acceptance**: it is the validator declining to check +913 files because there is no grammar to check them against. + +== 3. Why this blocks IANA + +The current registration (`a2ml/docs/iana/application-vnd.a2ml-registration.txt`, +Revision 2, 2026-04-03) registers: + +[cols="1,3"] +|=== +| Type | `application/vnd.a2ml` +| Required parameters | None +| Optional parameters | `charset` only +|=== + +One type, no dialect discrimination. Registering it as it stands would point IANA +at a published specification that does not describe 59% of its own corpus, and +would give a recipient no way to know which grammar a given document follows. + +**Hold the submission until §4 is done.** The application is otherwise complete +and owner-submittable. + +== 4. The fix + +`RECORD-DIALECT-SPEC.adoc` §14 already proposes the right mechanism, and +explicitly declines to impose it alone: + +[quote, 'a2ml/RECORD-DIALECT-SPEC.adoc §14'] +____ +This specification does *not* unilaterally resolve that inconsistency. It records +only the requirement that, once the family type is registered, the record dialect +be distinguishable from the markup dialect — RECOMMENDED via a `dialect=record` +parameter on the family media type rather than a separate type, since both +dialects share one core model. +____ + +That section also records a problem this document must not gloss: **three +inconsistent media-type forms** are already written down across the repository — +`application/vnd.a2ml+text`, `text/a2ml`, and `application/vnd.{format}+a2ml` — +none of which matches the `application/vnd.a2ml` in the registration text. Those +must be reconciled to the family type in the same pass, or the spec corpus will +contradict the submission. + +Stated in full: + +=== 4.1 One family type, one parameter + +.... +application/vnd.a2ml; dialect=markup +application/vnd.a2ml; dialect=record +application/vnd.a2ml; dialect=manifest +.... + +Not three types. A2ML is one format with one core; the dialect names which +surface grammar a given document uses to reach it. + +*Default.* `dialect=markup` when the parameter is absent, matching both the +historical surface and `SPEC.adoc` §2. + +*Precedent.* K9 already registers an optional parameter this way — +`security-level` with a default of `kennel`, declared advisory with the +authoritative value inside the document. `dialect=` should follow the same +pattern: advisory on the wire, authoritative from the document's own shape. + +=== 4.2 Write the manifest-dialect spec + +The one genuinely missing artefact. It needs: + +* the frontmatter delimiter rules (`---` open/close, position relative to an SPDX + comment header); +* the key-value grammar and permitted scalar types; +* nesting, sequences, and the subset of YAML deliberately *excluded* — anchors, + aliases, tags and multi-document streams should be out, since none appears in + the corpus and each is a known parser-differential risk; +* the mapping into the Idris2 core, which is what makes it A2ML rather than YAML. + +Either a sibling `MANIFEST-DIALECT-SPEC.adoc` or a new section of the record spec. +A sibling is preferable: the two grammars share nothing, and the record spec's +abstract is already specific to its own surface. + +=== 4.3 Make SPEC.adoc the family root + +`SPEC.adoc` currently never uses the word "dialect" and does not acknowledge that +surfaces 2 and 3 exist. It should: + +. name the Idris2 core as the canonical form and the surfaces as projections; +. name all three dialects and link their specs; +. define the `dialect=` parameter and its default; +. state the rule from §1 — a surface is legitimate iff it defines a translation + into the core. + +That last point is what makes future surfaces (TOML-shaped, or a genuinely +s-expression one) additive rather than fragmenting. + +=== 4.4 Then submit + +Update the registration with the family type plus `dialect=`, and submit. + +== 5. On generated surfaces + +The natural reading of §1 is that surfaces can be *generated* from the core rather +than hand-written, and that is worth stating as intent: + +* A document's authoritative content is its core representation. +* Any surface is a projection of it, chosen for the objective at hand — record + form where typing and diffability matter, manifest form where tooling expects + frontmatter, markup form for prose. +* Round-tripping is therefore a property to be *proved*, not assumed. + `a2ml/src/A2ML/Proofs.idr` is where that obligation belongs, and + `Converters.idr` is where the projections live. + +This is the strongest argument for fixing the manifest dialect rather than +tolerating it: a surface with no grammar cannot be a projection of anything, +because there is nothing to prove the projection against. + +[NOTE] +==== +`scm2a2ml` is *not* part of this picture, despite the name suggesting a fourth +surface. It converts **legacy Scheme `.scm` metadata files into A2ML** — a +one-way migration away from a superseded format, not an A2ML dialect. Nothing +should be built on the assumption that `.scm` is an A2ML surface. +==== + +== 6. K9 — same shape, and a blocker of its own + +A2ML and K9 are siblings: A2ML attests documents, K9 attests components, and the +estate uses them together. K9's structure mirrors §1 — a Nickel-typed contract +core with `.k9` / `.k9.ncl` surfaces. + +Two things need settling before K9 submits, and one is independent of A2ML: + +=== 6.1 The spec and the application disagree on the media type + +[cols="2,3"] +|=== +| `k9-svc/IANA-MEDIA-TYPE-APPLICATION.md` +| "This revision registers **`application/vnd.k9`** without a structured syntax suffix" + +| `k9-svc/SPEC.adoc` line 34 +| `Standard ID \| application/vnd.k9+nickel` +|=== + +One drops `+nickel`; the other still declares it. Whichever is right, both must +say the same thing before submission. **This is a live contradiction in the +current `main`, not a design question.** + +Note also that the `K9!` magic line on line 1 is not valid Nickel, so a `.k9.ncl` +file is not parseable by a stock Nickel toolchain — which is an argument for +`application/vnd.k9` without the suffix, since `+nickel` implies exactly that +parseability. + +=== 6.2 Coordinate the parameter design + +If A2ML adopts `dialect=` and K9 keeps `security-level=`, the two registrations +should be written to the same pattern — optional, defaulted, advisory-on-the-wire, +authoritative-in-document — so the pair reads as one family of decisions rather +than two unrelated ones. + +== 7. Sequence + +. Write `MANIFEST-DIALECT-SPEC.adoc` (§4.2). *The only substantial new writing.* +. Promote `SPEC.adoc` to family root (§4.3). +. Add `dialect=` to the A2ML registration (§4.1). +. Resolve the K9 spec/application contradiction (§6.1). +. Align the two parameter designs (§6.2). +. Submit both. + +Steps 1–3 are A2ML-only and independent of 4–5. + +== 8. What this document does not do + +It does not write the manifest grammar. That requires deciding which YAML subset +is in scope, and the corpus should be surveyed for the constructs actually used +before the grammar is fixed — writing a permissive spec first and discovering the +corpus needs less is easier than the reverse. + +It also does not change any file the estate validates against. It is a design +record, so that the next person to pick this up starts from the architecture +rather than rediscovering it from 1,543 files. From 3a8745afe62032d9327e74e662c717d4d4b6f800 Mon Sep 17 00:00:00 2001 From: "Jonathan D.A. Jewell" <6759885+hyperpolymath@users.noreply.github.com> Date: Mon, 3 Aug 2026 19:27:35 +0100 Subject: [PATCH 2/2] =?UTF-8?q?spec(a2ml):=20specify=20the=20manifest=20di?= =?UTF-8?q?alect=20=E2=80=94=20the=2059%=20that=20had=20no=20grammar?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit The largest A2ML surface in the estate had no specification. SPEC.adoc §2 has no key-value production and neither spec mentions frontmatter, so 894 files conformed to a grammar that existed only by imitation — the same condition RECORD-DIALECT-SPEC.adoc was written to fix for the record dialect, at twice the size. Written from a census, not from YAML. Every rule traces to a measured count. == What the corpus actually uses key: value 2706 double-quoted 1799 integer 880 block sequence (- ) 29 comment 893 block scalar (| >) 2 nested (indented) 19 flow seq [..] 2 bare scalar 6 indent widths: 0 (2684), 4 (15), 2 (7) 29 distinct keys And what it does NOT use, anywhere in 894 files: anchors & 0 aliases * 0 tags ! 0 flow maps {} 0 single quotes 0 multi-doc 0 Excluding those is not a restriction, it is a description — and it closes the three constructs most often implicated in YAML parser differentials for free. Anchors and aliases also defeat attestation over a canonical hash form, since they make a document's meaning non-local. == A spec bug the validator caught The first draft admitted the block-scalar introducer `|` but said nothing about its continuation lines, so the validator rejected Trustfile.a2ml: signature: | -----BEGIN HYBRID SIGNATURE----- BASE64_Ed448+Dilithium5_SIGNATURE_OF_THIS_FILE -----END HYBRID SIGNATURE----- A grammar that admits `|` but not its content rejects precisely the construct A2ML exists to carry. §4 now defines `block-body` as opaque: a `#` inside one is signature data, and a `---` inside one does not close the frontmatter. Corpus conformance went 893/894 -> 894/894. This is why the spec was written against a running validator rather than by inspection. == The 19 "malformed" files are not this dialect Nineteen files open `---` after their header and never close it. A naive classifier calls them broken manifests. They are not: 7 use `key = value` (record dialect) and 12 use `## heading` (markup), with `---` as a horizontal rule. So `---` is AMBIGUOUS ACROSS DIALECTS. §8 resolves it one way — in the manifest dialect a leading `---` opens frontmatter — and records that the family root must say which dialect claims the token in first position. The 19 files are listed as a known fix, not a surprise. == Reference validator a2ml/actions/validate/validate-manifest-dialect.py — 894/894 conforming, exits non-zero on failure so it can gate CI. Proved it can fail: injecting a YAML anchor into one file gives 893/894 and exit 1; removing it returns exit 0. == Deliberately not done Fixing the vocabulary. Key semantics belong to the base record vocabulary and the profile mechanism in SPEC.adoc; this document specifies only the surface. Removing the AI-MANIFEST exemption in validate-a2ml.sh. It should go now that a grammar exists, but that is a behaviour change to a gate the estate runs, and it belongs in its own commit. Tracked in §9. --- a2ml/MANIFEST-DIALECT-SPEC.adoc | 340 ++++++++++++++++++ .../validate/validate-manifest-dialect.py | 58 +++ 2 files changed, 398 insertions(+) create mode 100644 a2ml/MANIFEST-DIALECT-SPEC.adoc create mode 100644 a2ml/actions/validate/validate-manifest-dialect.py diff --git a/a2ml/MANIFEST-DIALECT-SPEC.adoc b/a2ml/MANIFEST-DIALECT-SPEC.adoc new file mode 100644 index 00000000..faa6d68f --- /dev/null +++ b/a2ml/MANIFEST-DIALECT-SPEC.adoc @@ -0,0 +1,340 @@ +// SPDX-License-Identifier: CC-BY-SA-4.0 +// SPDX-FileCopyrightText: 2026 Jonathan D.A. Jewell += A2ML Manifest Dialect Specification +:version: 0.1.0 +:date: 2026-08-03 +:status: Draft +:era: v1 +:toc: left +:toclevels: 3 +:icons: font +:sectnums: + +[abstract] +== Abstract + +This document specifies the *manifest dialect* of A2ML: the `---`-delimited +frontmatter surface used by `0-AI-MANIFEST.a2ml` and its siblings. At 894 +well-formed files it is the largest A2ML surface in the estate, and until now it +was described by no specification at all — `SPEC.adoc` §2 has no key-value +production, and neither it nor `RECORD-DIALECT-SPEC.adoc` mentions frontmatter. + +Like `RECORD-DIALECT-SPEC.adoc`, this is a specification of a *deployed* surface. +It was written from a census of every `.a2ml` file in `standards`, and it +describes what is there. Where the corpus is silent, this document says so rather +than inventing a rule. + +== 1. Status and scope + +*Status: Draft.* Not ratified and not yet wired into CI. It has, however, been +exercised by a reference validator against the full corpus: + +[source,console] +---- +$ python3 a2ml/actions/validate/validate-manifest-dialect.py + well-formed frontmatter files : 894 + CONFORM to the spec as written: 894 (100%) + rejected : 0 +---- + +The validator is a real gate, not a formality — injecting an excluded construct +(a YAML anchor) into one file gives 893/894 and exit 1. + +It MUST NOT be cited as a ratified standard until it is owner-ratified and wired +into CI (§9). + +This dialect is one of several surfaces over the shared A2ML typed core. The core +is normative and lives in `a2ml/src/A2ML/`; see `docs/DIALECT-ARCHITECTURE.adoc` +for the family design and `SPEC.adoc` §3 for the core itself. A surface is +legitimate only if it defines a translation into that core (§7). + +== 2. Census — what this specifies + +All 1,543 `.a2ml` files in `standards`, 2026-08-03: + +[cols="2,1,4"] +|=== +| Shape | Files | Note + +| Frontmatter, `---` opened and closed +| **894** +| Specified by this document + +| `---` opened, never closed +| 19 +| *Not* this dialect — see §8 + +| Other dialects (record, markup) and unclassified +| 630 +| `RECORD-DIALECT-SPEC.adoc`, `SPEC.adoc` §2 +|=== + +Within the 894 well-formed blocks: **2,706** key-value lines, **29** distinct +keys, and a value vocabulary of five kinds. + +== 3. Document structure + +.... +a2ml-manifest ::= header-comments open-delim frontmatter close-delim body +open-delim ::= "---" LF +close-delim ::= "---" LF +.... + +`header-comments`:: Zero or more lines that are blank or begin with `#` or `//`. +In 904 of 913 observed files this is exactly one line — the SPDX identifier. + +`open-delim`:: A line whose entire trimmed content is `---`. It MUST be the first +non-comment, non-blank line. **A document whose first non-comment line is not +`---` is not this dialect.** + +`frontmatter`:: The subject of §4. MAY be empty. + +`close-delim`:: The next line whose entire trimmed content is `---`. +**REQUIRED.** See §8 for why this is stated so strongly. + +`body`:: Everything after `close-delim`. Not specified here; it is prose or +markup-dialect content, and every one of the 894 observed files has a non-empty +body (median 3 lines). + +== 4. Frontmatter grammar + +.... +frontmatter ::= ( entry | seq-item | comment | blank )* +entry ::= indent key ":" SP value LF + | indent key ":" SP block-intro LF block-body +indent ::= ( SP SP )* ; 0, 2 or 4 spaces observed +key ::= ALPHA-or-"_" ( ALNUM | "_" | "." | "-" )* +value ::= dq-string | integer | nested | flow-seq | bare-scalar +seq-item ::= indent "-" SP any* LF +block-intro ::= ( "|" | ">" ) ( "-" | "+" )? +block-body ::= ( block-line )* +block-line ::= +comment ::= indent "#" any* LF +.... + +`block-body` is **opaque**. Its lines are not parsed as entries, comments or +delimiters; they are content. This matters — the only corpus use of a block +scalar is a PEM-armoured signature: + +[source,yaml] +---- +signature: | + -----BEGIN HYBRID SIGNATURE----- + BASE64_Ed448+Dilithium5_SIGNATURE_OF_THIS_FILE + -----END HYBRID SIGNATURE----- +---- + +A grammar that admits `|` but not its continuation lines would reject exactly the +construct A2ML exists to carry. A `#` inside a block body is signature data, not +a comment, and a line beginning `---` inside one does **not** close the +frontmatter. + +Indentation is **spaces only**. Tabs MUST NOT be used; none appears in the corpus. + +Observed indent widths: `0` (2,684 lines), `4` (15), `2` (7). Nesting is +therefore rare — 22 of 2,706 entries — but real, and a conforming parser MUST +support it. + +=== 4.1 Values + +[cols="2,1,4"] +|=== +| Kind | Count | Form + +| `dq-string` +| 1,799 +| `"` … `"`. Double quotes only. + +| `integer` +| 880 +| Optional `-`, then digits. + +| `nested` +| 19 +| Empty after the colon; the following more-indented lines are a nested mapping. + +| `bare-scalar` +| 6 +| Unquoted, no leading/trailing space, terminated by end of line. + +| `flow-seq` +| 2 +| `[` item ( `,` item )* `]` +|=== + +A conforming *producer* SHOULD emit `dq-string` for all string values. Bare +scalars are permitted for compatibility with the six existing occurrences and are +NOT RECOMMENDED for new documents: they are the construct most likely to be +misread as a boolean, null or number. + +Sequences may also appear in block form (`- item` on its own line); 29 such lines +were observed. + +=== 4.2 Excluded YAML constructs + +This dialect is *not* YAML. The following are **NOT PERMITTED**, and none occurs +anywhere in the corpus: + +[cols="2,1,4"] +|=== +| Construct | Occurrences | Rationale for exclusion + +| Anchors `&name` +| **0** +| Aliasing makes a document's meaning non-local, which defeats attestation over a + canonical hash form. + +| Aliases `*name` +| **0** +| As above; also the basis of billion-laughs style expansion attacks. + +| Tags `!` / `!!` +| **0** +| Type coercion by tag lets the same bytes mean different things to different + parsers. + +| Flow mappings `{…}` +| **0** +| No occurrences; excluding it removes a whole nesting path from the grammar. + +| Single-quoted strings +| **0** +| Two string forms with different escaping rules is a parser-differential risk for + no expressive gain. + +| Multi-document streams `---` … `---` … +| **0** +| The second `---` is the close delimiter. See §8. + +| Block scalars `\|` `>` +| 2 +| Permitted but NOT RECOMMENDED — see below. +|=== + +Block scalars appear twice. They are permitted for those two files rather than +retroactively invalidating them, but new documents SHOULD use a quoted string. +Their chomping and folding rules are the single largest source of disagreement +between YAML implementations. + +[NOTE] +==== +Excluding anchors, aliases and tags is not a restriction — it is a description. +They are absent from all 894 files. Writing them out of the grammar costs nothing +and removes the three constructs most often implicated in YAML parser +differentials. +==== + +== 5. Key vocabulary + +29 distinct keys across the corpus. The ten most frequent: + +`id` · `level` · `parent` · `version` · `context` · `algorithm` · `value` · +`integrity` · `meta` · `generated` + +This document does **not** fix the vocabulary. Key semantics belong to the base +record vocabulary and the profile mechanism in `SPEC.adoc`; this dialect +specifies only the *surface* in which keys are written. A future profile MAY +constrain which keys are required. + +== 6. Conformance + +A document conforms to the manifest dialect at level *R1* when: + +. its first non-comment, non-blank line is `---`; +. a matching `---` close delimiter exists; +. every non-blank, non-comment line between them parses as `entry`; +. no excluded construct from §4.2 appears; +. indentation is spaces only, in multiples of two. + +A validator MUST reject a document failing 1–5 rather than skipping it. This +matters: `a2ml/actions/validate/validate-a2ml.sh` currently *exempts* +`AI-MANIFEST*.a2ml` and `AI.a2ml` from identity checks, treating them as +"markdown prose". That exemption existed because there was no grammar to check +against. **With this document, the exemption should be removed** — see §9. + +== 7. Translation into the core + +The mapping into `a2ml/src/A2ML/`: + +[cols="2,3"] +|=== +| Manifest surface | Core + +| Frontmatter block | The document's metadata mapping +| `entry` with `dq-string` / `bare-scalar` | `String` value +| `entry` with `integer` | Numeric value +| `entry` with `nested` | Sub-mapping +| `entry` with sequence | List value +| Body after the close delimiter | Markup-dialect block content, per `SPEC.adoc` §2 +|=== + +This is the part that makes the dialect A2ML rather than "YAML in a file with an +unusual extension". `Translator.idr` and `Converters.idr` are where it belongs; +neither currently implements it, which is tracked in §9. + +Round-tripping — surface → core → surface — is a property to be *proved*, not +assumed. `Proofs.idr` is the place for that obligation. + +== 8. The 19 unterminated files are NOT this dialect + +Nineteen files open with `---` after their comment header and never close it. A +naive classifier reads them as malformed manifests. They are not: + +[source] +---- +# SPDX-License-Identifier: MPL-2.0 +# must.manifest.a2ml — Trident coherence manifest for the must verb. + +--- +trident_version = "1.0.0" <-- equals, not colon: record dialect +verb = "must" + +## Files (three; exactly) <-- markup heading +---- + +Of the 19: **7** use `key = value` (record dialect), **12** use `## heading` +(markup dialect). The `---` is a *horizontal rule*, not a frontmatter delimiter. + +**This is a genuine ambiguity in the family and this specification resolves it +one way only:** in the manifest dialect, a `---` as the first non-comment line +opens frontmatter. A document that uses `---` as a rule MUST NOT also place it +first, or it is indistinguishable from a manifest whose close delimiter is +missing. + +Two consequences: + +* A manifest parser fed one of these 19 files would consume the entire file as + frontmatter and fail. That is correct behaviour under this spec. +* The 19 files should be corrected — either by removing the leading `---` or by + moving it below their first content line. They are recorded here so the fix is + a known task rather than a surprise. + +The wider point for `SPEC.adoc` as family root: **`---` is currently ambiguous +across dialects**, and the family spec should say which dialect claims it in +first position. + +== 9. Open items before ratification + +. *Reference validator.* ✅ Done — `a2ml/actions/validate/validate-manifest-dialect.py`, + 894/894 conforming. It found one spec bug during drafting: the grammar admitted + the block-scalar introducer `|` but not its continuation lines, so it rejected a + PEM-armoured signature — the very construct A2ML exists to carry. Fixed in §4. + *Still to do:* wire it into CI. +. *Remove the blanket exemption.* `validate-a2ml.sh` skips identity checks on + `AI-MANIFEST*.a2ml`. Once the validator above exists, the exemption should go. +. *Implement the translation.* §7 in `Translator.idr` / `Converters.idr`, with a + round-trip proof obligation in `Proofs.idr`. +. *Fix the 19.* §8. +. *Register the dialect.* Add `dialect=manifest` to the family media type — see + `docs/DIALECT-ARCHITECTURE.adoc` §4 and `RECORD-DIALECT-SPEC.adoc` §14. + +== 10. Versioning + +Versioned with SemVer, independently of `SPEC.adoc` and +`RECORD-DIALECT-SPEC.adoc`, but within the same era (`v1`) and over the same core. + +A change that would reject a currently-conforming corpus document is a MAJOR bump +and requires a migration note and a deprecation window. Permitting a construct +currently excluded by §4.2 is a MINOR bump — but the burden is on the proposer to +show the construct is needed, since each one re-opens a parser-differential +surface that is presently closed. diff --git a/a2ml/actions/validate/validate-manifest-dialect.py b/a2ml/actions/validate/validate-manifest-dialect.py new file mode 100644 index 00000000..b8d043e1 --- /dev/null +++ b/a2ml/actions/validate/validate-manifest-dialect.py @@ -0,0 +1,58 @@ +#!/usr/bin/env python3 +# SPDX-License-Identifier: MPL-2.0 +# SPDX-FileCopyrightText: 2026 Jonathan D.A. Jewell +# +# validate-manifest-dialect.py — reference validator for MANIFEST-DIALECT-SPEC.adoc +# +# Checks every .a2ml file whose first non-comment line is `---` and which has a +# matching close delimiter. Files that open `---` and never close it are NOT this +# dialect (spec §8) and are skipped, not failed. +# +# Exits non-zero if any file fails, so it can gate CI. +# +# Current corpus: 894/894 conform. +import sys,re,subprocess,collections +files=subprocess.run(['git','ls-files','*.a2ml'],capture_output=True,text=True).stdout.split() +KEY=re.compile(r'^([ ]*)([A-Za-z_][\w.\-]*)[ ]*:[ ]?(.*)$') +SEQ=re.compile(r'^([ ]*)-[ ]+\S') +EXCL=[('anchor',re.compile(r':\s*&\w|^\s*&\w')),('alias',re.compile(r':\s*\*\w')), + ('tag',re.compile(r':\s*!!?\w')),('flow-map',re.compile(r':\s*\{')), + ('single-quote',re.compile(r":\s*'"))] +tot=ok=0; fails=collections.Counter(); examples=collections.defaultdict(list) +for f in files: + L=open(f,encoding='utf-8',errors='replace').read().split('\n') + i=0 + while iblk: continue # opaque block content + blk=None + if not s.strip() or re.match(r'^\s*#',s): continue + if '\t' in s: errs.append('tab-indent') + for nm,rx in EXCL: + if rx.search(s): errs.append(f'excluded:{nm}') + m=KEY.match(s) + if m: + if len(m.group(1))%2: errs.append('odd-indent') + if re.match(r'^[|>][-+]?$',m.group(3).strip()): blk=len(m.group(1)) + elif SEQ.match(s): pass + elif re.match(r'^[ ]*[|>][-+]?\s*$',s): pass + else: errs.append('unparsed-line') + if errs: + for e in set(errs): + fails[e]+=1 + if len(examples[e])<3: examples[e].append(f"{f}") + else: ok+=1 +print(f" well-formed frontmatter files : {tot}") +print(f" CONFORM to the spec as written: {ok} ({100*ok//tot}%)") +print(f" rejected : {tot-ok}") +for k,v in fails.most_common(): + print(f" {k:22} {v} e.g. {examples[k][0] if examples[k] else ''}") + +sys.exit(0 if tot-ok==0 else 1)