A2ML: dialect architecture + the manifest-dialect spec (59% of the corpus) - #569
Conversation
…submission
A design record, not a change to anything validated. It exists so the next
person starts from the architecture rather than rediscovering it from 1,543
files.
== The architecture is already real; the spec does not say so
SPEC.adoc §3.1 already makes the Idris2 model in a2ml/src/A2ML/ authoritative
and the prose shape informative. Eleven modules implement it, and the presence
of Surface.idr AND Translator.idr AND Converters.idr is itself the evidence that
the design always assumed several surfaces mapping onto one core.
So A2ML is a typed core with surface projections, not a markup format with
variants. A surface is legitimate exactly when it defines a translation into the
core. That rule makes new surfaces additive rather than fragmenting, and it is
why leaving one unspecified is the actual problem.
== Census, re-measured 2026-08-03
1,543 .a2ml files, classified by first non-comment non-blank line:
manifest (--- frontmatter) 913 specified by NOTHING
record ([section]) 376 RECORD-DIALECT-SPEC.adoc
markup (@directive) 139 SPEC.adoc §2
unclassified 115 see below
The manifest dialect is 59% of the corpus and has no grammar. `grep -niE
'frontmatter|yaml'` across both specs returns zero hits, and SPEC.adoc §2 has no
key-value production at all.
An earlier census reported 906/456/154. Totals agree at 1,543 and the ordering is
unchanged; the difference is that this pass separates the unclassified group.
Both are recorded so neither figure looks authoritative on its own.
New finding: 27 of the 115 unclassified files open with `;` or `;;` — Scheme/Lisp
comment syntax. A small s-expression-shaped population exists that neither
written spec counts. That strengthens the family-type design rather than
complicating it: surfaces accrete whether or not a spec permits them.
== Why it blocks IANA
The registration (Revision 2, 2026-04-03) declares application/vnd.a2ml with
`charset` as the only optional parameter. Submitting it would point IANA at a
specification that does not describe 59% of its own corpus, and give a recipient
no way to know which grammar a document follows.
Worse, three OTHER media-type forms are already written down across the repo —
application/vnd.a2ml+text (2 files), text/a2ml (3), application/vnd.{format}+a2ml
(2) — none matching the registration. Reconciling those is part of the same pass.
== The fix, which is mostly already decided
RECORD-DIALECT-SPEC.adoc §14 recommends a `dialect=` parameter on one family type
rather than separate types, and explicitly declines to impose it alone. This
document takes that up:
1. Write MANIFEST-DIALECT-SPEC.adoc — the only substantial new writing.
2. Promote SPEC.adoc to family root: name the core canonical, name the
dialects, define `dialect=`, state the translation rule.
3. Add dialect= to the registration and reconcile the three stale forms.
4. Then submit.
K9 supplies the precedent: it already registers an optional parameter
(`security-level`, default `kennel`, advisory on the wire and authoritative in
the document). dialect= should follow that shape exactly.
== K9 has an independent blocker, found while checking the precedent
k9-svc/IANA-MEDIA-TYPE-APPLICATION.md says it "registers application/vnd.k9
without a structured syntax suffix". k9-svc/SPEC.adoc line 34 still declares
`application/vnd.k9+nickel`. The spec and the application disagree on the media
type, on current main. That is a live contradiction, not a design question, and
it must be settled before K9 submits.
Note also that the mandatory `K9!` magic line is not valid Nickel, so a .k9.ncl
file is not parseable by a stock Nickel toolchain — which argues for dropping
`+nickel`, since the suffix implies exactly that parseability.
== Verified, not asserted
Every path cited exists; the validator exemption is quoted from
a2ml/actions/validate/validate-a2ml.sh lines 20-26 and 151-156; the "435 files
... conform to a grammar that existed only by imitation" quote is
RECORD-DIALECT-SPEC.adoc lines 23-24; the three stale media-type strings were
counted, not remembered.
Deliberately NOT done: writing the manifest grammar. That needs the corpus
surveyed for the YAML constructs actually used first — a permissive spec written
blind is harder to narrow than a narrow one is to widen.
|
Note Automatic reviews are paused because your trial's included automatic processing has been used for this period. Upgrade now, or comment "Gitar review" to run a review anytime. Code Review ✅ ApprovedDocuments the A2ML dialect architecture and identifies the missing manifest grammar blocking IANA submission. No issues found.
OptionsDisplay: compact → Showing less information. Comment with these commands to change the behavior for this request:
Important Your trial ends in 7 days — upgrade now to keep code review, CI analysis, auto-apply, custom automations, and more. Was this helpful? React with 👍 / 👎 | Gitar |
![gitar-bot[bot]](https://avatars.githubusercontent.com/in/827041?s=60&v=4){kind=small}
The largest A2ML surface in the estate had no specification. SPEC.adoc §2 has no
key-value production and neither spec mentions frontmatter, so 894 files
conformed to a grammar that existed only by imitation — the same condition
RECORD-DIALECT-SPEC.adoc was written to fix for the record dialect, at twice the
size.
Written from a census, not from YAML. Every rule traces to a measured count.
== What the corpus actually uses
key: value 2706 double-quoted 1799
integer 880 block sequence (- ) 29
comment 893 block scalar (| >) 2
nested (indented) 19 flow seq [..] 2
bare scalar 6
indent widths: 0 (2684), 4 (15), 2 (7) 29 distinct keys
And what it does NOT use, anywhere in 894 files:
anchors & 0 aliases * 0 tags ! 0
flow maps {} 0 single quotes 0 multi-doc 0
Excluding those is not a restriction, it is a description — and it closes the
three constructs most often implicated in YAML parser differentials for free.
Anchors and aliases also defeat attestation over a canonical hash form, since
they make a document's meaning non-local.
== A spec bug the validator caught
The first draft admitted the block-scalar introducer `|` but said nothing about
its continuation lines, so the validator rejected Trustfile.a2ml:
signature: |
-----BEGIN HYBRID SIGNATURE-----
BASE64_Ed448+Dilithium5_SIGNATURE_OF_THIS_FILE
-----END HYBRID SIGNATURE-----
A grammar that admits `|` but not its content rejects precisely the construct
A2ML exists to carry. §4 now defines `block-body` as opaque: a `#` inside one is
signature data, and a `---` inside one does not close the frontmatter.
Corpus conformance went 893/894 -> 894/894. This is why the spec was written
against a running validator rather than by inspection.
== The 19 "malformed" files are not this dialect
Nineteen files open `---` after their header and never close it. A naive
classifier calls them broken manifests. They are not: 7 use `key = value`
(record dialect) and 12 use `## heading` (markup), with `---` as a horizontal
rule.
So `---` is AMBIGUOUS ACROSS DIALECTS. §8 resolves it one way — in the manifest
dialect a leading `---` opens frontmatter — and records that the family root
must say which dialect claims the token in first position. The 19 files are
listed as a known fix, not a surprise.
== Reference validator
a2ml/actions/validate/validate-manifest-dialect.py — 894/894 conforming, exits
non-zero on failure so it can gate CI.
Proved it can fail: injecting a YAML anchor into one file gives 893/894 and
exit 1; removing it returns exit 0.
== Deliberately not done
Fixing the vocabulary. Key semantics belong to the base record vocabulary and the
profile mechanism in SPEC.adoc; this document specifies only the surface.
Removing the AI-MANIFEST exemption in validate-a2ml.sh. It should go now that a
grammar exists, but that is a behaviour change to a gate the estate runs, and it
belongs in its own commit. Tracked in §9.
hyperpolymath
changed the title
Update — the manifest dialect is now specifiedThe one substantial piece of writing this PR originally deferred is done: Written from a census, not from YAMLEvery rule traces to a measured count across the 894 well-formed files:
And what the corpus does not contain, anywhere:
Excluding those isn't a restriction — it's a description. It also closes the three constructs most often implicated in YAML parser differentials, for free. Anchors and aliases additionally defeat attestation over a canonical hash form, since they make a document's meaning non-local. The validator caught a spec bugThe first draft admitted the block-scalar introducer signature: |
-----BEGIN HYBRID SIGNATURE-----
BASE64_Ed448+Dilithium5_SIGNATURE_OF_THIS_FILE
-----END HYBRID SIGNATURE-----A grammar that admits Corpus conformance: 893/894 → 894/894. That's why the spec was written against a running validator rather than by inspection. The 19 "malformed" files are a different dialectNineteen files open So Reference validator
Proved it can fail: injecting a YAML anchor gives 893/894 and exit 1; removing it returns exit 0. Remaining before submission
|
![github-advanced-security[bot]](https://avatars.githubusercontent.com/in/57789?s=60&v=4){kind=small}
| @@ -0,0 +1,58 @@ | |||
| #!/usr/bin/env python3 | |||
|
2 participants
A design record, not a change to anything validated. It exists so the next person starts from the architecture rather than rediscovering it from 1,543 files.
The architecture is already real — the spec just doesn't say so
SPEC.adoc§3.1 already makes the Idris2 model ina2ml/src/A2ML/authoritative and the prose shape informative. Eleven modules implement it, and the presence ofSurface.idrandTranslator.idrandConverters.idris itself the evidence: the design always assumed several surfaces mapping onto one core.So A2ML is a typed core with surface projections, not a markup format with variants. A surface is legitimate exactly when it defines a translation into the core — which makes new surfaces additive rather than fragmenting, and is why leaving one unspecified is the actual problem.
Census, re-measured 2026-08-03
1,543
.a2mlfiles, classified by first non-comment non-blank line:manifest(---frontmatter)record([section])RECORD-DIALECT-SPEC.adocmarkup(@directive)SPEC.adoc§2The manifest dialect is 59% of the corpus and has no grammar.
grep -niE 'frontmatter|yaml'across both specs returns zero hits, andSPEC.adoc§2 has no key-value production at all.New finding: 27 of the 115 unclassified files open with
;or;;— Scheme/Lisp comment syntax. A small s-expression-shaped population exists that neither spec counts. That strengthens the family-type design: surfaces accrete whether or not a spec permits them.Why it blocks IANA
The registration (Revision 2, 2026-04-03) declares
application/vnd.a2mlwithcharsetas its only optional parameter. Submitting it would point IANA at a specification that does not describe 59% of its own corpus, and give a recipient no way to know which grammar a document follows.Worse — three other media-type forms are already written down across the repo, none matching the registration:
application/vnd.a2ml+texttext/a2mlapplication/vnd.{format}+a2mlThe fix — mostly already decided
RECORD-DIALECT-SPEC.adoc§14 recommends adialect=parameter on one family type rather than separate types, and explicitly declines to impose it alone. This document takes that up:MANIFEST-DIALECT-SPEC.adoc— the only substantial new writingSPEC.adocto family root: name the core canonical, name the dialects, definedialect=, state the translation ruledialect=to the registration and reconcile the three stale formsK9 supplies the precedent: it already registers an optional parameter —
security-level, defaultkennel, advisory on the wire and authoritative in the document.dialect=should follow that shape exactly.K9 has an independent blocker — found while checking that precedent
k9-svc/IANA-MEDIA-TYPE-APPLICATION.mdapplication/vnd.k9without a structured syntax suffixk9-svc/SPEC.adocline 34Standard ID | application/vnd.k9+nickelThe spec and the application disagree on the media type, on current
main. That's a live contradiction, not a design question, and it must be settled before K9 submits.Note also that the mandatory
K9!magic line is not valid Nickel, so a.k9.nclfile isn't parseable by a stock Nickel toolchain — which argues for dropping+nickel, since the suffix implies exactly that parseability.Verified, not asserted
Every path cited exists. The validator exemption is quoted from
a2ml/actions/validate/validate-a2ml.shlines 20–26 and 151–156; the "435 files … conform to a grammar that existed only by imitation" quote isRECORD-DIALECT-SPEC.adoclines 23–24; the three stale media-type strings were counted, not remembered.Two claims failed verification while drafting and were corrected rather than shipped: the validator path (it's under
a2ml/actions/validate/, not.githooks/), and the census figures.Deliberately not done
Writing the manifest grammar. That needs the corpus surveyed for the YAML constructs actually used first — a permissive spec written blind is much harder to narrow later than a narrow one is to widen.
🤖 Generated with Claude Code