Unify frontmatter under one AST node. (#25)

Author: samuel-williams-shopify

context/markdown-architecture.md

@@ -19,15 +19,15 @@ The Markdown serializer lives in `src/markdown/`. Keep its construct handlers an
 
 Ordinary `Code` nodes retain their opening fence in `fence: Option<CodeFence>` independently of source positions. Indented code and manually constructed or older deserialized code without metadata use `None`. Fence length describes the opening sequence even if the closer is longer or absent. Indentation is measured in columns relative to the container, excluding list and blockquote prefixes; tabs, including partially consumed container tabs, contribute their expanded width. Markdown serialization continues to choose ordinary code formatting from `MarkdownOptions`, so this metadata describes the source rather than the rendered output.
 
-The Rust helper also exposes frontmatter fences: generic nodes use their existing marker and opening length; legacy YAML/TOML use their fixed three-character, unindented delimiter. Inline code and other nodes return `None`. Keep fence information separate from language and info strings.
+The Rust helper also exposes frontmatter's stored marker and opening length, with zero indentation. Inline code and other nodes return `None`. Keep fence information separate from language and info strings.
 
 ## Frontmatter
 
-`Constructs::frontmatter` recognizes closed fences only at the beginning of a document, outside containers and without opening indentation. Untagged `---` and `+++` retain the inherited `Yaml` and `Toml` nodes. Tagged delimiter forms (including `---yaml`) and language-tagged backtick or tilde fences use `Frontmatter`.
+`Constructs::frontmatter` recognizes closed fences only at the beginning of a document, outside containers and without opening indentation. Every form produces `Node::Frontmatter`: untagged `---` and `+++`, tagged delimiter forms (including `---yaml`), and language-tagged backtick or tilde fences.
 
-The generic node follows Markly/cmarkly's format-agnostic model: `info` is the entire opaque info string, trimmed only at its outer spaces and tabs; `value` is the raw body, including its trailing line ending. Neither is decoded or interpreted. `language()` and `Node::code_language()` return the first word, while `Node::code_info()` returns the full info string. Opening and closing fence lengths are retained. Closing code fences follow the usual marker, minimum length, and indentation rules; the Rust extension requires a closing fence for every form.
+The node follows Markly/cmarkly's format-agnostic model: `info` is the entire opaque info string, trimmed only at its outer spaces and tabs; `value` is the raw body, including its trailing line ending. `language()` and `Node::code_language()` return the first word of the info string, defaulting to `yaml` for untagged dash delimiters and `toml` for untagged plus delimiters. `Node::code_info()` returns the authored info string, which is empty when the language is implicit. Callers decode the body using the appropriate language parser. Opening and closing fence lengths are retained. Closing code fences follow the usual marker, minimum length, and indentation rules; the Rust extension requires a closing fence for every form.
 
-Both HTML paths omit frontmatter. Markdown serialization preserves info and body while normalizing fence spacing. It lengthens code fences when edited content could close them, and switches conflicting dash/plus delimiters to tildes. Edited bodies without a trailing line ending gain one before the closing fence. Keep these safety checks in the Markdown handler, separate from the public AST type.
+Both HTML paths omit frontmatter. All frontmatter uses one Markdown serialization handler, which preserves info and body while normalizing fence spacing. It lengthens code fences when edited content could close them, and switches conflicting dash/plus delimiters to tildes, making an implicit language explicit in the replacement fence. Edited bodies without a trailing line ending gain one before the closing fence. Keep these safety checks in the Markdown handler, separate from the public AST type.
 
 ## Tests and generated data
 

readme.md

@@ -110,9 +110,9 @@ if let Node::Frontmatter(frontmatter) = &document.children().unwrap()[0] {
 }
 ````
 
-Tagged forms such as `--- json` and `---yaml` use the same node. Untagged `---` and `+++` keep their existing `Yaml` and `Toml` nodes. Fences without a language, fences later in a document, and unclosed fences remain ordinary Markdown.
+All frontmatter uses `Node::Frontmatter`. Untagged `---` and `+++` imply YAML and TOML respectively, with an empty `info` string. Tagged forms such as `--- json` and `---yaml` supply an explicit language hint. `language()` and `Node::code_language()` return the effective language; `Node::code_info()` returns the authored info string. Backtick/tilde fences without a language, fences later in a document, and unclosed fences remain ordinary Markdown.
 
-Markdown serialization retains the stored info string and body while normalizing spacing around the info string. It can lengthen fences or change their marker when edited content would close them prematurely, and adds a line ending before the closing fence when an edited body needs one. Round trips preserve content and metadata while allowing these formatting adjustments.
+Markdown serialization retains the stored info string and body while normalizing spacing around the info string. It can lengthen fences or change their marker when edited content would close them prematurely, and adds a line ending before the closing fence when an edited body needs one. When changing an untagged dash/plus delimiter to a code fence, it writes the implied language explicitly. Round trips preserve the body and effective language while allowing these formatting adjustments.
 
 ## Releasing
 

releases.md

@@ -2,6 +2,9 @@
 
 ## Unreleased
 
+- Unify all frontmatter under `Node::Frontmatter`, replacing `Node::Yaml`, `Node::Toml`, and their structs. Update Rust matches and struct literals; serialized ASTs now use `type: "frontmatter"` with info and fence fields.
+- Return YAML/TOML defaults from `Frontmatter::language()` and `Node::code_language()` for untagged dash/plus delimiters, while keeping the authored info string empty.
+- Retain the trailing body line ending for untagged YAML/TOML, matching tagged frontmatter, and serialize all forms through the same handler. Edited bodies that require a different fence retain their effective language.
 - Clarify frontmatter placement and serialization, and provide runnable AST renderer examples.
 
 ## v0.6.0

src/configuration.rs

@@ -114,10 +114,11 @@ pub struct Constructs {
     /// Frontmatter with an unindented opening fence on the first document line,
     /// outside list items and blockquotes, and a matching closing fence.
     ///
-    /// Untagged `---` and `+++` produce YAML and TOML nodes. A format hint after
-    /// either delimiter, or a language-tagged backtick/tilde code fence, produces
-    /// a language-agnostic [`Frontmatter`][crate::mdast::Frontmatter] node. The
-    /// parser preserves its raw body and full info string; HTML omits it.
+    /// All forms produce a [`Frontmatter`][crate::mdast::Frontmatter] node.
+    /// Untagged `---` and `+++` imply YAML and TOML. A format hint after either
+    /// delimiter, or a language-tagged backtick/tilde code fence, supplies an
+    /// explicit language. The parser preserves the raw body and authored info
+    /// string; HTML omits the frontmatter.
     ///
     /// ````markdown
     /// > | ---

src/construct/frontmatter.rs

@@ -25,11 +25,11 @@
 //!
 //! Frontmatter can only occur once and requires a closing fence followed by a
 //! line ending or end of file. Untagged code fences remain ordinary code blocks.
-//! Untagged `---` and `+++` retain their YAML and TOML AST nodes. Tagged forms,
-//! such as `--- json`, `---yaml`, `+++ toml`, or a fenced code block with a
-//! language, produce a format-agnostic [`Frontmatter`][crate::mdast::Frontmatter].
-//! The parser preserves the opaque info string and raw body without interpreting
-//! or decoding them, following Socketry's Markly/cmarkly frontmatter model.
+//! All forms produce a format-agnostic [`Frontmatter`][crate::mdast::Frontmatter].
+//! Untagged `---` and `+++` imply YAML and TOML. Tagged forms, such as `--- json`,
+//! `---yaml`, `+++ toml`, or a fenced code block with a language, supply an
+//! explicit format hint. The parser preserves the opaque info string and raw
+//! body, following Socketry's Markly/cmarkly frontmatter model.
 //!
 //! ## Extension
 //!

src/markdown/handle/frontmatter.rs

@@ -24,20 +24,22 @@ impl Handle for Frontmatter {
         _parent: Option<&Node>,
         _node: &Node,
     ) -> Result<String, Message> {
+        // Validate the fence and the authored format hint.
+        let mut info = self.info.trim_matches([' ', '\t']);
         if !matches!(self.fence, '`' | '~' | '-' | '+')
-            || self.info.trim_matches([' ', '\t']).is_empty()
+            || (matches!(self.fence, '`' | '~') && info.is_empty())
             || self.info.contains(['\r', '\n'])
             || (self.fence == '`' && self.info.contains('`'))
         {
             return Err(Message {
                 place: None,
-                reason:
-                    "Frontmatter requires a valid fence and a nonempty, single-line info string"
-                        .into(),
+                reason: "Frontmatter requires a valid fence and a single-line info string; code fences require a language hint".into(),
                 rule_id: Box::new("invalid-frontmatter".into()),
                 source: Box::new("mdast-util-to-markdown".into()),
             });
         }
+
+        // Choose fences that cannot be closed by the edited body.
         let mut marker = self.fence;
         if matches!(marker, '-' | '+') {
             let delimiter = marker.to_string().repeat(3);
@@ -58,12 +60,21 @@ impl Handle for Frontmatter {
         };
         let opening = marker.to_string().repeat(opening_length);
         let closing = marker.to_string().repeat(closing_length);
-        let info = self.info.trim_matches([' ', '\t']);
-        let separator = if matches!(marker, '-' | '+') || info.starts_with(marker) {
-            " "
-        } else {
-            ""
-        };
+
+        // Carry the implicit language into a replacement code fence.
+        if marker != self.fence && self.info.split_ascii_whitespace().next().is_none() {
+            info = self
+                .language()
+                .expect("dash and plus fences imply a language");
+        }
+
+        // Render the authored body between its opening and closing fences.
+        let separator =
+            if !info.is_empty() && (matches!(marker, '-' | '+') || info.starts_with(marker)) {
+                " "
+            } else {
+                ""
+            };
         let mut output = format!("{opening}{separator}{}\n", info);
         output.push_str(&self.value);
         if !self.value.is_empty() && !self.value.ends_with(['\r', '\n']) {

src/markdown/state.rs

@@ -370,9 +370,7 @@ impl<'a> State<'a> {
             Node::Table(table) => self.render_table(table),
             Node::TableRow(row) => self.render_table_row(&row.children),
             Node::TableCell(_) => self.container_phrasing(node, info),
-            Node::Toml(toml) => Ok(render_frontmatter("+++", &toml.value)),
             Node::Frontmatter(frontmatter) => frontmatter.handle(self, info, parent, node),
-            Node::Yaml(yaml) => Ok(render_frontmatter("---", &yaml.value)),
         }
     }
 
@@ -700,16 +698,6 @@ pub(crate) fn is_phrasing(node: &Node) -> bool {
     )
 }
 
-fn render_frontmatter(marker: &str, value: &str) -> String {
-    let mut output = format!("{}\n", marker);
-    output.push_str(value);
-    if !value.ends_with('\n') && !value.ends_with('\r') {
-        output.push('\n');
-    }
-    output.push_str(marker);
-    output
-}
-
 fn render_jsx_attribute(attribute: &AttributeContent) -> String {
     match attribute {
         AttributeContent::Expression(expression) => format!("{{{}}}", expression.value),

src/markdown/state/tests.rs

@@ -1,7 +1,7 @@
 // Released under the MIT License.
 // Copyright, 2026, by Samuel Williams.
 
-use super::{escape_table_pipes, has_matching_ticks, render_frontmatter, Info, Join, State};
+use super::{escape_table_pipes, has_matching_ticks, Info, Join, State};
 use crate::{
     markdown::{r#unsafe::Unsafe, util::safe::SafeConfig, Options},
     mdast::{
@@ -223,12 +223,6 @@ fn detects_matching_and_nonmatching_code_fences() {
     assert!(!has_matching_ticks(b"``", 0, 1));
 }
 
-#[test]
-fn preserves_existing_line_endings_in_frontmatter() {
-    assert_eq!(render_frontmatter("---", "title\n"), "---\ntitle\n---");
-    assert_eq!(render_frontmatter("+++", "title\r"), "+++\ntitle\r+++");
-}
-
 #[test]
 fn joins_flow_nodes_with_each_supported_separator() {
     let cases = [

src/markdown/util/format_heading_as_setext.rs

@@ -40,9 +40,7 @@ fn include_literal_with_line_break(node: &Node, regex: &Regex) -> bool {
         Node::MdxTextExpression(x) => regex.is_match(&x.value),
         Node::MdxjsEsm(x) => regex.is_match(&x.value),
         Node::Text(x) => regex.is_match(&x.value),
-        Node::Toml(x) => regex.is_match(&x.value),
         Node::Frontmatter(x) => regex.is_match(&x.value),
-        Node::Yaml(x) => regex.is_match(&x.value),
         // Anything else.
         _ => {
             if let Some(children) = node.children() {
@@ -70,7 +68,7 @@ mod tests {
         markdown::{state::State, Options},
         mdast::{
             Break, Code, Frontmatter, Heading, Html, Image, InlineCode, InlineMath, Math,
-            MdxFlowExpression, MdxTextExpression, MdxjsEsm, Node, Text, Toml, Yaml,
+            MdxFlowExpression, MdxTextExpression, MdxjsEsm, Node, Text,
         },
     };
     use alloc::string::String;
@@ -155,14 +153,6 @@ mod tests {
                 closing_fence_length: 3,
                 position: None,
             }),
-            Node::Toml(Toml {
-                value: literal("a\nb"),
-                position: None,
-            }),
-            Node::Yaml(Yaml {
-                value: literal("a\nb"),
-                position: None,
-            }),
         ] {
             assert!(
                 format_heading_as_setext(&heading(1, child.clone()), &state),