Skip to content
Closed
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
6 changes: 4 additions & 2 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -32,7 +32,7 @@ The dialect deviates from CommonMark for three reasons:
- Panels: `::: callout-note` (also `tip`, `important`, `warning`, `caution`) creates a callout. Add `collapse="true"` or `collapse="false"` for initially closed/open disclosure, or use `::: details` without a callout kind. The first heading is the panel title, not a document heading. These normalize to `div[data-panel]` with an optional `header` title and independent `data-callout` / `data-disclosure` properties. Ordinary divs stay ordinary.
- Raw passthrough: a Pandoc-style raw attribute names the payload's format. Use exactly `{=name}` as a fenced code block's info string or immediately after inline code. Both forms render as an inert `<script type="application/vnd.mdhtml.raw" data-format="name">`. Payload text stays literal unless it contains an HTML script-data hazard. [The dialect specification](docs/DIALECT.md#converter-specific-raw-data) defines the encoding rule.
- Template tokens: Jinja, Mustache, or other configured delimiters. Token recognition is opt-in. Tokens become semantic operations in inert HTML template elements. Overlapping opening delimiters use the longest match. Optional balanced scanning handles nested expressions.
- Cross-references: Quarto-style bracketed references to identified elements. `[@sec-pay]` renders as `<a data-ref href="#sec-pay"></a>`. Each converter resolves the reference for its output format. `[-@sec-pay]` adds the independent `bare` token. `[Clause @sec-pay]` supplies override text. `[@sec-a; @sec-b]` groups references in a `span` marked with `data-refs`. A trailing `{ref=page}` selects the `page` variant. The parser never resolves numbers or checks that targets exist.
- Cross-references: Quarto-style bracketed references to identified elements. `[@sec-pay]` renders as `<a data-ref href="#sec-pay"></a>`. Each converter resolves the reference for its output format. `[-@sec-pay]` adds the independent `bare` token. `[Clause @sec-pay]` supplies override text. `[@sec-a; @sec-b]` groups references in a `span` marked with `data-refs`. A trailing `{ref=page}` selects the `page` variant. The parser never resolves numbers or checks that targets exist. A reference to a term of a definition list inside a `::: {.references}` container is a citation, rendered by [its own rules](docs/DIALECT.md#citations).
- Table captions and figures: place `: caption {attrs}` immediately after the table's last row, with no blank line. This uses Quarto's caption format but requires the caption to follow the table. The attributes apply to the table. With `implicit_figures=True`, a paragraph containing only one image becomes a `<figure>`. Its alt text becomes the `<figcaption>`. The image's id and classes move to the figure. The image has `alt=""` to avoid announcing the caption twice through assistive technology.
- Inline footnotes: pandoc-style `^[an inline note]`, numbered together with `[^id]` references.

Expand Down Expand Up @@ -494,6 +494,8 @@ In `resolve` mode, cross-references become links with resolved text. `[@sec-pay]

`reftypes=dict(exh=('Exhibit', 'Exhibits'))` adds prefix words to the built-in `sec`, `fig`, `tbl`, and `eq` types. Missing targets, unknown tokens, and unknown types that need a prefix raise errors. The Word-only `page` and `rel` variants display the full number.

Citations are references to the terms of a definition list inside a `::: {.references}` container. Each term is the "Author, year" label with the citation key as its id. `[@dai2015]` renders "(Dai and Le, 2015)", `[@a; @b]` renders "(A; B)", `[@dai2015]{ref=text}` renders "Dai and Le (2015)", and `[-@dai2015]` renders the label alone, for an author who writes the parentheses. `cite='numeric'` numbers the entries in list order instead: "[1]", "[1, 3]", "Dai and Le [1]", with each term relabelled to its number. Every baked citation sits in `<span class="cite">`.

Two other modes support previews and drafts:

- `refs='ids'` displays each target id as a link, such as `<a href="#sec-pay" class="xref">sec-pay</a>`. Authored text remains as a prefix. Variants are ignored. This mode does not validate references. Headings are numbered when the argument or frontmatter supplies a scheme. It never numbers headings automatically. Captions remain as authored. Use it when targets can be outside the fragment being previewed, where numbering would otherwise restart for each fragment.
Expand Down Expand Up @@ -570,7 +572,7 @@ portable = md2gfm(markdown, number_headings='legal')

The GFM conversion applies these rules:

- Cross-references become plain text, such as "See Section 1.(a)". They use the same `reftypes`, `number_headings`, and automatic-numbering rules as `mdhtml2html`.
- Cross-references become plain text, such as "See Section 1.(a)". They use the same `reftypes`, `number_headings`, and automatic-numbering rules as `mdhtml2html`. Citations render as in HTML, in the `cite` style.
- Heading numbers become part of the heading text. Heading attribute lists are removed.
- A `: caption` line immediately after a table becomes a "Table 1: caption" paragraph.
- With `implicit_figures=True`, an image-only paragraph receives a "Figure 1: alt" caption paragraph.
Expand Down
35 changes: 32 additions & 3 deletions docs/DIALECT.md
Original file line number Diff line number Diff line change
Expand Up @@ -399,12 +399,41 @@ The `md` `{ref=...}` key is consumed when lowering a reference. On any other ele

The parser does not resolve numbers or require targets to exist. Converters report an unresolved target as a conversion error when their output format supports live references.

Reference targets are the id-bearing headings, paragraphs, figures, tables, display equations, spans, and definition terms. Headings resolve to a heading number ("Section 1"), figures, tables and equations to a caption number ("Table 2", "Equation 3"), and paragraphs only through `{ref=text}`. Spans and definition terms resolve to their own text with no prefix word, so `the [@def-term] period` reads as running prose citing the defined term, and a rendering variant that needs a number (`leaf`, `rel`) is an error for them. Converters with a link mechanism link the cited text back to its definition site; `md2gfm` renders the text alone.
Reference targets are the id-bearing headings, paragraphs, figures, tables, display equations, spans, and definition terms. A definition term inside a `.references` container is a citation target, rendered by the rules in the citations section below. Headings resolve to a heading number ("Section 1"), figures, tables and equations to a caption number ("Table 2", "Equation 3"), and paragraphs only through `{ref=text}`. Spans and definition terms resolve to their own text with no prefix word, so `the [@def-term] period` reads as running prose citing the defined term, and a rendering variant that needs a number (`leaf`, `rel`) is an error for them. Converters with a link mechanism link the cited text back to its definition site; `md2gfm` renders the text alone.

The shipped exporters lower references from one shared vocabulary (`mdhtml.export`) at three levels of liveness: `mdhtml2docx` bakes REF fields Word keeps live, `mdhtml2html` bakes links with computed text, and `md2gfm` bakes plain text. Prefix words come from `REFTYPES` (`sec`, `fig`, `tbl`; extended per call with `reftypes=`) and heading numbering from `SCHEMES` (`'legal'`, `'decimal'`, or a `{lvlText: numFmt}` dict). Automatic numbering numbers headings exactly when some reference needs a heading number. For HTML, GFM, and Typst, `number_headings=None` inherits the input's metadata setting, falling back to automatic numbering; `False` disables heading numbering, and numeric references to unnumbered headings are errors. Scheme level 0 is the h1 document title, with an empty lvlText: it shows no number, and bumping it restarts every level below, so `%2` is the h2 counter and each document in a file that opens with its own h1 numbers from 1; a title is cited with `{ref=text}`, never by number.

`mdhtml2html` also offers `refs='ids'` for live-preview contexts where targets may sit outside the fragment: each reference bakes as a working link showing its target id (class `xref`), with no registry, numbering, or failure modes - and captions render as authored, since per-fragment numbers would lie. `id_prefix` namespaces the fragment's ids against a host page (the authored id kept in `data-id`), and `fn_salt` adds a further prefix to the `fn-*`/`fnref-*` footnote namespace only, keeping footnote pairs distinct across fragments that share one `id_prefix`.

### Citations

A bibliography is a definition list inside a `::: {.references}` container. Each term is the citation label, written as "Author, year", and carries the citation key as its id. The definition is the entry.

```markdown
::: {.references}
Dai and Le, 2015 {#dai2015}
: Andrew M. Dai and Quoc V. Le. 2015. [Semi-supervised Sequence Learning](http://arxiv.org/abs/1511.01432). *Advances in Neural Information Processing Systems (NIPS '15)*.
:::
```

```html
<div class="references"><dl><dt id="dai2015">Dai and Le, 2015</dt><dd>Andrew M. Dai and Quoc V. Le. 2015. <a href="http://arxiv.org/abs/1511.01432">Semi-supervised Sequence Learning</a>. <em>Advances in Neural Information Processing Systems (NIPS '15)</em>.</dd></dl></div>
```

A reference to such a term is a citation. It uses the cross-reference grammar and carrier unchanged: the `references` class on the container makes its terms citation targets, and the parser emits nothing new. Converters render a citation by these rules instead of the term-text rule:

```markdown
[@dai2015], [see @dai2015], and [@mou2016; @dai2015].
[@dai2015]{ref=text} and [-@dai2015].
```

- The default form is parenthetical: "(Dai and Le, 2015)". Prefix text goes inside the parentheses: "(see Dai and Le, 2015)". A group renders inside one pair of parentheses with "; " between entries: "(Mou et al., 2016; Dai and Le, 2015)".
- `text` renders the textual form, "Dai and Le (2015)". The label splits at its last ", " into author and year. A label with no ", " renders as written in every form.
- `bare` renders the label with no parentheses, "Dai and Le, 2015", for an author who writes the parentheses and any prefix text themselves.
- `page`, `leaf`, and `rel` are errors for a citation: an entry has no page or outline number.

A converter may offer a numeric style, which numbers the entries in list order. `mdhtml2html(cite='numeric')` renders the default form as "[1]", a group as "[1, 3]", the textual form as "Dai and Le [1]", and the bare form as "[1]", and relabels each term with its number. `md2gfm` and `mdhtml2typst` take the same `cite` option. The HTML exporter wraps every citation in `<span class="cite">` with its links inside, so a page can attach a hover card from the entry each link targets.

## Raw HTML and md in HTML

Raw HTML participates in the final whole-fragment HTML5 parse. There is no separate balance operation.
Expand Down Expand Up @@ -599,15 +628,15 @@ Callbacks are not separately parsed or restricted to the source node's inline/bl

Beyond the element mapping above, a few structural patterns carry a *meaning* every conforming converter must honor — the in-package `mdhtml2html`, `mdhtml2md`, and `mdhtml2typst` exporters and external `mdhtml2*` packages alike. These rules bind the interpretation, not the medium: each converter renders the meaning as its format best can, and each states its degradation where the format lacks the feature.

- **Cross-references.** An `a` with `data-ref` (or a `span` with `data-refs` grouping several) is a symbolic reference to be resolved and rendered per the captions and cross-references section; a converter never emits the empty carrier unresolved.
- **Cross-references.** An `a` with `data-ref` (or a `span` with `data-refs` grouping several) is a symbolic reference to be resolved and rendered per the captions and cross-references section; a converter never emits the empty carrier unresolved. A reference whose target is a term in a `.references` container is a citation and renders by the citations section's rules.
- **Raw data.** A raw-data `script` carrier addressed to the converter's own format is decoded and spliced; payloads for other formats are dropped (carried opaquely, never rendered as text), per the converter-specific raw data section.
- **Custom elements** without a native rendering are transparent wrappers: render the children, drop the tag.
- **Panels.** `div[data-panel]` carries the title/body structure and independent callout/disclosure properties defined above. Titles are labels, not headings. Formats without folding always render the complete body.
- **Table widths.** `colwidths` and `width` on a table are layout requests the HTML exporter honors (`colgroup`, inline style width); formats that own their table layout (docx, typst) may ignore them.
- **Range markers.** Template `section`/`inverted` and `end` operations are paired siblings around the content they control. Converters render unfilled range instructions visibly when appropriate; the content between markers remains normal flow content.
- **Active-code carriers.** A `text/<lang>-block` script is template code, not content: dropped from final documents by default, echoable as code where an audit register wants it, and never executed by any converter (`instantiate` alone executes).

The class words with assigned converter behavior — `math` for math carriers and `footnotes` on the footnote `section` — are reserved by this section. `details` and `callout-*` are authoring aliases normalized to panels by the `md` importer, not class-based export rules. Other class words are inert data for styling.
The class words with assigned converter behavior — `math` for math carriers, `footnotes` on the footnote `section`, and `references` on a bibliography's container — are reserved by this section. `details` and `callout-*` are authoring aliases normalized to panels by the `md` importer, not class-based export rules. Other class words are inert data for styling.

## Warnings

Expand Down
30 changes: 26 additions & 4 deletions py/src/lib.rs
Original file line number Diff line number Diff line change
Expand Up @@ -404,6 +404,21 @@ impl Resolver {

fn set_capnum(&mut self, id: &str, label: &str, n: u32) { self.inner.set_capnum(id, label, n); }

/// Record a bibliography entry's number, for the numeric citation style.
fn set_citenum(&mut self, id: &str, n: u32) { self.inner.set_citenum(id, n); }

/// How citations render: `'author-year'` (the default) or `'numeric'`.
fn set_cite_style(&mut self, style: &str) -> PyResult<()> { self.inner.set_cite_style(cite_style(style)?); Ok(()) }

/// Is `tgt` a bibliography entry?
fn is_cite(&self, tgt: &str) -> bool { self.inner.is_cite(tgt) }

/// The `(open, close)` text an exporter puts around a citation, or a group of them.
fn cite_wrap(&self, tokens: HashSet<String>) -> (String, String) { let (o, c) = self.inner.cite_wrap(&tokens); (o.to_string(), c.to_string()) }

/// The separator for a group of citations, or None when any target is not one.
fn cite_sep(&self, tgts: Vec<String>) -> Option<String> { self.inner.cite_sep(&tgts).map(str::to_string) }

/// Raise for a reference to an unknown target id.
fn check(&self, tgt: &str) -> PyResult<()> { self.inner.check(tgt).map_err(vr) }

Expand All @@ -418,6 +433,13 @@ impl Resolver {
}
}

/// A `cite=` option value as a `CiteStyle`.
fn cite_style(s: &str) -> PyResult<resolve::CiteStyle> {
match s {
"author-year" => Ok(resolve::CiteStyle::AuthorYear), "numeric" => Ok(resolve::CiteStyle::Numeric), _ => Err(vr(format!("unknown cite style {s:?}")))
}
}

#[pymodule]
fn _native(m: &Bound<'_, PyModule>) -> PyResult<()> {
m.add_function(wrap_pyfunction!(md2mdhtml, m)?)?;
Expand Down Expand Up @@ -790,11 +812,11 @@ fn attr_node<'py>(py: Python<'py>, attrs: &Attr) -> PyResult<Bound<'py, PyDict>>
// ---------------------------------------------------------------------------

#[pyfunction]
#[pyo3(signature = (src, reftypes, number_headings, hl, toc, refs, id_prefix, fn_salt, hl_lang, code_wrap, hl_fn, auto_ids, gh_ids))]
#[pyo3(signature = (src, reftypes, number_headings, hl, toc, refs, id_prefix, fn_salt, hl_lang, code_wrap, hl_fn, auto_ids, gh_ids, cite))]
fn export_html(
py: Python<'_>, src: &str, reftypes: Option<HashMap<String, (String, String)>>, number_headings: Option<&Bound<'_, PyAny>>, hl: Option<&str>, toc: bool,
refs: &str, id_prefix: &str, fn_salt: &str, hl_lang: Option<Py<PyAny>>, code_wrap: Option<Py<PyAny>>, hl_fn: Option<Py<PyAny>>, auto_ids: bool, gh_ids: bool
) -> PyResult<(String, Vec<String>)> {
refs: &str, id_prefix: &str, fn_salt: &str, hl_lang: Option<Py<PyAny>>, code_wrap: Option<Py<PyAny>>, hl_fn: Option<Py<PyAny>>, auto_ids: bool,
gh_ids: bool, cite: &str) -> PyResult<(String, Vec<String>)> {
use mdhtml::export_html::{HlMode, HtmlExportOptions, NumberHeadings, RefsMode};
let number_headings = match number_headings {
None => None,
Expand Down Expand Up @@ -829,7 +851,7 @@ fn export_html(
}, toc, refs: match refs {
"ids" => RefsMode::Ids, "lenient" => RefsMode::Lenient, _ => RefsMode::Resolve
}, id_prefix: id_prefix.to_string(), fn_salt: fn_salt.to_string(), hl_lang: hl_lang_c.as_ref().map(|c| c as _),
code_wrap: code_wrap_c.as_ref().map(|c| c as _), hl_fn: hl_fn_c.as_ref().map(|c| c as _), auto_ids, gh_ids };
code_wrap: code_wrap_c.as_ref().map(|c| c as _), hl_fn: hl_fn_c.as_ref().map(|c| c as _), auto_ids, gh_ids, cite: cite_style(cite)? };
let result = if hl_lang.is_none() && code_wrap.is_none() && hl_fn.is_none() { py.detach(|| mdhtml::export_html::export_html(src, &opts)) }
else { mdhtml::export_html::export_html(src, &opts) };
if let Some(e) = hook_err.into_inner().unwrap() { return Err(e); }
Expand Down
6 changes: 4 additions & 2 deletions python/mdhtml/export.py
Original file line number Diff line number Diff line change
Expand Up @@ -94,7 +94,7 @@ def go(text, lang):


def mdhtml2html(src, dest=None, reftypes: dict | None = None, number_headings=None, hl: str | None = "spans", auto_ids: bool = True,
toc: bool = False, refs: str = "resolve", id_prefix: str = "", fn_salt: str = "", hl_lang=None, code_wrap=None, gh_ids: bool = False) -> Html:
toc: bool = False, refs: str = "resolve", id_prefix: str = "", fn_salt: str = "", hl_lang=None, code_wrap=None, gh_ids: bool = False, cite: str = "author-year") -> Html:
"""Lower MDHTML (a string or DocumentFragment; never mutated) to finished HTML: cross-references
baked as links, headings, captions and equations numbered, `{=html}` raw data spliced, `colwidths` lowered,
and code highlighted. A `div[data-panel]` with open/closed disclosure lowers to `<details>`,
Expand Down Expand Up @@ -123,12 +123,14 @@ def mdhtml2html(src, dest=None, reftypes: dict | None = None, number_headings=No
without it, code blocks render plain and a warning reports it.
`number_headings=None` inherits the input's metadata setting, else numbers automatically when
a reference needs it. `False` disables heading numbering, including automatic numbering.
`cite` chooses how citations (references to a `.references` list's terms) render: `'author-year'`, the
default, or `'numeric'`, which numbers the entries in list order and relabels them.
Returns an `Html` str carrying `.warnings`; `dest` also writes it to a file."""
if refs not in ("resolve", "ids", "lenient"): raise ValueError(f"unknown refs mode {refs!r}")
number_headings = _headnums(getattr(src, 'meta', {}), number_headings)
if not isinstance(src, str): src = src.to_html()
hl_fn = None if hl is None else _hl_fn(hl)
out, warnings = _export_html(src, reftypes, number_headings, hl, toc, refs, id_prefix, fn_salt, hl_lang, code_wrap, hl_fn, auto_ids, gh_ids)
out, warnings = _export_html(src, reftypes, number_headings, hl, toc, refs, id_prefix, fn_salt, hl_lang, code_wrap, hl_fn, auto_ids, gh_ids, cite)
res = Html(out, warnings)
if dest is not None: Path(dest).write_text(res, encoding="utf-8")
return res
Loading
Loading