docs: generate a browsable libosrm C++ API reference

README.md

@@ -30,6 +30,7 @@ Related [Project-OSRM](https://github.com/Project-OSRM) repositories:
 - [Hosted documentation](http://project-osrm.org)
 - [osrm-routed HTTP API documentation](docs/http.md)
 - [libosrm API documentation](docs/libosrm.md)
+- [libosrm C++ API reference](https://project-osrm.org/libosrm-api/) (generated from the public headers)
 
 ## Contact
 

docs/libosrm.md

@@ -2,6 +2,8 @@
 
 OSRM can be used as a library (libosrm) via C++ instead of using it through the HTTP interface and `osrm-routed`. This allows for fine-tuning OSRM and has much less overhead. Here is a quick introduction into how to use `libosrm` in the current version.
 
+A generated C++ reference for these types --- every class, member and signature, each linked to the line of the header it was read from --- is published at [project-osrm.org/libosrm-api](https://project-osrm.org/libosrm-api/). Regenerate it with `python tools/api-docs/build.py`.
+
 Take a look at the example code that lives in the [example directory](https://github.com/Project-OSRM/osrm-backend/tree/master/example). Here is all you ever wanted to know about `libosrm`, that is a short description of what the types do and where to find documentation on it:
 
 ## Important interface objects

tools/api-docs/Doxyfile

@@ -0,0 +1,51 @@
+# Doxygen configuration for the generated libosrm C++ API reference.
+#
+# Only ever run by tools/api-docs/build.py, from inside the throwaway tree that script
+# prepares under build/api-docs/src. XML only: Doxygen's own HTML output is not used, the
+# pages are rendered from this XML by Sourcey.
+#
+# INPUT is deliberately the set of headers docs/libosrm.md points embedders at, plus the
+# types those headers expose in their signatures. Widening it to include/ as a whole pulls
+# in the internal engine, extractor, partitioner and customizer trees, which are not part
+# of the libosrm contract.
+
+PROJECT_NAME           = "libosrm"
+PROJECT_BRIEF          = "The C++ library interface to the Open Source Routing Machine"
+
+INPUT                  = include/osrm/osrm.hpp \
+                         include/engine/api \
+                         include/engine/status.hpp \
+                         include/engine/engine_config.hpp \
+                         include/engine/approach.hpp \
+                         include/engine/bearing.hpp \
+                         include/engine/hint.hpp \
+                         include/util/coordinate.hpp \
+                         include/util/json_container.hpp \
+                         include/storage/storage_config.hpp
+RECURSIVE              = YES
+STRIP_FROM_PATH        = .
+FILE_PATTERNS          = *.hpp *.h
+
+EXTRACT_ALL            = NO
+EXTRACT_PRIVATE        = NO
+EXTRACT_STATIC         = NO
+EXTRACT_LOCAL_CLASSES  = NO
+HIDE_UNDOC_MEMBERS     = YES
+HIDE_UNDOC_CLASSES     = YES
+INTERNAL_DOCS          = NO
+
+JAVADOC_AUTOBRIEF      = YES
+QT_AUTOBRIEF           = YES
+MARKDOWN_SUPPORT       = YES
+AUTOLINK_SUPPORT       = YES
+BUILTIN_STL_SUPPORT    = YES
+
+GENERATE_XML           = YES
+GENERATE_HTML          = NO
+GENERATE_LATEX         = NO
+XML_OUTPUT             = ../xml
+XML_PROGRAMLISTING     = NO
+
+QUIET                  = YES
+WARN_IF_UNDOCUMENTED   = NO
+WARN_AS_ERROR          = NO

tools/api-docs/README.md

@@ -0,0 +1,42 @@
+# libosrm C++ API reference generator
+
+`docs/libosrm.md` has, since 2016, described the libosrm contract by telling readers where to
+find each type on GitHub. This directory turns those same headers into a browsable reference,
+published at <https://project-osrm.org/libosrm-api/>.
+
+```sh
+python tools/api-docs/build.py                       # -> build/api-docs/site
+python tools/api-docs/build.py --output /tmp/api     # anywhere else
+```
+
+Requirements: `doxygen` and `node` on `PATH`. Nothing is added to the C++ build, no dependency
+is added to the project, and no header is modified — the renderer is fetched on demand with
+`npx --yes sourcey@3.6.5` and everything intermediate is written under `build/`, which is
+already ignored.
+
+## What runs
+
+| Step | File | What it does |
+|---|---|---|
+| 1 | `promote_comments.py` | Copies the headers into `build/api-docs/src`, turning the `//` prose that already sits above a declaration into `///` so Doxygen picks it up. Line-preserving, so the source links stay exact. Conservative: file banners, tooling directives and commented-out code are left alone. |
+| 2 | `Doxyfile` | XML-only Doxygen run over the public headers. `EXTRACT_ALL = NO`, so an undocumented symbol stays out of the reference rather than showing up as a bare signature. |
+| 3 | `sourcey.config.template.ts` | Rendering config. `__COMMIT__` is substituted with `$GITHUB_SHA`, `--commit`, or `git rev-parse HEAD`, so every symbol links to `blob/<commit>/<file>#L<line>`. |
+
+## Scope
+
+`INPUT` in the `Doxyfile` is deliberately the set of headers `docs/libosrm.md` points at, plus
+the types those headers expose in their signatures:
+
+`include/osrm/osrm.hpp`, `include/engine/api/**`, `include/engine/{status,engine_config,approach,bearing,hint}.hpp`,
+`include/util/{coordinate,json_container}.hpp`, `include/storage/storage_config.hpp`.
+
+Pointing Doxygen at `include/` as a whole would pull in the extractor, partitioner, customizer
+and internal engine trees, which are not part of the library contract embedders code against.
+
+## Publishing
+
+The output is copied into the `libosrm-api/` directory of
+[`Project-OSRM/project-osrm.github.com`](https://github.com/Project-OSRM/project-osrm.github.com),
+minus `_og/` (GitHub Pages runs Jekyll on that repository and Jekyll drops leading-underscore
+paths; those files are only social-preview images). That repository already stores the built
+documentation for each release, so the reference lives alongside it.

tools/api-docs/build.py

@@ -0,0 +1,121 @@
+#!/usr/bin/env python3
+"""Generate the libosrm C++ API reference from the public headers.
+
+    python tools/api-docs/build.py --output build/api-docs/site
+
+Requires `doxygen` and `node` on PATH. The renderer is fetched with
+`npx --yes sourcey@3.6.5`, pinned to an exact version so the output is reproducible.
+
+The run is hermetic with respect to the working tree: the only thing it writes inside the
+repository is the `--work` directory (default `build/api-docs`, which is already ignored by
+the build/ rule in .gitignore). It never edits a header.
+
+Steps:
+  1. copy the Doxyfile's INPUT paths into work/src, promoting `//` prose to `///`
+     (see promote_comments.py -- line-preserving, so source links stay exact),
+  2. run Doxygen there to produce XML,
+  3. render the XML with Sourcey into --output.
+"""
+from __future__ import annotations
+
+import argparse
+import os
+import re
+import shutil
+import subprocess
+import sys
+
+HERE = os.path.dirname(os.path.abspath(__file__))
+REPO = os.path.dirname(os.path.dirname(HERE))
+SOURCEY = "sourcey@3.6.5"
+SOURCE_EXTS = {".h", ".hpp", ".hxx", ".ipp"}
+
+sys.path.insert(0, HERE)
+from promote_comments import promote  # noqa: E402
+
+
+def run(cmd, cwd=None):
+    print("$", " ".join(cmd), flush=True)
+    proc = subprocess.run(cmd, cwd=cwd, shell=False)
+    if proc.returncode != 0:
+        sys.exit(f"failed ({proc.returncode}): {' '.join(cmd)}")
+
+
+def doxyfile_inputs(path):
+    """Read the INPUT list out of the Doxyfile so it is declared in exactly one place."""
+    text = open(path, encoding="utf-8").read()
+    text = re.sub(r"\\\s*\n", " ", text)
+    for line in text.splitlines():
+        if line.strip().startswith("INPUT ") or line.strip().startswith("INPUT="):
+            return line.split("=", 1)[1].split()
+    sys.exit(f"no INPUT line in {path}")
+
+
+def stage_sources(inputs, dest):
+    copied = 0
+    for rel in inputs:
+        src = os.path.join(REPO, rel)
+        if os.path.isfile(src):
+            entries = [(os.path.dirname(rel), os.path.basename(rel))]
+        elif os.path.isdir(src):
+            entries = []
+            for base, _dirs, files in os.walk(src):
+                r = os.path.relpath(base, REPO)
+                entries += [(r, f) for f in files]
+        else:
+            sys.exit(f"Doxyfile INPUT path does not exist: {rel}")
+        for rel_dir, name in entries:
+            if os.path.splitext(name)[1] not in SOURCE_EXTS:
+                continue
+            text = open(os.path.join(REPO, rel_dir, name), encoding="utf-8",
+                        errors="replace").read()
+            out = promote(text)
+            assert out.count("\n") == text.count("\n"), name
+            os.makedirs(os.path.join(dest, rel_dir), exist_ok=True)
+            with open(os.path.join(dest, rel_dir, name), "w", encoding="utf-8",
+                      errors="replace", newline="") as fh:
+                fh.write(out)
+            copied += 1
+    return copied
+
+
+def main():
+    ap = argparse.ArgumentParser()
+    ap.add_argument("--output", default=os.path.join(REPO, "build", "api-docs", "site"))
+    ap.add_argument("--work", default=os.path.join(REPO, "build", "api-docs"))
+    ap.add_argument("--commit", default=os.environ.get("GITHUB_SHA"),
+                    help="commit to link source lines to; defaults to $GITHUB_SHA or HEAD")
+    args = ap.parse_args()
+
+    commit = args.commit or subprocess.run(
+        ["git", "-C", REPO, "rev-parse", "HEAD"], capture_output=True, text=True,
+        check=True).stdout.strip()
+    if not re.fullmatch(r"[0-9a-f]{40}", commit):
+        sys.exit(f"not a full commit sha: {commit!r}")
+
+    work = os.path.abspath(args.work)
+    src = os.path.join(work, "src")
+    shutil.rmtree(src, ignore_errors=True)
+    shutil.rmtree(os.path.join(work, "xml"), ignore_errors=True)
+    os.makedirs(src, exist_ok=True)
+
+    doxyfile = os.path.join(HERE, "Doxyfile")
+    count = stage_sources(doxyfile_inputs(doxyfile), src)
+    print(f"staged {count} headers from {commit}")
+    shutil.copy(doxyfile, os.path.join(src, "Doxyfile"))
+    run(["doxygen", "Doxyfile"], cwd=src)
+
+    template = open(os.path.join(HERE, "sourcey.config.template.ts"), encoding="utf-8").read()
+    with open(os.path.join(work, "sourcey.config.ts"), "w", encoding="utf-8",
+              newline="\n") as fh:
+        fh.write(template.replace("__COMMIT__", commit))
+
+    out = os.path.abspath(args.output)
+    shutil.rmtree(out, ignore_errors=True)
+    npx = "npx.cmd" if os.name == "nt" else "npx"
+    run([npx, "--yes", SOURCEY, "build", "-o", out], cwd=work)
+    print(f"\nlibosrm API reference written to {out}")
+
+
+if __name__ == "__main__":
+    main()

tools/api-docs/promote_comments.py

@@ -0,0 +1,85 @@
+"""Promote a libosrm header's plain `//` prose into Doxygen `///` doc comments.
+
+The public headers document themselves with ordinary `//` comments sitting directly above
+the declaration they describe. Doxygen ignores those, so a plain Doxygen run over the public
+headers yields signatures with no prose. This filter promotes a run of `//` lines to `///`
+**only when the run is immediately followed by a declaration**, so the prose that is already
+in the tree lands on the symbol it already describes. Nothing is invented, and nothing is
+edited in the repository: the promotion happens on a throwaway copy under the build
+directory.
+
+The transform is line-preserving -- every promoted line keeps its original line number -- so
+the source links Doxygen emits still point at the right line of the real header.
+
+Deliberately conservative. A run is left alone when it:
+  * is the file banner (the first comment block in the file, i.e. the BSD licence header),
+  * carries a tooling directive (clang-format / NOLINT / cppcheck / SPDX / @file),
+  * looks like commented-out code,
+  * is not immediately followed by a declaration.
+
+Usage:  python tools/api-docs/promote_comments.py <file>   # writes the result to stdout
+"""
+
+from __future__ import annotations
+
+import io, re, sys
+
+CODEISH = re.compile(
+    r"^\s*(?:#|}|\)|template\s*<|typedef\b|using\b|return\b|if\b|for\b|while\b|else\b)"
+    r"|[;{}]\s*$")
+DIRECTIVE = re.compile(r"clang-format|NOLINT|cppcheck|SPDX|coverity|codespell|@file", re.I)
+DECL = re.compile(
+    r"^\s*(?:QPDF_DLL\b|QPDF_DLL_CLASS\b|template\s*<|class\b|struct\b|enum\b|union\b|namespace\b"
+    r"|typedef\b|using\b|static\b|virtual\b|explicit\b|constexpr\b|inline\b|friend\b"
+    r"|[A-Za-z_~][A-Za-z0-9_:<>,\s\*&\[\]]*\s*\()")
+COMMENT = re.compile(r"^(\s*)//(?!/)(?!!)(.*)$")
+
+
+def promote(text: str) -> str:
+    lines = text.split("\n")
+    out = list(lines)
+    n = len(lines)
+    i = 0
+    first_block = True
+    while i < n:
+        m = COMMENT.match(lines[i])
+        if not m:
+            if lines[i].strip():
+                first_block = False
+            i += 1
+            continue
+        start = i
+        while i < n and COMMENT.match(lines[i]):
+            i += 1
+        end = i  # exclusive
+        block = lines[start:end]
+        nxt = lines[end] if end < n else ""
+
+        if first_block:                       # file banner / licence
+            first_block = False
+            continue
+        body = " ".join(COMMENT.match(b).group(2) for b in block)
+        if not body.strip():
+            continue
+        if DIRECTIVE.search(body):
+            continue
+        if any(CODEISH.search(COMMENT.match(b).group(2)) for b in block):
+            continue                          # commented-out code
+        if not nxt.strip() or not DECL.match(nxt):
+            continue                          # not attached to a declaration
+        for k in range(start, end):
+            g = COMMENT.match(lines[k])
+            out[k] = f"{g.group(1)}///{g.group(2)}"
+    return "\n".join(out)
+
+
+def main() -> None:
+    path = sys.argv[1]
+    with io.open(path, encoding="utf-8", errors="replace") as fh:
+        text = fh.read()
+    sys.stdout.reconfigure(encoding="utf-8", errors="replace", newline="\n")
+    sys.stdout.write(promote(text))
+
+
+if __name__ == "__main__":
+    main()

tools/api-docs/sourcey.config.template.ts

@@ -0,0 +1,41 @@
+// Rendering config for the generated libosrm C++ API reference.
+//
+// tools/api-docs/build.py copies this next to the Doxygen XML and substitutes __COMMIT__
+// with the commit being documented, so every symbol links to the exact file and line it was
+// read from.
+//
+// Exported as a plain object rather than through sourcey's `defineConfig` helper: the
+// renderer is invoked with `npx sourcey@3.6.5`, so there is no local node_modules for an
+// `import { defineConfig } from "sourcey"` to resolve against. `defineConfig` is an identity
+// helper that exists for editor types, and the schema below is what sourcey reads either way.
+export default ({
+  name: "libosrm C++ API",
+  repo: "https://github.com/Project-OSRM/osrm-backend",
+  editBranch: "__COMMIT__",
+  navigation: {
+    tabs: [
+      {
+        tab: "API Reference",
+        slug: "api",
+        doxygen: {
+          xml: "xml",
+          language: "cpp",
+          groups: false,
+          index: "flat",
+          sourceUrl:
+            "https://github.com/Project-OSRM/osrm-backend/blob/__COMMIT__/",
+        },
+      },
+    ],
+  },
+  navbar: {
+    links: [
+      { type: "github", href: "https://github.com/Project-OSRM/osrm-backend" },
+    ],
+  },
+  footer: {
+    links: [
+      { type: "github", href: "https://github.com/Project-OSRM/osrm-backend" },
+    ],
+  },
+});