diff --git a/CONTRIBUTING.md b/CONTRIBUTING.md index e8dfaf9..7246316 100644 --- a/CONTRIBUTING.md +++ b/CONTRIBUTING.md @@ -19,11 +19,17 @@ You need uv and just. ```bash just bake # render the working tree into $TMPDIR/python-template-bake -just test # bake both publish_to_pypi variants and, in each, run +just test # bake twice (the yes/no questions answered yes, then no) and, in each, run # quality-check, build-docs, every pre-commit hook, dist, # and the tag/dev version checks ``` +Before the checks, `just test` copies `fixtures/` into the baked project: +`always/` (a function whose docstring draws a figure, an `.rst` page with IPython +blocks) and, in the bake with `docs_notebooks`, `notebooks/` (a Markdown page with +a code cell, a notebook). `build-docs` runs them, and `just test` checks that +their output reaches the HTML. + `just test` is what the Template workflow runs on every pull request. pre-commit needs git 2.31 or newer. diff --git a/README.md b/README.md index 9a71b29..ee4a5c3 100644 --- a/README.md +++ b/README.md @@ -38,6 +38,7 @@ git-style markers) and commit it like any other change. | `project_short_description` | — | One line | | `python_min` | 3.11 | Oldest supported Python; CI tests it through 3.14 | | `publish_to_pypi` | yes | Adds trusted publishing to PyPI on release | +| `docs_notebooks` | no | Runs notebooks, and code cells in Markdown pages, when the docs build ([myst-nb](https://myst-nb.readthedocs.io)); adds Jupyter to the dev tools | | `test_os` | Linux, macOS, Windows | Where the tests run; Linux is required (coverage uploads from it) | | `copyright_year` | 2026 | LICENSE | @@ -103,7 +104,7 @@ first time is by hand; `copier update` takes over from then on. | Types | [pyrefly](https://pyrefly.org) | | Tests | pytest 9 (strict, warnings are errors, `network`/`slow` markers), parallel with pytest-xdist, pytest-cov, Codecov | | Docstrings | NumPy style, [docstr-coverage](https://github.com/HunterMcGushion/docstr_coverage) ≥ 95% | -| Docs | Sphinx + PyData theme, pandas-style API pages (a page per class, with tables of its attributes and methods, and a page per member), copy buttons, live preview (`just docs-serve`), built with warnings as errors; versioned on GitHub Pages (`dev/`, one folder per release, `stable/`) with a version dropdown | +| Docs | Sphinx + PyData theme, pandas-style API pages (a page per class, with tables of its attributes and methods, and a page per member), copy buttons, examples that run when the docs build (matplotlib's `.. plot::` in docstrings, IPython's `.. ipython::` in `.rst` pages, and with `docs_notebooks`, notebooks and code cells in Markdown pages), live preview (`just docs-serve`), built with warnings as errors; versioned on GitHub Pages (`dev/`, one folder per release, `stable/`) with a version dropdown | | pre-commit | pre-commit-hooks, rST checks, validate-pyproject, [zizmor](https://docs.zizmor.sh); `uv lock`, ruff and pyrefly run from `.venv`, so `uv.lock` pins their versions | | Changelog | Keep a Changelog, hand-edited; `just changelog` drafts entries from Conventional Commits ([git-cliff](https://git-cliff.org)) | | CI | Tests (the chosen operating systems × all supported Pythons), Code Quality, Documentation, Publish (tag push → PyPI trusted publishing with attestations → GitHub Release from CHANGELOG → Zenodo) | @@ -121,6 +122,7 @@ pass [actionlint](https://github.com/rhysd/actionlint) and zizmor. copier.yml questions, computed values (python_max, uv_version) and the after-copy message template/ the project; files ending in .jinja are rendered, everything else is copied as is justfile `just bake`, `just test` for working on the template +fixtures/ examples `just test` adds to the baked project, one of each kind the docs can run .github/workflows/ template.yml bakes the template and runs every check in the result ``` diff --git a/copier.yml b/copier.yml index a98885f..355fe44 100644 --- a/copier.yml +++ b/copier.yml @@ -107,6 +107,11 @@ publish_to_pypi: help: Publish releases to PyPI (trusted publishing)? default: true +docs_notebooks: + type: bool + help: "Run notebooks and Markdown code cells in the docs (myst-nb; adds Jupyter to the dev tools)?" + default: false + test_os: type: str help: Operating systems the tests run on diff --git a/fixtures/always/docs/session.rst b/fixtures/always/docs/session.rst new file mode 100644 index 0000000..c619b6d --- /dev/null +++ b/fixtures/always/docs/session.rst @@ -0,0 +1,17 @@ +IPython session +=============== + +Code in an rst page that runs when the docs build. + +.. ipython:: python + + from testpackage.plotting import square + + print(f"square(5) = {square(5)}") + +.. ipython:: python + + import matplotlib.pyplot as plt + + @savefig session_squares.png width=4in + plt.plot([0, 1, 2], [square(x) for x in [0, 1, 2]]); diff --git a/fixtures/always/src/testpackage/plotting.py b/fixtures/always/src/testpackage/plotting.py new file mode 100644 index 0000000..4fac585 --- /dev/null +++ b/fixtures/always/src/testpackage/plotting.py @@ -0,0 +1,27 @@ +"""A function whose docstring example draws a figure when the docs build.""" + + +def square(x: float) -> float: + """ + Square a number. + + Parameters + ---------- + x : float + The number. + + Returns + ------- + float + `x` squared. + + Examples + -------- + .. plot:: + + >>> import matplotlib.pyplot as plt + >>> from testpackage.plotting import square + >>> xs = [0, 1, 2, 3] + >>> _ = plt.plot(xs, [square(x) for x in xs]) + """ + return x * x diff --git a/fixtures/notebooks/docs/guide.md b/fixtures/notebooks/docs/guide.md new file mode 100644 index 0000000..1b81bf6 --- /dev/null +++ b/fixtures/notebooks/docs/guide.md @@ -0,0 +1,15 @@ +--- +file_format: mystnb +kernelspec: + name: python3 +--- + +# Guide + +A page of prose whose code cells run when the docs build. + +```{code-cell} +from testpackage.plotting import square + +print(f"square(4) = {square(4)}") +``` diff --git a/fixtures/notebooks/docs/notebook.ipynb b/fixtures/notebooks/docs/notebook.ipynb new file mode 100644 index 0000000..4a459a5 --- /dev/null +++ b/fixtures/notebooks/docs/notebook.ipynb @@ -0,0 +1,42 @@ +{ + "cells": [ + { + "cell_type": "markdown", + "id": "c1", + "metadata": {}, + "source": [ + "# Notebook\n", + "\n", + "Its cells run when the docs build." + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "c2", + "metadata": {}, + "outputs": [], + "source": [ + "import matplotlib.pyplot as plt\n", + "\n", + "from testpackage.plotting import square\n", + "\n", + "xs = [0, 1, 2, 3]\n", + "print(f\"square(3) = {square(3)}\")\n", + "_ = plt.plot(xs, [square(x) for x in xs])" + ] + } + ], + "metadata": { + "kernelspec": { + "display_name": "Python 3", + "language": "python", + "name": "python3" + }, + "language_info": { + "name": "python" + } + }, + "nbformat": 4, + "nbformat_minor": 5 +} diff --git a/justfile b/justfile index b01e756..f7691fc 100644 --- a/justfile +++ b/justfile @@ -12,7 +12,7 @@ bake dest=(tmp / "python-template-bake") *data: uvx copier copy --defaults --vcs-ref=HEAD --data project_name="Test Package" {{ data }} . {{ dest }} @echo "Baked into {{ dest }}" -# Bake both PyPI variants and run every check, release step and version check in each +# Bake with every yes/no question answered yes, then no, and run every check, release step and version check in each test: #!/usr/bin/env bash set -euo pipefail @@ -21,29 +21,55 @@ test: rm -rf "$root" mkdir -p "$root" git_() { git -c user.name="Template Test" -c user.email="test@example.invalid" "$@"; } - for pypi in true false; do - dest="$root/pypi-$pypi" - echo "::group::bake publish_to_pypi=$pypi" + # opt answers both yes/no questions: publish_to_pypi and docs_notebooks. + for opt in true false; do + dest="$root/optional-$opt" + echo "::group::bake optional=$opt" uvx copier copy --defaults --vcs-ref=HEAD \ - --data project_name="Test Package" --data publish_to_pypi="$pypi" \ + --data project_name="Test Package" \ + --data publish_to_pypi="$opt" --data docs_notebooks="$opt" \ "$template" "$dest" cd "$dest" + # Examples that run when the docs build: a figure in a docstring and an .rst + # page with IPython blocks; with docs_notebooks, a Markdown page with a code + # cell and a notebook too. + cp -r "$template/fixtures/always/." . + pages='session' + if [ "$opt" = true ]; then + cp -r "$template/fixtures/notebooks/." . + pages='session\n guide\n notebook' + fi + sed -i "s/^ usage$/ usage\n $pages/" docs/index.rst + grep -qx ' session' docs/index.rst git_ init --quiet && git_ checkout --quiet -b main uv lock + # Without docs_notebooks, Jupyter stays out of the dev tools. + if [ "$opt" = false ] && grep -q '^name = "ipykernel"$' uv.lock; then + echo "uv.lock has ipykernel without docs_notebooks." >&2; exit 1 + fi git_ add --all git_ commit --quiet -m "chore: start from python-template" git_ tag v0.1.0 uv sync --locked echo "::endgroup::" - echo "::group::checks publish_to_pypi=$pypi" + echo "::group::checks optional=$opt" uv run just quality-check uv run just build-docs + # The examples ran: each page's output and figures are in its HTML. + sed 's/<[^>]*>//g' docs/_build/html/session.html | grep -qF 'square(5) = 25' + grep -q 'session_squares.png' docs/_build/html/session.html + grep -q 'class="plot-directive"' docs/_build/html/_autosummary/testpackage.plotting.square.html + if [ "$opt" = true ]; then + grep -qF 'square(4) = 16' docs/_build/html/guide.html + grep -qF 'square(3) = 9' docs/_build/html/notebook.html + grep -q '&2; exit 1;; esac echo "::endgroup::" - echo "::group::changelog draft publish_to_pypi=$pypi" + echo "::group::changelog draft optional=$opt" git_ commit --quiet --allow-empty -m "feat(io): read the new format" git_ commit --quiet --allow-empty -m "fix!: drop the old reader" notes="$(uv run just changelog)" diff --git a/template/.pre-commit-config.yaml b/template/.pre-commit-config.yaml index 967e6d9..9ed47c3 100644 --- a/template/.pre-commit-config.yaml +++ b/template/.pre-commit-config.yaml @@ -47,13 +47,13 @@ repos: name: ruff check entry: uv run --no-sync ruff check --fix --show-fixes --force-exclude language: system - types_or: [python, pyi] + types_or: [python, pyi, jupyter] require_serial: true - id: ruff-format name: ruff format entry: uv run --no-sync ruff format --force-exclude language: system - types_or: [python, pyi] + types_or: [python, pyi, jupyter] require_serial: true - id: pyrefly name: pyrefly diff --git a/template/AGENTS.md.jinja b/template/AGENTS.md.jinja index c238490..7044554 100644 --- a/template/AGENTS.md.jinja +++ b/template/AGENTS.md.jinja @@ -23,6 +23,7 @@ Contribution mechanics are in [CONTRIBUTING.md](CONTRIBUTING.md). - `src/{{ package_name }}/`: the package (`py.typed` ships, so annotations are public API). - `tests/`: pytest suite. - `docs/`: Sphinx site (PyData theme). `docs/api.rst` lists the API by topic; each class gets a page with tables of its attributes and methods, and each member a page of its own (`docs/_templates/autosummary/`, `docs/_ext/api_pages.py`). + Its examples run when the docs build (see [Examples in the docs](#examples-in-the-docs)). ## Commands @@ -36,7 +37,7 @@ runs the same recipes. | Tests (no network/slow) | `just test` (parallel; extra args go to pytest) | | Lint / fix and format | `just lint` / `just format` | | Type check | `just type-check` (pyrefly) | -| Docs (warnings are errors) | `just build-docs` | +| Docs (runs every example; warnings are errors) | `just build-docs` | | Live docs preview | `just docs-serve` (port 8000) | | All pre-commit hooks | `just pre-commit` | | Build and check dists | `just dist` | @@ -74,6 +75,95 @@ After editing dependencies in `pyproject.toml`, run `uv lock` and commit line above, with a reason. - The library never configures logging; modules use `logging.getLogger(__name__)`. +## Examples in the docs + +Examples run when the docs build, so they show real output, and the build fails +when one breaks. + +- **A figure in a docstring:** put a `.. plot::` directive in the Examples + section. Its code runs (doctest `>>>` lines work), and the figure appears + under it. + + ```rst + Examples + -------- + .. plot:: + + >>> import matplotlib.pyplot as plt + >>> _ = plt.plot([0, 1, 2], [0, 1, 4]) + ``` + +- **Code in an `.rst` page:** a `.. ipython:: python` block runs, and shows each + line with its output, as in an IPython session. Changing a + `.. code-block:: python` to `.. ipython:: python` is enough to make it run. For + a figure, put `@savefig name.png` on the line above the plotting call. + + ```rst + .. ipython:: python + + import matplotlib.pyplot as plt + import {{ package_name }} + + {{ package_name }}.__version__ + + @savefig squares.png width=5in + plt.plot([0, 1, 2], [0, 1, 4]); + ``` + + A block that raises or warns fails the build; `:okexcept:` or `:okwarning:` + under the directive allows one that is meant to. All `.rst` pages share one + session, so each page imports and defines what it uses. +{%- if docs_notebooks %} +- **Code cells in a Markdown page:** a page (`.md`) under `docs/` can mix prose + with code cells that run, each showing what it prints and plots, as in a + notebook. The header makes the cells run: + + ````markdown + --- + file_format: mystnb + kernelspec: + name: python3 + --- + + # Getting started + + Some prose. + + ```{code-cell} + import {{ package_name }} + + print({{ package_name }}.__version__) + ``` + ```` + +- **A notebook:** an `.ipynb` under `docs/`, listed in a toctree like any page. + Commit it without outputs: the build runs every cell. +- **When they run:** `just build-docs` runs everything, as CI does. + `just docs-serve` reruns a Markdown page or notebook only when its code + changes, and an `.rst` page's blocks whenever that page is rebuilt. A cell + times out after 30 s; for a slower one, set `mystnb: {execution_timeout: 120}` + in the page's header, or `"mystnb": {"execution_timeout": 120}` in the + notebook's metadata. +{%- else %} +- **When they run:** `just build-docs` runs everything, as CI does; + `just docs-serve` reruns a page's blocks whenever that page is rebuilt. +- **Notebooks, and code cells in Markdown pages,** are off in this project. + `copier update --data docs_notebooks=true` turns them on; it adds myst-nb, and + Jupyter with it, to the dev tools. +{%- endif %} +- **Data:** the docs build on GitHub Actions, so an example uses synthetic data, + a small file in the repository, or a public download, never a machine-specific + path. Seed random numbers so the figures don't change from build to build. +- **Optional dependencies:** if an example needs an extra, add + `{{ distribution_name }}[extra]` to the `dev` group, so `uv sync` installs it + here and in CI. +{%- if docs_notebooks %} +- **A notebook that can't run in CI** (it needs data only one machine has) is + committed with its outputs and `"mystnb": {"execution_mode": "off"}` in its + metadata. The page then shows the outputs from its last run, so rerun it when + the code it uses changes. +{%- endif %} + ## Tests - `network`: needs live network access; `slow`: expensive. Both are skipped by diff --git a/template/CONTRIBUTING.md.jinja b/template/CONTRIBUTING.md.jinja index 75ec081..1d8e8aa 100644 --- a/template/CONTRIBUTING.md.jinja +++ b/template/CONTRIBUTING.md.jinja @@ -32,7 +32,7 @@ shows them all). | `just quality-check` | `lint`, `type-check`, `docstr` and `test`: what CI checks | | `just test` | pytest in parallel, skipping tests marked `network` or `slow` (extra args go to pytest; plain `uv run pytest` runs serially, for debugging) | | `just format` | fix lint and format the code with ruff | -| `just build-docs` | build the HTML docs into `docs/_build/`; warnings are errors | +| `just build-docs` | build the HTML docs into `docs/_build/`, running every example; warnings are errors | | `just docs-serve` | preview the docs at , rebuilt on every save (on a remote machine, forward the port, e.g. VS Code's Ports panel) | | `just changelog` | draft CHANGELOG entries from the commits since the last release | | `just pre-commit` | run every hook on every file | diff --git a/template/docs/conf.py.jinja b/template/docs/conf.py.jinja index c6d8611..2de0eaf 100644 --- a/template/docs/conf.py.jinja +++ b/template/docs/conf.py.jinja @@ -25,12 +25,18 @@ extensions = [ "sphinx.ext.intersphinx", "sphinx.ext.napoleon", "sphinx.ext.viewcode", + "matplotlib.sphinxext.plot_directive", # `.. plot::` runs code, shows the figure + "IPython.sphinxext.ipython_directive", # `.. ipython::` runs code in an .rst page + "IPython.sphinxext.ipython_console_highlighting", # colors its In/Out prompts +{%- if docs_notebooks %} + "myst_nb", # notebooks (.ipynb, or Markdown with code cells), run at build time +{%- endif %} "sphinx_autodoc_typehints", "sphinx_copybutton", "api_pages", # _ext/api_pages.py: class pages with member tables ] templates_path = ["_templates"] -exclude_patterns = ["_build"] +exclude_patterns = ["_build"{% if docs_notebooks %}, "**.ipynb_checkpoints"{% endif %}] # -- HTML ---------------------------------------------------------------------- @@ -67,6 +73,30 @@ autodoc_default_options = { } autosummary_generate = True # stubs go to docs/_autosummary/ (gitignored) +# Examples run when the docs build, so they show real output and fail the build +# when they break. A `.. plot::` directive (in a docstring's Examples section or +# on any page) shows its code and the figure it draws. +plot_include_source = True +plot_html_show_source_link = False +plot_html_show_formats = False +plot_formats = [("png", 100)] + +# In an .rst page, `.. ipython:: python` runs its code and shows each line with +# its output; a block that raises or warns fails the build. Its figures (a line +# `@savefig name.png` above the plotting call) are saved under docs/_build. +ipython_savefig_dir = "_build/savefig" +{%- if docs_notebooks %} + +# Code cells run too, in notebooks and in Markdown pages whose header names a +# kernel, and show every output; a cell that raises fails the build. Outputs are +# cached in docs/_build/.jupyter_cache, so `just build-docs` (which starts clean, +# as CI does) runs them all, and `just docs-serve` reruns a page only when its +# code changes. A notebook that cannot run in CI keeps the outputs it was +# committed with if its metadata has "mystnb": {"execution_mode": "off"}. +nb_execution_mode = "cache" +nb_execution_show_tb = True # print the traceback of a failing cell in the log +{%- endif %} + intersphinx_mapping = { "python": ("https://docs.python.org/3", None), "numpy": ("https://numpy.org/doc/stable/", None), diff --git a/template/justfile.jinja b/template/justfile.jinja index 250af6b..ce5f7f1 100644 --- a/template/justfile.jinja +++ b/template/justfile.jinja @@ -40,14 +40,14 @@ test *args: cov *args: uv run pytest -n auto --maxprocesses=8 -m "not network and not slow" --cov --cov-report=term --cov-report=xml --junitxml=junit.xml -o junit_family=legacy "$@" -# Build the HTML docs, failing on warnings +# Build the HTML docs, running every example, failing on warnings build-docs: rm -rf docs/_build docs/_autosummary uv run sphinx-build -M html docs docs/_build -W --keep-going # Serve the docs at http://127.0.0.1:PORT, rebuilding on every save (Ctrl-C stops) docs-serve port="8000": - uv run sphinx-autobuild docs docs/_build/html --port "$1" --watch src --re-ignore '_autosummary/' + uv run sphinx-autobuild docs docs/_build/html --port "$1" --watch src --re-ignore '_autosummary/' --re-ignore '_build/' --re-ignore '__pycache__/' # Everything the Code Quality workflow checks, plus the tests quality-check: lint type-check docstr test diff --git a/template/pyproject.toml.jinja b/template/pyproject.toml.jinja index d88b2af..e743ca5 100644 --- a/template/pyproject.toml.jinja +++ b/template/pyproject.toml.jinja @@ -36,6 +36,11 @@ Changelog = "https://github.com/{{ github_username }}/{{ repository_name }}/blob dev = [ "docstr-coverage>=2.3", "git-cliff>=2.10", # `just changelog` + "ipython>=9.0", # `.. ipython::` in the docs + "matplotlib>=3.8", # `.. plot::` in the docs +{%- if docs_notebooks %} + "myst-nb>=1.2", # notebooks and Markdown code cells in the docs +{%- endif %} "pre-commit>=4.0", "pydata-sphinx-theme>=0.16", "pyrefly>=1.3",