[feat/precompile-macro-infra] feat(cmake): opt-in precompiled static library, starting with pytypes.h - #6132
Open
henryiii wants to merge 4 commits into
Open
[feat/precompile-macro-infra] feat(cmake): opt-in precompiled static library, starting with pytypes.h#6132henryiii wants to merge 4 commits into
henryiii wants to merge 4 commits into
Conversation
…h pytypes.h Split the out-of-line pytypes.h definitions into pytypes-inl.h (fmtlib/ CLI11 style): inline by default, compiled once into a per-project static library when PYBIND11_PRECOMPILED is defined. Infrastructure: - pybind11_precompile() creates the lazy pybind11::precompiled STATIC library from the installed or in-tree src/ sources; PRECOMPILE / NO_PRECOMPILE keywords on pybind11_add_module and a global PYBIND11_PRECOMPILE switch select it per target. - A link-time guard symbol encodes PYBIND11_INTERNALS_VERSION, Py_GIL_DISABLED, PYBIND11_SIMPLE_GIL_MANAGEMENT, and PYBIND11_DETAILED_ERROR_MESSAGES, so a configuration mismatch is one readable undefined symbol. - src/ is installed to share/pybind11/src (wheel stays pure); src/pybind11_combined.cpp is a single-TU build for non-CMake use. - Tests: PYBIND11_TEST_PRECOMPILE builds the whole suite against the library, two new test_cmake_build cases, packaging file lists, tidy preset, and a 3-platform CI job. Assisted-by: ClaudeCode:claude-fable-5
In precompiled mode the -inl.h definitions are intentionally non-inline, so misc-definitions-in-headers fires on every one. The header-only tidy build already analyzes all -inl.h bodies via the bottom-of-header includes. Assisted-by: ClaudeCode:claude-fable-5
…macros pybind11::pybind11 only carries headers; Py_GIL_DISABLED lives on Python::Module via pybind11::module. Without it the library is ABI-mismatched on free-threaded builds, and on Windows the pyconfig.h autolink pragma in its objects requests pythonXY.lib instead of pythonXYt.lib. Assisted-by: ClaudeCode:claude-fable-5
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
🤖 AI text below 🤖
Description
Stacked on #6131 (which is stacked on #6130). Implements the opt-in pre-compilation mode discussed in #2445 / #4001, using the nanobind/CLI11 model: a static library built lazily inside the consumer's own project, never shipped as a binary, so all per-DSO behavior (local internals key, module-local self-checks, libc++ exception classes) is preserved. Header-only stays the default and is byte-for-byte unchanged (verified: test binary size identical).
What this first slice contains:
pytypes.hsplit: out-of-line definitions (error_fetch_and_normalize,error_string,raise_from,memoryview::from_buffer) move topytypes-inl.h, taggedPYBIND11_INLINE, included at the bottom of the header unlessPYBIND11_PRECOMPILEDis defined. Later PRs movedetail/class.h, the non-template parts ofdetail/type_caster_base.h,detail/internals.h, and thepybind11.hdispatcher machinery through the same mechanism.pybind11_precompile()creates a lazypybind11::precompiledSTATICEXCLUDE_FROM_ALLtarget fromsrc/*.cpp;pybind11_add_module(... PRECOMPILE), a globalPYBIND11_PRECOMPILEswitch, and aNO_PRECOMPILEopt-out (needed for modules that redefinePYBIND11_INTERNALS_VERSIONper TU, likeexo_planet_pybind11). Works in both subdirectory and installed/config modes.PYBIND11_MODULEcalls a do-nothing library function whose name encodesPYBIND11_INTERNALS_VERSION,Py_GIL_DISABLED,PYBIND11_SIMPLE_GIL_MANAGEMENT, andPYBIND11_DETAILED_ERROR_MESSAGES, so a config mismatch (or forgetting to link) is one readable undefined symbol. It caught theexo_planet_pybind11internals-version mismatch on the first test run.src/installs toshare/pybind11/src(wheel remainspy3-none-any; it ships sources, not binaries);src/pybind11_combined.cppis a single-TU amalgam for non-CMake builds;pybind11_SRC_DIRexported from the CMake config.-DPYBIND11_TEST_PRECOMPILE=ONbuilds the entire test suite against the library (any definition missed by the split becomes a link error across ~50 modules; full pytest passes in both modes locally), two newtest_cmake_buildcases (subdirectory_precompile,installed_precompile), packaging file-set updates, clang-tidy preset coverage, and a 3-platform CI job.setup_helpers/ CLI / pkg-config / Meson support and docs+benchmarks follow in later PRs.Suggested changelog entry:
pybind11_add_module(... PRECOMPILE)(or the globalPYBIND11_PRECOMPILECMake option) compiles part of pybind11 once into a per-project static library instead of into every translation unit, reducing build times. Header-only remains the default.📚 Documentation preview 📚: https://pybind11--6132.org.readthedocs.build/