Feature/python standalone testing

AGENTS.md

@@ -9,7 +9,7 @@ AGAVE (Advanced GPU Accelerated Volume Explorer) is a C++17/Qt6 desktop applicat
 | `agave_app/`      | Qt6 GUI layer — widgets, dialogs, dock panels, OpenGL viewport (`GLView3D`)                                                                       |
 | `renderlib/`      | Core rendering engine — image I/O (`io/`), graphics abstraction (`gfxapi/`) with OpenGL and Vulkan backends (`gfxOpenGL/`, `gfxVulkan/`), camera, scene, gesture handling (`gesture/`), JSON serialization (`json/`) |
 | `agave_pyclient/` | Python WebSocket client for remote control of AGAVE in server mode                                                                                |
-| `agave_pyvk/`     | Standalone Python client including all data loading and rendering functionality via the Vulkan graphics backend                                                                             |
+| `agave_pyvk/`     | Standalone Python client wrapping renderlib, including all data loading and rendering functionality via the Vulkan graphics backend                                                                             |
 | `test/`           | C++ unit tests (Catch2)                                                                                                                           |
 | `webclient/`      | JavaScript client                                                                                                                                 |
 

README.md

@@ -25,6 +25,17 @@ or searches the platform's usual Vulkan SDK installation directory.
 
 You may also refer to the github actions workflows for reference building AGAVE.
 
+### Activate your Python environment first
+
+Before configuring CMake, activate the Python virtual environment you want
+`agave_pyvk` installed into. CMake locates the interpreter from your `PATH`
+and records it in the build tree; the ABI-specific `agave_py2` module
+(`_native.<abi>.so`) is compiled against **that** interpreter. Later
+`cmake --build` invocations never re-detect Python, so building with a
+different env active will silently produce a wrong-ABI extension. If you
+switch envs, reconfigure explicitly (see
+[`agave_pyvk/README.md`](agave_pyvk/README.md)).
+
 ### Windows
 
 Make sure you are in an environment where vsvarsall has been run, e.g. a "VS2026 x64 Native Tools Command Prompt"
@@ -121,10 +132,10 @@ If cmake fails please refer to the Dockerfile or the github actions workflows fo
 
 ### Iterative standalone Python development
 
-The platform commands above create a persistent `build` directory. CMake uses
-the active Python interpreter and C++ compiler environment, and its default
-build compiles the shared static renderlib, the desktop application,
-and the ABI-specific `agave_py2` native module:
+With your target Python environment activated, the platform commands above
+create a persistent `build` directory. CMake records that interpreter, and
+its default build compiles the shared static renderlib, the desktop
+application, and the ABI-specific `agave_py2` native module:
 
 ```console
 cmake --build .
@@ -142,6 +153,10 @@ in the Python package. The editable install performs only normal Python
 packaging; it does not invoke CMake or rebuild renderlib. Python source changes
 are visible immediately.
 
+If you later need to build the Python module for a _different_ interpreter
+without rebuilding the rest of the tree, see the retargeting recipe in
+[`agave_pyvk/README.md`](agave_pyvk/README.md).
+
 Optional: To save a little time in building the Python module only, use the same
 platform-specific CMake configure command with these options:
 

agave_app/ViewerState.cpp

@@ -94,30 +94,16 @@ stateToPythonScript(const Serialize::ViewerState& s)
   std::ostringstream ss;
   ss << "# AGAVE Python script (standalone)" << std::endl;
   ss << "# Generated by AGAVE version " << qApp->applicationVersion().toStdString() << std::endl;
-  ss << "# We recommend exiting any AGAVE instances before running this script." << std::endl;
-  ss << "# pip install agave_pyclient" << std::endl;
+  ss << "# AGAVE application path: " << qApp->applicationFilePath().toStdString() << std::endl;
+  ss << "# pip install agave_pyvk" << std::endl;
   ss << "# python myscript.py" << std::endl << std::endl;
-  ss << "import agave_pyclient as agave" << std::endl << std::endl;
+  ss << "import agave_pyvk as agave" << std::endl << std::endl;
   renderlib::RendererType rendererType = renderlib::RendererType_Pathtrace;
   if (s.rendererType == Serialize::RendererType_PID::RAYMARCH) {
     rendererType = renderlib::RendererType_Raymarch;
   }
   std::string mode = renderlib::rendererTypeToString(rendererType);
-  std::string appPath = qApp->applicationFilePath().toStdString();
-  // Escape the path for embedding inside a Python double-quoted string literal.
-  std::string escapedAppPath;
-  escapedAppPath.reserve(appPath.size());
-  for (char c : appPath) {
-    if (c == '\\' || c == '"') {
-      escapedAppPath.push_back('\\');
-    }
-    escapedAppPath.push_back(c);
-  }
-  // TODO give user option to save a self-contained script or require agave server
-  // For now, most users will probably not mind that we are putting the executable
-  // path in the Python script.
-  // Pass auto_launch=False or omit agave_path to customize this behavior.
-  ss << "r = agave.AgaveRenderer(agave_path=\"" << escapedAppPath << "\", mode=\"" << mode << "\")" << std::endl;
+  ss << "r = agave.AgaveRenderer(mode=\"" << mode << "\")" << std::endl;
   ss << "with r:" << std::endl;
   std::string obj = "    r.";
 

agave_app/agaveGui.cpp

@@ -248,7 +248,7 @@ agaveGui::createActions()
   connect(m_dumpJsonAction, SIGNAL(triggered()), this, SLOT(saveJson()));
 
   m_dumpPythonAction = new QAction(tr("&Save to Python script"), this);
-  m_dumpPythonAction->setStatusTip(tr("Save a Python script usable with agave_pyclient"));
+  m_dumpPythonAction->setStatusTip(tr("Save a Python script usable with agave_pyvk"));
   connect(m_dumpPythonAction, SIGNAL(triggered()), this, SLOT(savePython()));
 
   m_testMeshAction = new QAction(tr("&Open mesh..."), this);
@@ -1407,8 +1407,7 @@ agaveGui::appToViewerState()
 
   v.pathTracer.primaryStepSize = m_renderSettings.m_RenderSettings.m_StepSizeFactor;
   v.pathTracer.secondaryStepSize = m_renderSettings.m_RenderSettings.m_StepSizeFactorShadow;
-  v.pathTracer.multichannelBlendMode =
-    static_cast<int32_t>(m_renderSettings.m_RenderSettings.m_MultichannelBlendMode);
+  v.pathTracer.multichannelBlendMode = static_cast<int32_t>(m_renderSettings.m_RenderSettings.m_MultichannelBlendMode);
 
   if (m_appScene.m_volume) {
     for (size_t i = 0; i < m_appScene.m_volume->sizeC(); ++i) {

agave_pyclient/agave_pyclient/agave.py

@@ -967,18 +967,28 @@ def load_data_and_get_info(
         flushes the command buffer to the server, waits for the JSON text
         frame the server sends back, and returns the parsed dictionary.
 
-        The returned dict contains at least the following keys (see
-        ``LoadDataCommand::execute`` in ``renderlib/command.cpp``):
+        The returned dict is built by ``volumeInfoToJson`` in
+        ``renderlib/VolumeInfoToJson.cpp``, shared with every other volume
+        loading path, and contains at least the following keys:
 
         - ``commandId``           : int, id of the LOAD_DATA command
+        - ``name``                : str, the file path the volume came from
         - ``x``, ``y``, ``z``, ``c``, ``t`` : int, image extents
         - ``pixel_size_x/y/z``    : float, physical voxel size
+        - ``spatial_units``       : str
         - ``channel_names``       : list[str]
         - ``channel_min_intensity``: list[int], one value per channel
         - ``channel_max_intensity``: list[int], one value per channel
         - ``volume_dimensions``   : nested dict mirroring the C++
           ``VolumeDimensions`` struct
 
+        The top-level fields describe the volume that was actually loaded,
+        after any ``channels``, ``region``, or ``multiresolution_level``
+        subsetting. ``volume_dimensions`` describes the source file as the
+        reader reported it, so it may be larger. ``commandId`` is a transport
+        field and is the only key not also returned by the in-process
+        ``agave_pyvk`` loaders.
+
         Note: any other commands queued in the buffer prior to calling
         this method will be flushed at the same time.
 

agave_pyvk/README.md

@@ -41,29 +41,83 @@ remaining native dependencies are the same ones used by renderlib.
 
 ## Editable development install
 
-Configure the normal top-level AGAVE build using the Python environment where
-the package will be installed. Then build renderlib, the AGAVE application, and
-the ABI-specific native module from that build directory:
+### How the build is organized
+
+`agave_pyvk` produces a single Python-linked artifact: the nanobind extension
+`_native.<abi>.so` (target `agave_py2`). Everything below it — `renderlib`,
+libCZI, tensorstore, curl, and the AGAVE app — is plain C++ that does **not**
+link `libpython` and therefore does **not** need to be rebuilt when you switch
+Python interpreters. Only `agave_py2` is ABI-specific.
+
+`pip` never builds the extension in this project. The `pyproject.toml` uses
+plain `setuptools`; the `.so` is produced by CMake and staged into
+`agave_pyvk/agave_pyvk/` by the `stage_agave_pyvk` target. `pip install -e .`
+just registers the pure-Python package and picks up whatever `.so` CMake last
+staged.
+
+### First-time build
+
+Configure the top-level AGAVE build against the Python environment you intend
+to install into. The interpreter is baked into the CMake cache at configure
+time — it is not re-detected by `cmake --build`.
 
 ```console
-cmake --build .
+cmake -S <agave-root> -B <agave-root>/build -G Ninja \
+      -DPython_EXECUTABLE="$(python -c 'import sys; print(sys.executable)')" \
+      -DCMAKE_BUILD_TYPE=Release
+cmake --build <agave-root>/build --target stage_agave_pyvk
 ```
 
-From the repository's `agave_pyvk` directory, install the package into that
-same Python environment:
+Then from `agave_pyvk/`:
 
 ```console
 python -m pip install -e .
 ```
 
-The CMake build stages `_native` and its runtime dependencies into the package
-directory. Pip performs only normal Python packaging and does not invoke CMake.
-Python source edits are visible immediately.
+Python source edits are visible immediately; native edits require re-running
+the `stage_agave_pyvk` target.
+
+### Switching Python interpreters — minimal rebuild
+
+The recommended workflow is: **activate the target Python environment before
+you touch CMake.** `cmake --build .` alone will never re-detect the interpreter
+— `Python_EXECUTABLE` is pinned in `CMakeCache.txt` at first configure — so
+building from the wrong env silently produces a wrong-ABI extension.
+
+If you want to retarget an existing `build` tree at a different Python without
+rebuilding `renderlib` or the app, activate the new env and then run two
+commands from the build directory:
+
+```console
+cd <agave-root>/build
+cmake -DPython_EXECUTABLE="$(python -c 'import sys; print(sys.executable)')" ..
+cmake --build . --target stage_agave_pyvk
+```
+
+FindPython re-detects because `Python_EXECUTABLE` changed; nanobind's ABI
+suffix cache is already cleared on every reconfigure by
+`agave_pyvk/CMakeLists.txt`. Only `agave_py2` recompiles (a few seconds).
+`stage_agave_pyvk` deletes any previously staged `_native*.so` / `_native*.pyd`
+from the package directory and installs the fresh one, so the package never
+ends up with two mismatched extensions.
+
+If `import agave_pyvk` reports an ABI mismatch, the current interpreter and the
+staged `.so` disagree. Re-run the two commands above from the environment you
+want to use.
+
+### Standalone build (skip the AGAVE app and tests)
+
+```console
+cmake -S <agave-root> -B <agave-root>/build -G Ninja \
+      -DPython_EXECUTABLE="$(python -c 'import sys; print(sys.executable)')" \
+      -DAGAVE_BUILD_APP=OFF -DAGAVE_BUILD_TESTS=OFF -DAGAVE_BUILD_PYVK=ON \
+      -DCMAKE_BUILD_TYPE=Release
+cmake --build <agave-root>/build --target stage_agave_pyvk
+```
 
-From this point you can import agave_pyvk and its AgaveRenderer class.
+### Building a wheel
 
-To produce a wheel file from the same staged native build, install the `build`
-frontend and run:
+Once the native module is staged for the desired interpreter:
 
 ```console
 python -m pip install build

agave_pyvk/agave_pyvk/__init__.py

@@ -1,13 +1,66 @@
 """Headless in-process Vulkan bindings for AGAVE."""
 
 import os
+import sys
 from pathlib import Path
 
 
 _dll_directory_handle = None
 if os.name == "nt":
     _dll_directory_handle = os.add_dll_directory(str(Path(__file__).resolve().parent))
 
+
+def _check_native_abi() -> None:
+    """Fail with a clear message if the staged _native extension is ABI-incompatible.
+
+    The extension is built by CMake and staged into this package; there is no
+    automatic rebuild when the active interpreter changes. Without this check,
+    a mismatch surfaces as a misleading "circular import" ImportError.
+    """
+    pkg_dir = Path(__file__).resolve().parent
+    current_tag = sys.implementation.cache_tag  # e.g. "cpython-314"
+    if os.name == "nt":
+        ext_glob, ext_suffix = "_native*.pyd", ".pyd"
+    else:
+        ext_glob, ext_suffix = "_native*.so", ".so"
+
+    candidates = sorted(pkg_dir.glob(ext_glob))
+    if not candidates:
+        return  # let the normal import machinery raise ModuleNotFoundError
+
+    def _tag(path: Path) -> str:
+        # For example:
+        # "_native.cpython-312-x86_64-linux-gnu.so"
+        # -> "cpython-312-x86_64-linux-gnu"
+        name = path.name[: -len(ext_suffix)]
+        prefix = "_native."
+
+        if not name.startswith(prefix):
+            return ""
+
+        return name[len(prefix):]
+
+    current_tag = sys.implementation.cache_tag
+
+    def is_compatible(path: Path) -> bool:
+        tag = _tag(path)
+        return tag == current_tag or tag.startswith(f"{current_tag}-")
+
+    if any(is_compatible(path) for path in candidates):
+        return
+
+    found = ", ".join(sorted({_tag(p) or p.name for p in candidates}))
+    raise ImportError(
+        f"agave_pyvk native extension ABI mismatch: found [{found}] in {pkg_dir}, "
+        f"but current interpreter is {current_tag} ({sys.executable}). "
+        "Reconfigure the CMake build with "
+        "-DPython_EXECUTABLE=$(python -c 'import sys; print(sys.executable)') "
+        "and rebuild the stage_agave_pyvk target."
+    )
+
+
+_check_native_abi()
+
 from .agave import AgaveRenderer
 
 __author__ = "Daniel Toloudis, Allen Institute"