Fix plugin installation across development and frozen environments - #830
Open
Irozuku wants to merge 10 commits into
Open
Fix plugin installation across development and frozen environments#830Irozuku wants to merge 10 commits into
Irozuku wants to merge 10 commits into
Conversation
Plugins are installed at runtime by shelling out to pip. A uv managed .venv ships no pip at all, and a PyInstaller bundle has no interpreter to bootstrap one with ensurepip, so pip has to be declared explicitly. The floor is 22.2, the first release with "pip install --report".
Plugins cannot live in the interpreter's own environment. The packaged distributions are read only (a PyInstaller bundle, the squashfs image inside an AppImage), and in a uv managed checkout every uv run re-syncs .venv against uv.lock and uninstalls whatever is not locked, plugins included. Add <DASHAI_LOCAL_PATH>/plugins/py<major>.<minor> instead, activated on sys.path and exported through PYTHONPATH so child processes such as the Huey consumer see it too. The directory is scoped by interpreter version because plugins may ship compiled extension modules.
pip's --target mode hardcodes ignore_installed, so installing a plugin straight into the plugins directory re-downloads its whole dependency tree, torch included. Split the installation in two phases instead: 1. resolve the requirement against the running environment with pip install --dry-run --report, which lists only what is missing; 2. install exactly those, pinned to the resolved artifact URLs, with --target <plugins dir> --no-deps. pip is invoked as "sys.executable -m pip" rather than as a bare pip executable, since a bare pip found on PATH belongs to some unrelated interpreter and would put the plugin where dashAI can never import it. pip uninstall refuses to touch a --target directory, so removal walks the RECORD file of every distribution the plugin owns and that no other installed plugin still needs, tracked in a ledger next to them.
execute_pip_command ran "pip install <name>" through a PATH lookup and installed into the interpreter's own environment. Neither half survives the move to uv: a uv managed .venv has no pip script, so the call either fails outright or reaches an unrelated global pip whose entry points dashAI never sees, and anything that did land in .venv was deleted by the next uv run. Route install_plugin and uninstall_plugin through the plugins installer instead. get_available_plugins now activates the plugins directory and invalidates the import caches first, so a plugin installed while the app is running is discovered without a restart, and an entry point that fails to import is skipped rather than taking the whole registry down.
create_app has to make the directory importable before build_config_dict runs, because collecting the initial components already enumerates the installed plugin entry points. The CLI activates it before copying the environment for the Huey consumer, which imports plugin components in its own process and therefore needs the directory on its PYTHONPATH.
A failed install only raised out of the endpoint, so the frontend got a bare Internal Server Error with nothing to act on. Catch PluginInstallError in the install and upgrade paths and return the pip error text as the response detail.
A PyInstaller bundle ships no python executable, so "sys.executable -m pip" re-enters the launcher. Add a runtime hook that answers the two interpreter invocations that path needs, "-m <module>" and a script path, and falls through to the normal CLI for anything else. pip uses the same form to spawn its own build isolation, so both are required. The spec now carries pip itself, plus the dist-info metadata of every dependency collected recursively. Without the metadata the resolution phase considers torch and friends missing and downloads the whole tree into the user's plugins directory.
pip reports an unhandled crash as an "ERROR: Exception:" line followed by the traceback that explains it. The error formatter kept only the lines containing "ERROR", so a real failure surfaced as the single useless string "ERROR: Exception:" and the cause was unrecoverable. Report everything from the first error line onwards, capped, and log the complete pip output whenever pip exits non zero.
Installing a plugin from the Windows executable died with "Unable to locate finder for 'pip._vendor.distlib'". pip imports pip._vendor.distlib.scripts before installing a wheel, and that module builds its script wrapper table at import time through distlib.resources.finder(), which dispatches on the type of the package's loader against a registry holding only the stdlib loaders and zipimporter. PyInstaller's loader is in neither, so the import raised. Resolution was unaffected because pip imports that module only when it is about to install something, which is why the failure looked like a network or metadata problem rather than an import error. Register the bundle's loader before handing control to pip. distlib's register_finder applies type() to its argument, so it takes the loader instance and not the loader class.
Irozuku
added
bug
Irozuku
marked this pull request as ready for review
August 19, 2026 18:54
Irozuku
marked this pull request as draft
August 19, 2026 18:58
Installing a plugin from the AppImage failed with "pip exited with code 2" and no output at all. sys.executable inside a bundled launcher is dashAI itself, not a bare interpreter, so "sys.executable -m pip install ..." re-entered the app; its CLI rejected the arguments and click exits with code 2 for a usage error, while the relaunch logic in the AppImage entry point kept the message off our pipes. __main__ already documents the same caveat for the Huey consumer. Resolve the interpreter instead of assuming it: try sys.executable first, the interpreters next to sys.prefix first when running inside an AppImage, and verify each candidate actually executes Python before using it. PATH is deliberately not consulted, since an interpreter found there belongs to a different environment. A silent pip failure now names the command that produced it, which is what this report was missing.
Irozuku
marked this pull request as ready for review
August 19, 2026 19:31
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Plugin installation was broken in several environments: a
uvmanaged.venvhas nopipscript, and frozen builds (PyInstaller/AppImage) ship a read only environment where plugins can't be installed at all.This branch moves plugin distributions into a writable per user, per interpreter directory outside the environment and drives installation through
sys.executable -m pip, so it works fromuv run, a dev checkout, and a frozen bundle.Type of Change
Changes (by file)
DashAI/back/plugins/environment.py(new): Resolves and activates a writable, interpreter scoped plugins directory (~/.DashAI/plugins/pyX.Y), appends it tosys.path/PYTHONPATH, and invalidates import caches.DashAI/back/plugins/installer.py(new): Installs/uninstalls plugins viasys.executable -m pip. Uses a two phase install (--dry-run --report, then--target --no-deps) so already satisfied dependencies (e.g.torch) aren't re downloaded. A.dashai-plugins.jsonledger tracks distribution ownership for clean uninstalls. RaisesPluginInstallErrorwith pip's real error.DashAI/back/plugins/utils.py: Replaced the bare pip subprocess with the new installer.get_available_plugins()now activates the plugins directory, deduplicates plugins, and skips plugins that fail to import instead of crashing the registry.DashAI/back/api/api_v1/endpoints/plugins.py: SurfacesPluginInstallErrorto the client as an HTTP 500 with pip's traceback instead of a generic error.DashAI/back/app.py: Activates the plugins directory before the initial config/component collection.DashAI/__main__.py: Activates the plugins directory before copying the environment to the Huey consumer.pyproject.toml: Addedpip>=22.2as a runtime dependency.dashai.spec: Bundles pip and dependency.dist-infometadata into the frozen build and registers the interpreter runtime hook.hooks/rthook_python_surrogate.py(new): Makes the frozen launcher act as a Python interpreter (-m/.pyscript) and registers a distlib resource finder so pip can install wheels inside the bundle.tests/back/plugins/*: Added coverage for the environment resolver, installer (resolve/install/uninstall/ledger), utils, and the PyInstaller surrogate hook.Testing
uv run pytest tests/back/plugins/covers the new installer and environment logic.uv rundev server and from a frozen build across different operative systemsNotes
site-packageswhen nothing is found in the plugins directory, preserving backwards compatibility.