Feat: Add PiD (Pixel Diffusion Decoder) 4× super-resolution decode for FLUX / FLUX.2 / SD3 / SDXL / Z-Image / Qwen-Image

LICENSE-PiD.txt

@@ -0,0 +1,68 @@
+PiD (Pixel Diffusion Decoder) — License notice
+
+Upstream project: https://github.com/nv-tlabs/PiD
+Vendored under:   invokeai/backend/pid/
+
+================================================================================
+CODE (Apache License 2.0)
+================================================================================
+
+The PiD source code, including the `pid/_src/` subtree and the `pid/_ext/imaginaire/`
+framework subset, is licensed under the Apache License, Version 2.0.
+
+Copyright 2026 NVIDIA CORPORATION & AFFILIATES.
+
+Portions of the framework (pid/_ext/imaginaire/) were originally adapted from
+the cosmos-predict2.5 project (https://github.com/nvidia-cosmos/cosmos-predict2.5/).
+
+Files vendored into invokeai/backend/pid/ retain their original SPDX-License-Identifier
+headers. The Apache 2.0 license text is available at:
+
+    http://www.apache.org/licenses/LICENSE-2.0
+
+================================================================================
+MODEL WEIGHTS (NVIDIA Source Code License v1 — non-commercial)
+================================================================================
+
+The pre-trained PiD decoder checkpoints distributed by NVIDIA at
+
+    https://huggingface.co/nvidia/PiD
+
+are released under the NSCLv1 license. Per NSCLv1, the weights may only be used
+for non-commercial (research or evaluation) purposes:
+
+    https://huggingface.co/nvidia/PixelDiT-1300M-1024px/blob/main/LICENSE
+
+This restriction applies to the weights only, not to the InvokeAI source code
+or the vendored PiD source code (which remain Apache 2.0). Users are responsible
+for ensuring their use of the PiD weights complies with NSCLv1.
+
+================================================================================
+LOCAL MODIFICATIONS
+================================================================================
+
+The following changes were applied to the upstream PiD subset when vendoring:
+
+* All `pid.*` imports were rewritten to `invokeai.backend.pid.*`.
+* `pid/_src/configs/`, `pid/_src/tokenizers/`, `pid/_src/checkpointer/`,
+  `pid/_src/inference/_demo_*.py`, `from_*.py`, `create_dataset.py`,
+  `rae_generation.py`, and `scale_rae_generation.py` were dropped (not needed
+  for the decoder-only inference subset).
+* `pid/_ext/imaginaire/checkpointer/`, `trainer.py`, `visualize/`, `flags.py`,
+  `config.py`, `types/`, `utils/easy_io/`, `utils/callback.py`,
+  `utils/config_helper.py`, `utils/validator{,_params}.py` and the
+  `lazy_config/omegaconf_patch.py` were dropped.
+* The upstream `utils/log.py` (loguru-based) and `utils/misc.py` were replaced
+  with stdlib-based stubs covering only the API surface used by the decoder.
+* `lazy_config/file_io.py` (iopath PathManager) and `lazy_config/registry.py`
+  (fvcore Registry) were replaced with stdlib-only implementations.
+* `lazy_config/lazy.py` was reduced to a minimal `LazyCall`/`LazyConfig` stub;
+  the upstream yaml/cloudpickle/dill/detectron2 config save/load paths are
+  intentionally not supported.
+* `lazy_config/instantiate.py` was reduced to a stdlib-only implementation;
+  the upstream omegaconf `DictConfig`/`ListConfig` branches were dropped, so
+  no `omegaconf` dependency is required.
+* `_src/utils/model_loader.py` (which depended on Imaginaire's distributed
+  checkpointer + easy_io) and `_src/inference/inference_utils.py` (S3 / video
+  helpers) were removed; their decode-path equivalents are reimplemented in
+  `invokeai/backend/pid/decode.py`.

docs/src/content/docs/features/pid-decode.mdx

@@ -0,0 +1,80 @@
+---
+title: PiD Super-Resolution Decode
+lastUpdated: 2026-07-01
+sidebar:
+  order: 5
+---
+
+import { Steps, Aside, Tabs, TabItem } from '@astrojs/starlight/components'
+
+**PiD** (Pixel Diffusion Decoder) is an alternative way to turn a model's latents into an image. Instead of the usual VAE decode, it runs a short pixel-space diffusion that produces a **4× super-resolved** result in a single, few-step pass — so a 512×512 generation comes out as a detailed 2048×2048 image.
+
+Because it decodes in pixel space and is conditioned on your prompt, PiD often recovers finer texture and edge detail than a plain VAE decode followed by an upscaler.
+
+<Aside type="note">
+PiD replaces the VAE decode step of a single generation. It is not a separate "upscale" pass you run on an existing image — you enable it before generating.
+</Aside>
+
+## Supported models
+
+PiD works with these base models:
+
+| Base model | PiD decoder to install |
+|---|---|
+| FLUX.1 | PiD Decoder FLUX |
+| FLUX.2 Klein (4B / 9B) | PiD Decoder FLUX.2 |
+| Stable Diffusion 3 | PiD Decoder SD3 |
+| SDXL | PiD Decoder SDXL |
+| Z-Image / Z-Image Turbo | **PiD Decoder FLUX** (Z-Image shares FLUX.1's VAE) |
+| Qwen-Image | PiD Decoder Qwen-Image |
+
+<Aside type="tip">
+Z-Image has no PiD decoder of its own — when a Z-Image model is active, the decoder picker lists the **FLUX** decoders. That's expected.
+</Aside>
+
+## What you need to install
+
+PiD needs two extra models, both available in **Model Manager → Starter Models**:
+
+<Steps>
+1. A **PiD Decoder** for your base model (e.g. *PiD Decoder FLUX (2K)*). Some bases offer a *2K* and a *2K-to-4K* preset; SDXL and Qwen-Image ship only the *2K-to-4K* preset.
+2. The **Gemma 2 2B (PiD caption encoder)** — PiD uses it to condition the decode on your prompt. It installs automatically as a dependency of any PiD decoder, and is shared across all of them.
+</Steps>
+
+Each PiD decoder is roughly 5 GB and the shared Gemma-2 encoder is roughly 5 GB.
+
+<Aside type="note">
+The Starter Models install NVIDIA's **legacy** PiD checkpoints (the ones now under `checkpoints_deprecated/` on Hugging Face). NVIDIA's newer **v1.5** decoders use a different network and are not yet supported — installing one is rejected during model identification rather than failing at decode time.
+</Aside>
+
+## Enabling PiD
+
+Open the **Generation** settings for a supported model and expand the advanced options. You'll find a **PiD** control with three modes:
+
+<Tabs>
+  <TabItem label="Off">
+    Standard VAE decode. No PiD models required.
+  </TabItem>
+  <TabItem label="Fit">
+    Generate at the requested size, decode 4× with PiD, then downscale the result back to the requested size. This is the safe default and works everywhere — the output matches your bounding box exactly, so it composites cleanly on the Canvas.
+  </TabItem>
+  <TabItem label="Native">
+    Treat the requested dimensions as the **4× target**: the image is generated at target ÷ 4 and PiD's full 4× output is used directly (no downscale), preserving all of the added detail. Great when you want a large, highly-detailed result.
+  </TabItem>
+</Tabs>
+
+When PiD mode is not *Off*, pick your **PiD Decoder** and **Gemma-2 Encoder** below the mode selector. The **PiD Steps** control (default 4) sets how many decode steps run — the released checkpoints are trained for 4.
+
+PiD is available in both the **Generate** tab (text-to-image) and on the **Canvas** (image-to-image), in both Fit and Native modes.
+
+## Tips & limitations
+
+- **Turn off "Scale Before Processing"** on the Canvas when using PiD — PiD already decodes at 4×, so pre-scaling would inflate the work and is blocked.
+- **Inpaint / Outpaint** are not supported with PiD yet; use text-to-image or image-to-image.
+- **SDXL Refiner** cannot be combined with PiD — disable one of them.
+- PiD's memory use scales with the *output* resolution. A 2048px output needs only a little more headroom than a normal decode, but Native mode at large target sizes (e.g. a 4096px result) is significantly heavier.
+- Turbo variants (e.g. Z-Image Turbo) work as usual — the low step count / no-CFG only affects generation; PiD's own step count is separate.
+
+<Aside type="caution">
+The PiD decoder *code* is Apache-2.0, but NVIDIA releases the pretrained *weights* under a non-commercial / research license. Keep that in mind if you redistribute the checkpoints.
+</Aside>

docs/src/generated/invocation-context.json

@@ -343,6 +343,21 @@
           "returns": "A LoadedModelWithoutConfig object.",
           "signature": "(source: str | AnyHttpUrl, loader: Optional[Callable[[Path], AnyModel]] = None) -> LoadedModelWithoutConfig"
         },
+        {
+          "description": "Move a model (and all of its submodels) from VRAM to RAM, freeing its VRAM but keeping it cached.\nUse this when an invocation is done with a model for the rest of the run - e.g. a one-shot text encoder -\nso the next, larger load does not have to compete with it for VRAM. The model stays in the RAM cache, so\na subsequent load only re-streams it back to VRAM rather than rebuilding it from disk.",
+          "name": "offload_from_vram",
+          "parameters": [
+            {
+              "default": "",
+              "description": "The key or ModelField representing the model to offload.",
+              "name": "identifier",
+              "type": "Union[str, ModelIdentifierField]"
+            }
+          ],
+          "return_type": "int",
+          "returns": "The number of VRAM bytes freed.",
+          "signature": "(identifier: Union[str, ModelIdentifierField]) -> int"
+        },
         {
           "description": "Search for models by attributes.",
           "name": "search_by_attrs",