Hailuo MiniMax H3 training support - #3003
Open
bghira wants to merge 9 commits into
Open
Conversation
Contributor
There was a problem hiding this comment.
Pull request overview
This PR adds end-to-end SimpleTuner integration for the MiniMax H3 model family (including pipelines/blocks and model registry wiring), and updates docs/tests to reflect the new support. It also extends the conditioning-data pipeline for i2v_first_frame (video → first-frame PNG) so autogenerated conditioning datasets and their metadata can be generated, duplicated, and sampled consistently.
Changes:
- Register the
minimaxh3model family and add a full MiniMax H3 implementation (scheduler, packing, modular pipeline blocks, encoders/decoders/denoise, audio autoencoder, etc.). - Add/adjust i2v-first-frame conditioning behavior across metadata duplication, conditioning generation, and sampler path resolution, with expanded unit tests.
- Update READMEs and QUICKSTART docs (all language variants) to include MiniMax H3 in supported-model and compatibility matrices; add an example training config.
Reviewed changes
Copilot reviewed 52 out of 52 changed files in this pull request and generated 1 comment.
Show a summary per file
| File | Description |
|---|---|
| tests/test_training_sample.py | Adds tests for explicit pairing metadata and i2v-first-frame metadata duplication behavior. |
| tests/test_sdr_sample_generator.py | Adds a test ensuring i2v-first-frame targets .png filenames. |
| tests/test_sampler.py | Adds a test validating conditioning sampling maps video paths to .png for i2v-first-frame. |
| tests/test_s2v_auto_split.py | Makes the audio-isolation test use temp dirs for better test hygiene. |
| tests/test_factory_edge_cases.py | Adds coverage ensuring i2v video datasets auto-generate first-frame conditioning when absent. |
| simpletuner/helpers/multiaspect/sampler.py | Adds i2v-first-frame conditioning path mapping (.mp4 → .png) during conditioning sample fetch. |
| simpletuner/helpers/models/model_metadata.json | Registers minimaxh3 metadata (class/module path, flavors). |
| simpletuner/helpers/models/minimaxh3/scheduler.py | Introduces a MiniMax-H3-specific rectified-flow Euler scheduler implementation. |
| simpletuner/helpers/models/minimaxh3/pipeline.py | Adds a conventional pipeline entry point for FL2VA/T2VA. |
| simpletuner/helpers/models/minimaxh3/pipeline_ref.py | Adds a conventional pipeline entry point for Ref2VA. |
| simpletuner/helpers/models/minimaxh3/packing.py | Implements packed-sequence geometry/constants and packing/unpacking utilities. |
| simpletuner/helpers/models/minimaxh3/modular_pipeline.py | Adds modular pipeline and LoRA loading/saving integration for MiniMax H3. |
| simpletuner/helpers/models/minimaxh3/modular_blocks_minimax_h3.py | Defines modular block graph(s) for FL2VA/T2VA and Ref2VA workflows. |
| simpletuner/helpers/models/minimaxh3/encoders.py | Adds text encoding and visual/audio reference encoding steps for MiniMax H3 workflows. |
| simpletuner/helpers/models/minimaxh3/denoise.py | Adds denoise loop with optional real CFG + skipped-layer guidance support. |
| simpletuner/helpers/models/minimaxh3/decoders.py | Adds video/audio decode steps to produce final outputs from denoised rows. |
| simpletuner/helpers/models/minimaxh3/before_encoder.py | Adds setup steps resolving canvas/frame counts and reference preparation. |
| simpletuner/helpers/models/minimaxh3/before_denoise.py | Adds layout building, noise preparation, and timestep-plan staging. |
| simpletuner/helpers/models/minimaxh3/autoencoder_audio.py | Adds MiniMax-H3 audio VAE (DAC/BigVGAN lineage) implementation. |
| simpletuner/helpers/models/minimaxh3/activations.py | Adds MiniMax-H3 MLP/SwiGLU activation helpers. |
| simpletuner/helpers/models/minimaxh3/init.py | Exposes MiniMax H3 components via package exports. |
| simpletuner/helpers/models/common.py | Adds minimaxh3 to supported families and upstream config sources. |
| simpletuner/helpers/metadata/utils/duplicator.py | Adds i2v-first-frame-aware path translation and metadata copying rules for conditioning datasets. |
| simpletuner/helpers/image_manipulation/training_sample.py | Allows explicit training_sample_path metadata to override training path resolution. |
| simpletuner/helpers/data_generation/sample_generator.py | Adds .png target extension for I2VFirstFrameSampleGenerator. |
| simpletuner/helpers/data_generation/conditioning.py | Supports generator-defined target extensions and refines bucket processing accounting/timeout logic. |
| simpletuner/helpers/data_backend/factory.py | Switches implicit i2v conditioning to first-frame generation and extends config versioning + text-embed caching hooks. |
| simpletuner/helpers/caching/vae.py | Includes minimaxh3 in video-latents preparation handling. |
| simpletuner/examples/minimaxh3-fl2va-convrot-int8.peft-lora/config.json | Adds an example config for training MiniMax H3 FL2VA convrot-int8 LoRA. |
| README.md | Adds MiniMax H3 to the supported model family list (license/conditions). |
| README.zh.md | Adds MiniMax H3 to the supported model family list (Chinese). |
| README.ja.md | Adds MiniMax H3 to the supported model family list (Japanese). |
| README.pt-BR.md | Adds MiniMax H3 to the supported model family list (pt-BR). |
| README.es.md | Adds MiniMax H3 to the supported model family list (Spanish). |
| README.hi.md | Adds MiniMax H3 to the supported model family list (Hindi). |
| documentation/quickstart/index.md | Adds MiniMax H3 to quickstart model overview tables and license footnotes. |
| documentation/quickstart/index.zh.md | Same as above (Chinese). |
| documentation/quickstart/index.ja.md | Same as above (Japanese). |
| documentation/quickstart/index.pt-BR.md | Same as above (pt-BR). |
| documentation/quickstart/index.es.md | Same as above (Spanish). |
| documentation/quickstart/index.hi.md | Same as above (Hindi). |
| documentation/QUICKSTART.md | Adds MiniMax H3 to compatibility matrices and model guide tables. |
| documentation/QUICKSTART.zh.md | Same as above (Chinese). |
| documentation/QUICKSTART.ja.md | Same as above (Japanese). |
| documentation/QUICKSTART.pt-BR.md | Same as above (pt-BR). |
| documentation/QUICKSTART.es.md | Same as above (Spanish). |
| documentation/QUICKSTART.hi.md | Same as above (Hindi). |
Suppressed comments (2)
simpletuner/helpers/data_generation/conditioning.py:518
- In CPU mode,
_check_completion_queue()only returns the number of successful items. If a worker completes a batch with partial failures (successful < total), those failures are never accounted for here, which later makesprocessed < dispatchedpermanently true and can force the end-of-bucket wait loop to run until the 300s timeout.
This issue also appears on line 552 of the same file.
if not self.gpu_mode:
done = self._check_completion_queue()
stats["processed"] += done
simpletuner/helpers/data_generation/conditioning.py:556
- The end-of-bucket wait loop compares
processed(successes) againstdispatched(attempted). If any dispatched items fail,processedcan never reachdispatched, so this loop will always run until the full 300s timeout even though all work has already finished (just with failures). Track failures and compare against(processed + errors)instead.
while stats["processed"] < dispatched and time.time() - start < 300:
completed = self._check_completion_queue()
if completed:
stats["processed"] += completed
time.sleep(0.1)
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
added 8 commits
August 5, 2026 16:22
When duplicating metadata, absolute conditioning paths outside the source dataset root now resolve to a basename under the target root instead of carrying relpath .. segments. Add coverage for inside-root paths, outside-root absolute paths, and i2v first-frame extension rewriting so future metadata copies cannot escape the target directory.
Add a shared SDNQ compile-mode helper and call it during argument parsing, trainer startup, and ConvRot SDNQ loader setup so eager versus torch.compile mode is applied before SDNQ modules are imported. Preserve the existing warning when SDNQ was imported too early, avoid duplicate warnings after a successful prior configuration, and cover the parse-time and loader-time paths in quantization config tests.
Register an h3_drift distiller that disables the active LoRA or LyCORIS adapter for a frozen parent prediction, then reduces video and optional audio prediction drift with token-balanced or modality-balanced weighting. Expose the method through the distillation factory, WebUI distillation selector, CLI option docs, and tests covering factory creation, adapter restoration after failures, video-only batches, and text-encoder training guards.
Add MiniMax-H3 target modality handling so the default path trains video-only, data backends can opt into joint audio-video rows, and image-latent batches drop stale cached audio instead of training an impossible soundtrack branch. Teach H3 validation and text encoding to handle CFG-distilled negative branches: caption tokens cap at 512 by default, empty negative prompts preserve positive prompt length with null token ids, and validation defers context-dependent negative prompt encoding per sample. Extend the H3 transformer/runtime with true one-frame image geometry, configurable SwiGLU gate ordering for Comfy-style checkpoints, reduced-precision AdaLN safeguards, cached static reference KV inference, musubi stream-out coverage, segmented checkpointing capability, field-registry entries, and regularisation parent audio targets. Update H3 docs, validation defaults, safety checks, collation/backend audio gating, and focused tests for image mode, target modes, reference cache, validation prompting, checkpoint controls, and audio regularisation targets.
Update the trainer test fixture to expect the default TorchDynamo mode when regional compilation is enabled, matching the current trainer path and avoiding max-autotune/cudagraph memory assumptions in that coverage.
Add separate 24G, 32G, 48G, and 80G MiniMax-H3 ConvRot INT8 LoRA example configs from the benchmark sweep, keeping the 24G preset on RamTorch and the larger presets on the measured segmented-checkpointing or no-checkpoint paths. Each preset keeps validation negative prompting disabled for the CFG-distilled default path, enables VAE slicing/tiling/temporal roll, and uses the shared 480p 39-frame FL2VA dataset config as the baseline workload.
When trainer initialization reads configuration from StateTracker, distinguish a genuinely missing sdnq_compile_mode attribute from an explicit invalid value. This avoids plain Mock-based config objects fabricating a child mock that fails SDNQ mode validation. Add regression coverage for config objects without a real sdnq_compile_mode while preserving validation for explicit compile/eager values.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
This pull request adds support and documentation for the "MiniMax H3" model across all supported languages. It updates the compatibility matrices, model lists, and quickstart guides to include MiniMax H3, specifying its license, conditions, and technical details.
MiniMax H3 Model Support
READMEfiles, including license details and usage conditions. [1] [2] [3] [4] [5] [6]Compatibility and Feature Matrix Updates
These changes ensure that users are informed about the availability, requirements, and technical specifics of MiniMax H3 in all documentation and compatibility resources.