Add Qwen text encoder path override - #2985
Merged
Merged
Conversation
Contributor
There was a problem hiding this comment.
Pull request overview
Adds a first-class configuration override (qwen_text_encoder_model_name_or_path) so users can point Qwen-based model families at an alternate/local Qwen-compatible text encoder without modifying the Hugging Face cache, and wires this through relevant model loaders and docs.
Changes:
- Introduces
qwen_text_encoder_model_name_or_pathacross the config surface area (WebUI field registry + env mapping) and routes Qwen text encoder resolution through it. - Updates multiple Qwen-using model families to honor the override (and to ignore + warn when multiple Qwen encoders exist).
- Adds targeted unit tests and updates OPTIONS + FLUX.2 quickstart documentation (including existing translations).
Reviewed changes
Copilot reviewed 29 out of 29 changed files in this pull request and generated 1 comment.
Show a summary per file
| File | Description |
|---|---|
| tests/test_qwen_text_encoder_override.py | New unit coverage for override path/subfolder resolution, warnings, CLI parsing, and env mapping. |
| tests/test_hunyuanvideo_model.py | Verifies HunyuanVideo prefers the new Qwen override over the legacy path. |
| tests/test_acestep_lora_targets.py | Extends test config with the new override field. |
| tests/test_ace_step_model.py | Adds coverage ensuring Ace Step v15 layout uses the override. |
| simpletuner/simpletuner_sdk/server/services/field_registry/sections/model.py | Registers the new WebUI/CLI field for the Qwen text encoder override. |
| simpletuner/helpers/models/common.py | Centralizes Qwen override detection, warnings, subfolder/path resolution helpers, and integrates them into tokenizer/text encoder loading. |
| simpletuner/helpers/models/hunyuanvideo/model.py | Prefers qwen_text_encoder_model_name_or_path when selecting the Qwen text encoder source. |
| simpletuner/helpers/models/flux2/model.py | Allows Klein Qwen3 text encoder loader to use the new override as the preferred explicit path. |
| simpletuner/helpers/models/ace_step/model.py | Allows v15 layout to resolve even if shared tokenizer dir is absent when an override is provided; uses override as tokenizer path. |
| simpletuner/helpers/models/anima/model.py | Routes Anima Qwen tokenizer/text encoder selection through the override (including single-file support path). |
| simpletuner/helpers/models/ideogram/model.py | Uses override as Qwen source and clears subfolders when override is set. |
| simpletuner/helpers/models/longcat_image/model.py | Loads text processor from override root when provided; otherwise uses model subfolder layout. |
| simpletuner/helpers/models/qwen_image/model.py | Routes processor loading through shared Qwen processor path/subfolder resolvers. |
| simpletuner/helpers/models/mageflow/model.py | Routes processor loading through shared Qwen processor path/subfolder resolvers. |
| simpletuner/helpers/models/krea2/model.py | Routes processor loading through shared Qwen processor path/subfolder resolvers. |
| simpletuner/helpers/models/boogu_image/model.py | Routes processor loading through shared Qwen processor path/subfolder resolvers. |
| simpletuner/helpers/configuration/env_file.py | Adds env var → CLI arg mapping for the new override. |
| documentation/OPTIONS.md | Documents the new CLI option and updates autogenerated usage/options excerpts. |
| documentation/OPTIONS.zh.md | Translation update for the new option. |
| documentation/OPTIONS.ja.md | Translation update for the new option. |
| documentation/OPTIONS.pt-BR.md | Translation update for the new option. |
| documentation/OPTIONS.es.md | Translation update for the new option. |
| documentation/OPTIONS.hi.md | Translation update for the new option. |
| documentation/quickstart/FLUX2.md | Updates Klein guidance to reference the new Qwen override option. |
| documentation/quickstart/FLUX2.zh.md | Translation update for Klein guidance. |
| documentation/quickstart/FLUX2.ja.md | Translation update for Klein guidance. |
| documentation/quickstart/FLUX2.pt-BR.md | Translation update for Klein guidance. |
| documentation/quickstart/FLUX2.es.md | Translation update for Klein guidance. |
| documentation/quickstart/FLUX2.hi.md | Translation update for Klein guidance. |
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Adds
qwen_text_encoder_model_name_or_pathas a first-class option for Qwen text encoder components.Why
Qwen-based models currently load their Qwen text encoder from the model package or an upstream Qwen repository. Users with a local or alternate Qwen-compatible text encoder need a supported config path instead of replacing files inside the Hugging Face cache.
Validation
python -m unittest tests.test_qwen_text_encoder_override tests.test_hunyuanvideo_model tests.test_ace_step_model tests.test_acestep_lora_targets tests.test_krea2_model tests.test_anima_model tests.test_boogu_image_model tests.test_kandinsky5_models tests.test_model_field_registry tests.test_parser_type_override -vpython -m compileall -qon touched Python filesgit diff --cached --checkNote: the separate feature worktree does not contain its own
.venv, so local hooks that hardcode.venv/bin/pythoncould not complete there. The validation above was rerun successfully using the project venv from the main checkout before committing.