Skip to content

Add Qwen text encoder path override - #2985

Merged
bghira merged 2 commits into
mainfrom
agent/add-qwen-text-encoder-path
Aug 3, 2026
Merged

Add Qwen text encoder path override#2985
bghira merged 2 commits into
mainfrom
agent/add-qwen-text-encoder-path

Conversation

@bghira

@bghira bghira commented Aug 3, 2026

Copy link
Copy Markdown
Owner

Summary

Adds qwen_text_encoder_model_name_or_path as a first-class option for Qwen text encoder components.

  • exposes the option in the WebUI/CLI field registry and env-file mapping
  • routes shared and custom Qwen text encoder loaders through the override when a model has one Qwen text encoder
  • logs a warning and ignores the override for model families that define multiple Qwen text encoders
  • updates OPTIONS and FLUX.2 quickstart docs, including existing translations

Why

Qwen-based models currently load their Qwen text encoder from the model package or an upstream Qwen repository. Users with a local or alternate Qwen-compatible text encoder need a supported config path instead of replacing files inside the Hugging Face cache.

Validation

  • python -m unittest tests.test_qwen_text_encoder_override tests.test_hunyuanvideo_model tests.test_ace_step_model tests.test_acestep_lora_targets tests.test_krea2_model tests.test_anima_model tests.test_boogu_image_model tests.test_kandinsky5_models tests.test_model_field_registry tests.test_parser_type_override -v
  • python -m compileall -q on touched Python files
  • git diff --cached --check
  • staged file scan for local absolute user paths before commit returned no matches

Note: the separate feature worktree does not contain its own .venv, so local hooks that hardcode .venv/bin/python could not complete there. The validation above was rerun successfully using the project venv from the main checkout before committing.

@bghira
bghira marked this pull request as ready for review August 3, 2026 09:01
@bghira
bghira requested a review from Copilot August 3, 2026 09:01

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Adds a first-class configuration override (qwen_text_encoder_model_name_or_path) so users can point Qwen-based model families at an alternate/local Qwen-compatible text encoder without modifying the Hugging Face cache, and wires this through relevant model loaders and docs.

Changes:

  • Introduces qwen_text_encoder_model_name_or_path across the config surface area (WebUI field registry + env mapping) and routes Qwen text encoder resolution through it.
  • Updates multiple Qwen-using model families to honor the override (and to ignore + warn when multiple Qwen encoders exist).
  • Adds targeted unit tests and updates OPTIONS + FLUX.2 quickstart documentation (including existing translations).

Reviewed changes

Copilot reviewed 29 out of 29 changed files in this pull request and generated 1 comment.

Show a summary per file
File Description
tests/test_qwen_text_encoder_override.py New unit coverage for override path/subfolder resolution, warnings, CLI parsing, and env mapping.
tests/test_hunyuanvideo_model.py Verifies HunyuanVideo prefers the new Qwen override over the legacy path.
tests/test_acestep_lora_targets.py Extends test config with the new override field.
tests/test_ace_step_model.py Adds coverage ensuring Ace Step v15 layout uses the override.
simpletuner/simpletuner_sdk/server/services/field_registry/sections/model.py Registers the new WebUI/CLI field for the Qwen text encoder override.
simpletuner/helpers/models/common.py Centralizes Qwen override detection, warnings, subfolder/path resolution helpers, and integrates them into tokenizer/text encoder loading.
simpletuner/helpers/models/hunyuanvideo/model.py Prefers qwen_text_encoder_model_name_or_path when selecting the Qwen text encoder source.
simpletuner/helpers/models/flux2/model.py Allows Klein Qwen3 text encoder loader to use the new override as the preferred explicit path.
simpletuner/helpers/models/ace_step/model.py Allows v15 layout to resolve even if shared tokenizer dir is absent when an override is provided; uses override as tokenizer path.
simpletuner/helpers/models/anima/model.py Routes Anima Qwen tokenizer/text encoder selection through the override (including single-file support path).
simpletuner/helpers/models/ideogram/model.py Uses override as Qwen source and clears subfolders when override is set.
simpletuner/helpers/models/longcat_image/model.py Loads text processor from override root when provided; otherwise uses model subfolder layout.
simpletuner/helpers/models/qwen_image/model.py Routes processor loading through shared Qwen processor path/subfolder resolvers.
simpletuner/helpers/models/mageflow/model.py Routes processor loading through shared Qwen processor path/subfolder resolvers.
simpletuner/helpers/models/krea2/model.py Routes processor loading through shared Qwen processor path/subfolder resolvers.
simpletuner/helpers/models/boogu_image/model.py Routes processor loading through shared Qwen processor path/subfolder resolvers.
simpletuner/helpers/configuration/env_file.py Adds env var → CLI arg mapping for the new override.
documentation/OPTIONS.md Documents the new CLI option and updates autogenerated usage/options excerpts.
documentation/OPTIONS.zh.md Translation update for the new option.
documentation/OPTIONS.ja.md Translation update for the new option.
documentation/OPTIONS.pt-BR.md Translation update for the new option.
documentation/OPTIONS.es.md Translation update for the new option.
documentation/OPTIONS.hi.md Translation update for the new option.
documentation/quickstart/FLUX2.md Updates Klein guidance to reference the new Qwen override option.
documentation/quickstart/FLUX2.zh.md Translation update for Klein guidance.
documentation/quickstart/FLUX2.ja.md Translation update for Klein guidance.
documentation/quickstart/FLUX2.pt-BR.md Translation update for Klein guidance.
documentation/quickstart/FLUX2.es.md Translation update for Klein guidance.
documentation/quickstart/FLUX2.hi.md Translation update for Klein guidance.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

@bghira
bghira merged commit e368e9a into main Aug 3, 2026
4 checks passed
@bghira
bghira deleted the agent/add-qwen-text-encoder-path branch August 3, 2026 15:29
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants