Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
11 changes: 6 additions & 5 deletions AGENTS.md
Original file line number Diff line number Diff line change
Expand Up @@ -6,12 +6,12 @@ Turn a character, style, or concept into a ready-to-train LoRA dataset. One refe

## Current state

_Last verified: 2026-10-24_
_Last verified: 2026-10-26_

- **Status:** in active development; main is v0.20.1 (git tag `v0.20.1`). **Every version bump must ship a GitHub Release** or the in-app update check never fires. The project was renamed twice (lora-dataset-studio → lora-distillery → dataset-deviser); older references under the previous names are stale.
- **Works:** all five stages end to end (preprocess → generate & curate → caption → export → train config); the three dataset types (character, style, concept) with their own shot plans and caption framing; local and cloud paths for every stage; gallery curation with selection carried forward between stages, including shift-click range select; advisory dedupe, quality flags and caption lint; trainer configs for ai-toolkit (incl. SDXL), kohya sd-scripts, musubi-tuner and Fizgig; opt-in private Hugging Face publish. Since 0.15.0: per-image fault isolation in ① (one bad source never ends the batch), a cooperative ⏹ Stop across ①/②/③ with documented resume paths, translated ComfyUI failures (missing node, bad input, unreachable server) and an in-app 🩺 setup check. Since 0.15.1: a ② **shot style** (default = keep the reference's own medium; eight presets + custom text) threaded into both prompt builders, the CLI, ④ metadata and ⑤'s sample prompt, a final-prompt preview, and a prev/next/save-and-next caption editor with the image on screen. Since 0.16.0: **one name box and one trigger box in the header** that ②/③/④/⑤ all read (they used to be per-tab and went stale), ④ stating the pair it stamped, an opt-in ④ **hand-off to [Idiot LoRa Builder](https://github.com/Fablestarexpanse/Idiot-Lora-Builder)** (writes its `.lora-studio/ratings.json` so its grid opens pre-triaged — nothing is launched), a ⑤ advisory when the dataset can't fill the training resolution, every bundled ComfyUI model filename overridable from `.env`, and a `cli build` preflight for the local engine. Since 0.17.0: the local ② engine is **Qwen-Image 2.1** (plain-English prompts shared with Gemini, ~26 s/shot, `LDS_QWEN21_*` filenames) instead of Qwen-Image-Edit 2511 + the Multiple-Angles LoRA, whose graph stays in `studio/comfy_workflows/legacy/`; a dead ComfyUI mid-batch now fails one shot, not the run. Since 0.17.1: the UI port is chosen by `_pick_port` in `app.py` — 7861 if bindable, else an OS-assigned free port (Windows reserves a ~500-port block around 7861), or `LDS_PORT` to pin one. Since 0.17.2: `comfy_api.upload_image` downscales a source over 90 MB before upload (ComfyUI answers 413 above its 100 MB `--max-upload-size`). Since 0.17.3: the cloud image model defaults to **`auto`** (Google shut down the `-preview` ids on 2026-06-25; `resolve_image_model` picks from the cached live list, Nano Banana 2.1 first), Groq captions use `qwen/qwen3.8-27b` (key `groq-qwen`, old key aliased), every API image goes through `config.api_image` (true mime, size cap), musubi commands include the cache steps and the per-architecture network module, ai-toolkit gains Qwen-Image 2.1 and a correct Krea 2 (Raw) preset and keeps every checkpoint, ① applies EXIF rotation, no longer 4x-upscales large JPEGs, and can jump a busy ComfyUI queue like ② (`--front` on `preprocess`/`build`). Since 0.18.0: ⑤ counts **epochs** (one checkpoint and one sample set each) with auto repeats sized to a step target and a live exposure line; every config ships a `validation/` pack (8 fixed prompts at seed 42, `validation.md`, a per-epoch score sheet); **Fizgig** is a ⑤ target (Krea 2 / Qwen Image 2.1 / Klein); ① stamps a `dd_source` provenance PNG chunk so ⑤ names upscaled sources and ④ knows real photos; ④ can hold out N real photos in a `-heldout` sibling as a likeness test. Since 0.18.1: a ⑤ optimizer dropdown, AdamW8bit by default, Prodigy (lr 1.0, its diffusion args) for ai-toolkit/musubi/kohya; Fizgig stays AdamW8bit. Since 0.19.0: ① training copies **keep their background** and the isolated copy goes to `prepped/refs/` for ② only (angle re-isolation is its own ② option, off); a header **identity policy** for characters (Identity only varies outfits and captions clothing; Signature costume keeps the outfit out of captions); four framing tiers in the shot plan and a ② coverage line with a dominance warning; a cut-off-reference warning; ② near-duplicate finder and primary-reference preview, ④ burst grouping; local ② sends several references (`LDS_QWEN21_MAX_REFS`, default 1) at `LDS_QWEN21_RESOLUTION` (1536); Gemini gets a per-shot aspect and a reference cap; an opt-in **anchor shot**; SAM3 load errors name a stale `HF_TOKEN`. Since 0.20.0: a **⚡ Quick build** tab (first tab: drop images, name, Cloud or Local, trainer; Build runs ①→②, untick, Finish runs ③→④→⑤) and `cli build` share `pipeline.build_start`/`build_finish`, and `cli build` writes ⑤ configs with `--trainer/--model`; a failed isolation keeps the photo and gives ② the whole image; builtin SAM3 retries `person` on real photos; a ComfyUI job's timeout no longer runs while it waits in the queue; `HUGGINGFACE_API_KEY` stands in when there is neither an `HF_TOKEN` nor an `hf auth login`; a caption that opens with the name in another case (`Ann` for trigger `ann`) now opens with the trigger's exact casing; `cli build` and Quick build Finish show ⑤'s undersized/upscaled warnings. Since 0.20.1: `.jfif` sources are listed (exported and uploaded as `.jpg`), ④ re-encodes BMP/TIFF to PNG because ai-toolkit silently skips them, and `validation.md` ends with a LoRA-strength sweep.
- **Status:** in active development; main is v0.21.0 (git tag `v0.21.0`). **Every version bump must ship a GitHub Release** or the in-app update check never fires. The project was renamed twice (lora-dataset-studio → lora-distillery → dataset-deviser); older references under the previous names are stale.
- **Works:** all five stages end to end (preprocess → generate & curate → caption → export → train config); the three dataset types (character, style, concept) with their own shot plans and caption framing; local and cloud paths for every stage; gallery curation with selection carried forward between stages, including shift-click range select; advisory dedupe, quality flags and caption lint; trainer configs for ai-toolkit (incl. SDXL), kohya sd-scripts, musubi-tuner and Fizgig; opt-in private Hugging Face publish. Since 0.15.0: per-image fault isolation in ① (one bad source never ends the batch), a cooperative ⏹ Stop across ①/②/③ with documented resume paths, translated ComfyUI failures (missing node, bad input, unreachable server) and an in-app 🩺 setup check. Since 0.15.1: a ② **shot style** (default = keep the reference's own medium; eight presets + custom text) threaded into both prompt builders, the CLI, ④ metadata and ⑤'s sample prompt, a final-prompt preview, and a prev/next/save-and-next caption editor with the image on screen. Since 0.16.0: **one name box and one trigger box in the header** that ②/③/④/⑤ all read (they used to be per-tab and went stale), ④ stating the pair it stamped, an opt-in ④ **hand-off to [Idiot LoRa Builder](https://github.com/Fablestarexpanse/Idiot-Lora-Builder)** (writes its `.lora-studio/ratings.json` so its grid opens pre-triaged — nothing is launched), a ⑤ advisory when the dataset can't fill the training resolution, every bundled ComfyUI model filename overridable from `.env`, and a `cli build` preflight for the local engine. Since 0.17.0: the local ② engine is **Qwen-Image 2.1** (plain-English prompts shared with Gemini, ~26 s/shot, `LDS_QWEN21_*` filenames) instead of Qwen-Image-Edit 2511 + the Multiple-Angles LoRA, whose graph stays in `studio/comfy_workflows/legacy/`; a dead ComfyUI mid-batch now fails one shot, not the run. Since 0.17.1: the UI port is chosen by `_pick_port` in `app.py` — 7861 if bindable, else an OS-assigned free port (Windows reserves a ~500-port block around 7861), or `LDS_PORT` to pin one. Since 0.17.2: `comfy_api.upload_image` downscales a source over 90 MB before upload (ComfyUI answers 413 above its 100 MB `--max-upload-size`). Since 0.17.3: the cloud image model defaults to **`auto`** (Google shut down the `-preview` ids on 2026-06-25; `resolve_image_model` picks from the cached live list, Nano Banana 2.1 first), Groq captions use `qwen/qwen3.8-27b` (key `groq-qwen`, old key aliased), every API image goes through `config.api_image` (true mime, size cap), musubi commands include the cache steps and the per-architecture network module, ai-toolkit gains Qwen-Image 2.1 and a correct Krea 2 (Raw) preset and keeps every checkpoint, ① applies EXIF rotation, no longer 4x-upscales large JPEGs, and can jump a busy ComfyUI queue like ② (`--front` on `preprocess`/`build`). Since 0.18.0: ⑤ counts **epochs** (one checkpoint and one sample set each) with auto repeats sized to a step target and a live exposure line; every config ships a `validation/` pack (8 fixed prompts at seed 42, `validation.md`, a per-epoch score sheet); **Fizgig** is a ⑤ target (Krea 2 / Qwen Image 2.1 / Klein); ① stamps a `dd_source` provenance PNG chunk so ⑤ names upscaled sources and ④ knows real photos; ④ can hold out N real photos in a `-heldout` sibling as a likeness test. Since 0.18.1: a ⑤ optimizer dropdown, AdamW8bit by default, Prodigy (lr 1.0, its diffusion args) for ai-toolkit/musubi/kohya; Fizgig stays AdamW8bit. Since 0.19.0: ① training copies **keep their background** and the isolated copy goes to `prepped/refs/` for ② only (angle re-isolation is its own ② option, off); a header **identity policy** for characters (Identity only varies outfits and captions clothing; Signature costume keeps the outfit out of captions); four framing tiers in the shot plan and a ② coverage line with a dominance warning; a cut-off-reference warning; ② near-duplicate finder and primary-reference preview, ④ burst grouping; local ② sends several references (`LDS_QWEN21_MAX_REFS`, default 1) at `LDS_QWEN21_RESOLUTION` (1536); Gemini gets a per-shot aspect and a reference cap; an opt-in **anchor shot**; SAM3 load errors name a stale `HF_TOKEN`. Since 0.20.0: a **⚡ Quick build** tab (first tab: drop images, name, Cloud or Local, trainer; Build runs ①→②, untick, Finish runs ③→④→⑤) and `cli build` share `pipeline.build_start`/`build_finish`, and `cli build` writes ⑤ configs with `--trainer/--model`; a failed isolation keeps the photo and gives ② the whole image; builtin SAM3 retries `person` on real photos; a ComfyUI job's timeout no longer runs while it waits in the queue; `HUGGINGFACE_API_KEY` stands in when there is neither an `HF_TOKEN` nor an `hf auth login`; a caption that opens with the name in another case (`Ann` for trigger `ann`) now opens with the trigger's exact casing; `cli build` and Quick build Finish show ⑤'s undersized/upscaled warnings. Since 0.20.1: `.jfif` sources are listed (exported and uploaded as `.jpg`), ④ re-encodes BMP/TIFF to PNG because ai-toolkit silently skips them, and `validation.md` ends with a LoRA-strength sweep. Since 0.21.0 (security): **Gradio 6.30** (the `<6` cap is gone; the 6.20 stuck-overlay bug does not occur on 6.30), dependency floors raised past known CVEs, every `launch()` argument in `app.LAUNCH_KWARGS` (a Host check against DNS rebinding, `.env` in `blocked_paths`, Gradio 6 run history off), Pillow limited to the PNG/JPEG/WebP/BMP/TIFF decoders, and a fix for Gradio's broken SemiBold font. Also fixed: Quick build Finish and `cli build` now use the saved Custom endpoint captioner (it failed with "no endpoint URL"; `cli build` checks it before generating), an untouched Textbox no longer reaches handlers as None under Gradio 6, and the test suite no longer overwrites `.cache/user_settings.json`.
- **In progress:** nothing half-built on main. The phased LoRA best-practice update (0.17.3–0.20.0) is complete; the optional local face model (drift flag, verified coverage) awaits a go-ahead. The attention-only toggle planned for 0.18.0 was deferred (ARCHITECTURE.md → Future ideas).
- **Known gaps / next steps:** Auto's model order (`IMAGE_MODEL_PREFERENCE`, Nano Banana 2.1 then Pro) rests on one A/B subject — see ARCHITECTURE.md; the local defaults (one reference, 1536, anchor off) likewise rest on one A/B subject. Otherwise pick up from `docs/ARCHITECTURE.md` → "Future ideas" / "Deferred", and read its *Maintainer principles* before changing anything; `gradio` is pinned `<6` because of a real stuck-loading-overlay regression — unpinning needs that verified fixed upstream; training is never launched for you and is not planned to be; the default export `output_root` writes into the repo root, so a smoke-test export leaves an untracked dataset folder to delete before committing.
- **Known gaps / next steps:** Auto's model order (`IMAGE_MODEL_PREFERENCE`, Nano Banana 2.1 then Pro) rests on one A/B subject — see ARCHITECTURE.md; the local defaults (one reference, 1536, anchor off) likewise rest on one A/B subject. Otherwise pick up from `docs/ARCHITECTURE.md` → "Future ideas" / "Deferred", and read its *Maintainer principles* before changing anything; Pillow 13 (fixes its remaining CVEs) is blocked by gradio's `pillow<13` cap and needs Python 3.11+ (see ARCHITECTURE.md, Future ideas); training is never launched for you and is not planned to be; the default export `output_root` writes into the repo root, so a smoke-test export leaves an untracked dataset folder to delete before committing.
- **Deep docs:** `docs/ARCHITECTURE.md` (module map, stage and backend detail, gotchas, backlog — the deep reference), `docs/comfyui-setup.md`. Worklogs under `docs/` are gitignored and local-only by design.

## Architecture in 60 seconds
Expand Down Expand Up @@ -61,7 +61,8 @@ python cli.py keys --setup

## Conventions & gotchas

- Python ≥3.10. Gradio ≥5.0, <6 (Gradio 6 has a loading-overlay bug).
- Python >=3.10. Gradio >=6.30, <7. Dependency floors in `requirements.txt` are security floors (setup installs without `-U`); run `pip-audit -r requirements.txt` before a release.
- Every `demo.launch()` argument lives in `app.LAUNCH_KWARGS`; `tests/test_launch_security.py` pins the localhost bind, Host check, `.env` block and run-history-off. Never add `share=True`.
- torch + torchvision installed separately by setup scripts (CUDA/CPU/macOS specific).
- `studio/env_keys.py` is the ONE implementation of reading/editing .env API keys. Per-shell versions were replaced.
- Caption encoding: `read_caption()` uses utf-8-sig + errors=replace (forgiving sidecar read).
Expand Down
61 changes: 40 additions & 21 deletions app.py
Original file line number Diff line number Diff line change
Expand Up @@ -15,12 +15,13 @@
import os
import re
import string
import warnings
from datetime import datetime
from pathlib import Path

import gradio as gr
import pandas as pd
from starlette.middleware import Middleware
from starlette.middleware.trustedhost import TrustedHostMiddleware

from studio import pipeline, shot_style
from studio import user_config as _uc_boot
Expand All @@ -40,6 +41,7 @@
from studio.config import (
CAPTIONERS,
CAPTIONERS_BY_KEY,
REPO_ROOT,
friendly_api_error,
list_images,
load_caption_model_cache,
Expand Down Expand Up @@ -1917,16 +1919,8 @@ def report(msg: str):

# ---------- layout ----------

# Gradio 5 warns that `head=` moves to `launch()` in Gradio 6 — but `launch()` does not
# accept it in 5.x, and `requirements.txt` pins `gradio<6` on purpose. Suppressed
# narrowly so a start-up console that should be empty stays empty; the move is recorded
# against lifting the gradio<6 cap in docs/ARCHITECTURE.md.
with warnings.catch_warnings():
warnings.filterwarnings("ignore", message=".*'head' parameter in the Blocks.*",
category=DeprecationWarning)
_blocks = gr.Blocks(title="Dataset Deviser", head=_PICKER_SCRIPT)

with _blocks as demo:
# Gradio 6 takes `head=` (the picker script) at launch(); see LAUNCH_KWARGS.
with gr.Blocks(title="Dataset Deviser") as demo:
gr.Markdown(
"# Dataset Deviser\n"
"Character, style, or concept → ready-to-train LoRA dataset. Every tab works standalone "
Expand Down Expand Up @@ -2847,6 +2841,13 @@ def _cap_cost(key: str, model: str, selected: list[str]) -> str:

demo.load(_check_for_update, None, update_notice)

# Gradio 6 starts a Textbox with no value= at None (Gradio 5: ""), so an untouched box
# (e.g. ③'s prefix inside a closed accordion) reached handlers as None and `.strip()` raised.
for _c in demo.blocks.values():
if isinstance(_c, gr.Textbox) and _c.value is None:
_c.value = ""


def _pick_port(preferred: int = 7861) -> int:
"""LDS_PORT if set, else ``preferred`` if bindable, else any free port.

Expand All @@ -2868,14 +2869,32 @@ def _pick_port(preferred: int = 7861) -> int:
raise OSError("No free local port; set LDS_PORT in .env.")


# Bound to localhost on purpose: no auth layer, and .env keys are reachable
# through the process. Do not expose publicly / use share=True.
# allowed_paths lets the galleries display images in user-chosen input/output
# folders on any drive (Gradio otherwise refuses paths outside the CWD/temp
# dir). It is fixed at launch, so it can't be narrowed per-request; the
# consequence is that the local file endpoint can serve any file the process
# can read. Safe ONLY because of the localhost-only, no-auth bind - see the
# Security posture note in docs/ARCHITECTURE.md.
# TrustedHostMiddleware stops DNS rebinding: Gradio's own CORS check only runs when
# the Host header is localhost, so a page on evil.example re-pointed at 127.0.0.1
# could otherwise read any of those files. blocked_paths keeps .env out regardless.
# Gradio's bundled SourceSansPro-SemiBold.woff2 is a 55-glyph subset (it has "A" and
# "B", not "C"-"z"), so bold text drew "A" in semibold and the rest in faux-bold.
# Point weight 600 at the complete Bold file Gradio also ships.
_FONT_FIX = ('@font-face { font-family: "Source Sans Pro"; font-weight: 600; '
'src: url("static/fonts/SourceSansPro/SourceSansPro-Bold.woff2") format("woff2"); }')

LAUNCH_KWARGS: dict = {
"server_name": "127.0.0.1", "inbrowser": True, "head": _PICKER_SCRIPT, "css": _FONT_FIX,
"allowed_paths": _allowed_media_paths(),
"blocked_paths": [str(REPO_ROOT / ".env")],
# Gradio 6's run history keeps every call's inputs/outputs and offers HF-bucket upload.
"run_history": False,
"app_kwargs": {"middleware": [Middleware(TrustedHostMiddleware,
allowed_hosts=["127.0.0.1", "localhost"])]},
}

if __name__ == "__main__":
# Bound to localhost on purpose: no auth layer, and .env keys are reachable
# through the process. Do not expose publicly / use share=True.
# allowed_paths lets the galleries display images in user-chosen input/output
# folders on any drive (Gradio otherwise refuses paths outside the CWD/temp
# dir). It is fixed at launch, so it can't be narrowed per-request; the
# consequence is that the local file endpoint can serve any file the process
# can read. Safe ONLY because of the localhost-only, no-auth bind above — see
# the Security posture note in docs/ARCHITECTURE.md.
demo.launch(server_name="127.0.0.1", server_port=_pick_port(), inbrowser=True,
allowed_paths=_allowed_media_paths())
demo.launch(server_port=_pick_port(), **LAUNCH_KWARGS)
Loading
Loading