Skip to content

README.md

Latest commit

 

History

History
264 lines (186 loc) · 3.84 KB

README.md

File metadata and controls

264 lines (186 loc) · 3.84 KB
Raw

AutoLoraFactory

Automated Stable Diffusion LoRA pipeline with dataset extraction, training, generation, and evaluation.

AutoLoraFactory is a Python-based automation pipeline for creating high-quality LoRA models from videos.

It automatically:

  • extracts frames from videos
  • detects and crops faces
  • removes duplicate images
  • selects high-quality samples
  • generates captions
  • trains LoRA models
  • generates images
  • evaluates outputs with CLIP
  • selects the best checkpoint automatically

Features

Dataset Automation

  • Video frame extraction with FFmpeg
  • Face detection using InsightFace
  • Automatic face crop & resize
  • Duplicate image removal
  • Quality-based image filtering
  • Automatic caption generation

LoRA Training

  • Fully automated LoRA training
  • Hyperparameter search
  • Automatic checkpoint evaluation
  • Best model selection
  • Batch experiment support

Image Evaluation

  • CLIP similarity scoring
  • Simple aesthetic scoring
  • Automatic ranking system

Pipeline Overview

Video (.mp4)
    ↓
Frame Extraction
    ↓
Face Detection & Crop
    ↓
Duplicate Removal
    ↓
Quality Filtering
    ↓
Caption Generation
    ↓
LoRA Training
    ↓
Image Generation
    ↓
CLIP Evaluation
    ↓
Best LoRA Selection

Project Structure

AutoLora/
├── input/                 # Input videos
├── dataset/
│   └── 10_person/         # Training dataset
├── lora/                  # Trained LoRA models
├── outputs/               # Generated images
├── best_lora/             # Best checkpoints
├── pipeline/
│   ├── extract.py
│   ├── train.py
│   ├── generate.py
│   ├── evaluate.py
│   └── utils.py
├── run.py
├── requirements.txt
├── LICENSE
└── README.md

Requirements

Environment

  • Python 3.10+
  • CUDA GPU
  • FFmpeg
  • Stable Diffusion WebUI
  • kohya_ss / sd-scripts

Python Packages

pip install \
  torch \
  torchvision \
  diffusers \
  transformers \
  insightface \
  opencv-python \
  pillow \
  tqdm \
  imagehash \
  xformers

Setup

1. Install FFmpeg

Make sure FFmpeg is installed and added to PATH.

ffmpeg -version

2. Prepare Stable Diffusion Model

Place your base model here:

C:/stable-diffusion-webui/models/Stable-diffusion/

Example:

v1-5-pruned-emaonly.safetensors

3. Install sd-scripts

GitHub:

https://github.com/kohya-ss/sd-scripts

Usage

Step 1 — Add Videos

Place .mp4 files inside:

input/

Step 2 — Run Pipeline

python run.py

The pipeline automatically handles:

  1. frame extraction
  2. face detection & crop
  3. duplicate removal
  4. quality filtering
  5. caption generation
  6. LoRA training
  7. image generation
  8. CLIP evaluation
  9. best checkpoint selection

Hyperparameter Search

Edit SEARCH in run.py.

SEARCH = {
    "dim": [8, 16, 32],
    "lr": [1e-4, 5e-5],
    "steps": [1000, 2000]
}

Evaluation Formula

score = CLIP * 0.7 + aesthetic * 0.3

Current metrics:

  • CLIP similarity score
  • simple brightness-based aesthetic score

Example Workflow

python run.py

That's it.

The pipeline handles the rest automatically.

Notes

  • CUDA environment required
  • Windows paths are currently hardcoded
  • Designed for personal research / experimentation
  • Path cleanup and config externalization recommended

Disclaimer

This project was built in 2 days.

I do not accept ugly code.
I will probably not maintain this repository.

Use it, modify it, break it — do whatever you want.

This project is intended for research and educational purposes only.
Users are responsible for complying with applicable laws and platform policies.

License

MIT License

Author

CyberSoulWing