Locally fine-tuned Russian models for the self-hosted AGmind RAG stack.
Teacher distillation, quantization to GGUF, inference on AMD (Vulkan, no CUDA) via
llama.cpp. Commercial-OK licenses, fully local. Trained on a single consumer GPU
(RTX 5090), deployed on AMD Strix Halo. Three models shipped so far — a document
splitter, query expansion, and a retrieval embedder.
| Project | What it does | Base | Key numbers | Status |
|---|---|---|---|---|
| agmind-rag-splitter-ru · 🤗 | Context-aware document splitter for RAG: semantic chunks, tables and code kept whole, lossless reconstruction | t-tech/T-lite-it-2.1 (Qwen3-8B) |
100% valid JSON, boundary-F1@±1 0.821 | ✅ on HF |
| qmd-query-expansion-ru · 🤗 | Search-query expansion for qmd (hyde/lex/vec, word forms for BM25); fixes the stock model's English hallucinations on Russian | Qwen/Qwen3-1.7B |
reward-score 94.8; 100% format | ✅ on HF |
| strizh-ru-retriever · 🤗 · GGUF | RU-first no-prefix dense retriever: 4 layers, distilled by layer-pruning + BGE-M3 hard negatives. A speed/drop-in specialist — beats rubert-tiny2 on Russian, below USER2-small/bge-m3 on quality | deepvk/RuModernBERT-small |
RU recall@10 0.75 (clean) / EN nDCG@10 0.34; 24M, fast on Strix Vulkan (slower on CPU) | ✅ on HF |
| next… | (150M embedder, reranker, grounded-QA) | planned |
- Teacher distillation. A strong model labels the task or defines the target space — no manual annotation, with strict validation gates, dedup and retries.
- One RTX 5090 (32 GB). Generative models: bf16 LoRA (+rsLoRA), response-only loss. Embedders: teacher layer-pruning (warm-start) + contrastive (MNRL) on hard negatives.
- Cyrillic bases. Russian-tuned tokenizers are more economical than vanilla ones.
- Deploy anywhere. GGUF →
llama.cppVulkan on AMD Strix Halo, no CUDA.
AGmind-ML/
├── agmind-rag-splitter-ru/ # document splitter
├── qmd-query-expansion-ru/ # query expansion
└── strizh-ru-retriever/ # retrieval embedder
Each project has its own README.md: task, method, metrics, reproduction.