adamw8bit

Here is 1 public repository matching this topic...

한국어 사전학습과 SFT를 위한 1.09B 파라미터 풀스크래치 LLM — RoPE, KV Cache, BF16·8-bit AdamW 최적화, 데이터셋 캐싱 및 학습 모니터링 지원.

python model pytorch transformer korean tinker rope adamw huggingface bf16 rmsnorm bitsandbytes swiglu adamw8bit

To associate your repository with the adamw8bit topic, visit your repo's landing page and select "manage topics."