feat: Optimize SFT dataloader with slice-based tokenization and caching#1695

SumanthRH merged 7 commits into

NovaSky-AI:mainNovaSky-AI/SkyRL:mainfrom

jlee-lila:optimize-sft-tokenizationjlee-lila/SkyRL:optimize-sft-tokenizationCopy head branch name to clipboard