Skip to content

feat: add FP8 blockwise weight quantization and FP8(E4M3) KV cache support - #586

Open
shsaihdsaiudh wants to merge 4 commits into
InfiniTensor:mainfrom
shsaihdsaiudh:feat/fp8-blockwise-quantization
Open

feat: add FP8 blockwise weight quantization and FP8(E4M3) KV cache support#586
shsaihdsaiudh wants to merge 4 commits into
InfiniTensor:mainfrom
shsaihdsaiudh:feat/fp8-blockwise-quantization

fix: log expected shutdown exceptions as warnings, not fatal

88ebee4
Select commit
Loading
Failed to load commit list.

Workflow runs completed with no jobs