Neural Magic
Neural Magic (Acquired by Red Hat) empowers developers to optimize & deploy LLMs at scale. Our model compression & acceleration enable top performance with vLLM
Pinned Loading
Repositories
Showing 10 of 116 repositories
- vllm Public Forked from vllm-project/vllm
A high-throughput and memory-efficient inference and serving engine for LLMs
- nm-vllm-omni-ent Public Forked from vllm-project/vllm-omni
A framework for efficient model inference with omni-modality models
- compressed-tensors Public Forked from vllm-project/compressed-tensors
A safetensors extension to efficiently store sparse quantized tensors on disk
- llm-compressor Public Forked from vllm-project/llm-compressor
State-of-the-art LLM compression, built for production inference with vLLM
- inspect_ai Public Forked from UKGovernmentBEIS/inspect_ai
Inspect: A framework for large language model evaluations
- vllm-envs Public
Fast disposable vLLM dev environments from any branch/commit, built on content-addressed layer caches
-
Most used topics
Loading…