Pinned Loading

1. Guide to optimizing inference perfor... Guide to optimizing inference performance of large MoE models across CPU+GPU using llama.cpp and its derivatives

   1

   # CPUmaxxing with GPU acceleration in llama.cpp

   2

   3

   ## Introduction

   4

   5

   So you want to try one of those fancy huge mixture-of-experts (MoE) models locally? Well, whether you've got a gaming PC or a large multi-GPU workstation, we've got you covered. As long as you've downloaded enough RAM beforehand.

2. ds-llm-webui ds-llm-webui Public

   A simple tool-use assistant for local LLMs powered by TabbyAPI

   TypeScript 10

3. ds-med-helper ds-med-helper Public

   A Streamlit-based web UI for physician medical documentation assistance using OpenAI API compatible LLM and ASR endpoints.

   Python

4. ds-medasr-server ds-medasr-server Public

   OpenAI-compatible MedASR Transcription Server

   Python

5. LLM-datagen LLM-datagen Public

   Scripts for generating synthetic LLM training data

   Python 5 1

6. LLM-notebooks LLM-notebooks Public

   Jupyter notebooks for cloud-based usage

   Jupyter Notebook 10 3