Local-first AI workspace — chat, vision, embeddings, and structured JSON, powered by WebGPU in your browser
Run open-source LLMs and vision models locally — no cloud API keys, no per-token billing.
Live app: https://web-gpu-studio.vercel.app/
On-device AI Models run in your browser via WebGPU acceleration |
WebGPU Studio is a Next.js application that brings a full AI copilot workspace to the browser. It combines WebLLM (WebGPU-accelerated LLM inference via MLC), Transformers.js (vision + server-side ONNX), and the Vercel AI SDK into a single polished UI — with streaming chat, image understanding, vector embeddings, and schema-validated JSON generation. Everything runs on open-source models from Hugging Face. Chat and structured output execute on your GPU through WebGPU; vision models stream through a server route; embeddings compute entirely client-side and cache in IndexedDB. Zero API keys required for core features (Auth0 is wired but currently disabled for public access). |
Open models Llama, Qwen, Gemma, SmolVLM, gte-small |
Chat Streaming text generation with domain presets Llama 3.2 · Qwen2.5 · Gemma via WebLLM |
Vision Image upload + multimodal Q&A SmolVLM2 500M / SmolVLM 256M |
Structured JSON Typed object generation with Zod schemas Plans, specs, step-by-step output |
Embeddings 384-dim vectors, similarity search, RAG prep gte-small in-browser |
| Feature | Description |
|---|---|
| Streaming chat | Token-by-token responses with stop/cancel and markdown rendering |
| WebGPU inference | Llama 3.2, Qwen2.5, and Gemma models run in a dedicated Web Worker via @built-in-ai/web-llm |
| Vision chat | Upload images and ask questions — SmolVLM models with domain presets (documents, products, medical, safety) |
| Structured output | generateObject() with Zod schemas — get typed JSON (title, summary, steps) from natural language |
| Embeddings lab | Compare texts, build a local library, semantic search, cosine similarity matrix — all client-side |
| Domain presets | Marketing, HR, customer service, and vision-specific system prompts auto-applied per model |
| Model progress UI | Real-time weight download progress bar during first load (WebLLM + Transformers.js) |
| Dark / light theme | Collapsible sidebar, responsive mobile nav, parallax welcome hero |
| Auth-ready | Auth0 integration present (currently disabled for open public use) |
| Layer | Technology | Role |
|---|---|---|
| App | Next.js 15 App Router | Routing, API routes, SSR shell |
| UI | React 19 + TypeScript 5 | Components, hooks, type safety |
| Styling | Tailwind CSS 3 + Radix UI | Layout, sliders, selects, tooltips |
| Animation | Motion | Welcome hero and parallax cards |
| State | Zustand 5 | Sidebar context, theme |
| Layer | Technology | Where it runs |
|---|---|---|
| Chat / Structured | @built-in-ai/web-llm + MLC-LLM | Browser WebGPU (Web Worker) |
| Vision | @built-in-ai/transformers-js | Server CPU via /api/chat |
| Embeddings | @huggingface/transformers | Browser WebGPU / WASM |
| Orchestration | Vercel AI SDK 6 | streamText, generateObject, streaming |
| Validation | Zod 4 | Structured JSON schemas |
| Markdown | react-markdown | Assistant message rendering |
WebGPU gives the browser direct access to the GPU compute pipeline. For LLM inference this means:
- No server GPU cost — chat runs on the user's machine
- Privacy — prompts never leave the browser for WebLLM models
- MLC-quantized models — 4-bit weights (q4f16) fit in consumer GPU memory
- Persistent cache — model weights stored in browser cache after first download
| Model | Size | Presets |
|---|---|---|
| Llama 3.2 1B Instruct | ~1B | General, Marketing, HR, Customer Service |
| Llama 3.2 3B Instruct | ~3B | General, Marketing |
| Qwen2.5 0.5B / 1.5B / 3B | 0.5–3B | General, Marketing, HR, Customer Service |
| Gemma 2B IT | ~2B | General, Marketing |
| Model | Size | Presets |
|---|---|---|
| SmolVLM2 500M Instruct | 500M | General, Documents, Product Recognition |
| SmolVLM 256M Instruct | 256M | General, Medical, Documents, Product, Safety |
| Model | Dimensions | Use case |
|---|---|---|
| Supabase/gte-small | 384 | Semantic search, similarity, RAG indexing |
┌──────────────────────────────────────────────────────────────────────┐
│ Browser (Client) │
├──────────────────────────────────────────────────────────────────────┤
│ Next.js Studio UI │
│ ├── / (Chat) → useChat → WebLLM Worker → WebGPU │
│ ├── /vision → useChat → POST /api/chat → server CPU │
│ ├── /structured → useStructured → WebLLM → generateObject │
│ └── /embeddings → useEmbeddings → HF Transformers.js │
│ │
│ Web Workers │
│ ├── web-llm-worker.ts MLC WebWorkerMLCEngineHandler │
│ └── transformers-js-worker.ts (vision client preload) │
├──────────────────────────────────────────────────────────────────────┤
│ Shared session layer (lib/ai/) │
│ ├── webllm-session.ts singleton worker + per-model engine cache│
│ └── transformers-session.ts vision worker lifecycle │
└──────────────────────────────────────────────────────────────────────┘
│
▼
┌──────────────────────────────────────────────────────────────────────┐
│ Next.js Server (Node.js) │
│ POST /api/chat streamText + transformers-js (vision) │
│ POST /api/model/progress model download progress stream │
└──────────────────────────────────────────────────────────────────────┘
User input → useChat.sendChat()
→ getWebLLMModel(modelId) // cached MLC engine per model
→ prepareWebLLMSession() // download weights with progress
→ streamText({ model, messages }) // Vercel AI SDK
→ token stream → ChatFeed UI
The WebLLM worker (web-llm-worker.ts) hosts a WebWorkerMLCEngineHandler so inference never blocks the main thread. A module-level worker singleton survives React Strict Mode remounts.
Image upload → base64 data URL → POST /api/chat
→ dynamic import @built-in-ai/transformers-js
→ dataUrlToUint8Array() for Node.js compatibility
→ streamText({ model: SmolVLM, messages: [text + image] })
→ plain text stream back to client
Vision uses CPU on the server (device: "cpu") for stable ONNX inference; chat stays on WebGPU in the browser.
Prompt → generateObject({ model: webLLM, schema: z.object({...}) })
→ typed JSON { title, summary, steps[] }
→ rendered in Structured section with copy-to-clipboard
Text lines → @huggingface/transformers pipeline("feature-extraction", "Supabase/gte-small")
→ 384-dim vectors (mean pooling, normalized)
→ cosine similarity matrix / semantic search / localStorage library
Models cache in the browser after first download — no server bundle bloat.
| Decision | Rationale |
|---|---|
| Split inference | WebGPU for chat (GPU-heavy), server CPU for vision (ONNX stability), browser for embeddings (zero server RAM) |
| Dynamic imports | @built-in-ai/transformers-js and @huggingface/transformers loaded at runtime to keep serverless functions small |
| Worker singleton | Prevents WebLLM init hangs from Strict Mode double-mount terminating workers mid-load |
| Domain system prompts | buildDomainSystemPrompt() in models.ts — auto-injects expertise context per preset |
| Auth disabled | Auth0 wiring preserved in comments; app runs fully public without login gates |
webgpu-studio/
├── public/ # Section preview images, logos
├── src/
│ ├── app/
│ │ ├── (studio)/ # Main studio layout + pages
│ │ │ ├── page.tsx # Chat (home)
│ │ │ ├── vision/ # Vision chat
│ │ │ ├── structured/ # JSON generation
│ │ │ ├── embeddings/ # Embedding lab
│ │ │ └── _components/ # Sidebar, chat feed, sections
│ │ ├── api/
│ │ │ ├── chat/route.ts # Vision streaming endpoint
│ │ │ └── model/progress/ # Download progress
│ │ ├── web-llm-worker.ts # WebGPU LLM worker entry
│ │ └── transformers-js-worker.ts
│ ├── hooks/ # useChat, useEmbeddings, useStructured, useModel
│ ├── lib/
│ │ ├── ai/
│ │ │ ├── models.ts # Presets, model IDs, domain prompts
│ │ │ ├── webllm-session.ts # Worker + engine lifecycle
│ │ │ └── transformers-session.ts
│ │ └── utils/ # Embeddings math, storage, logger
│ └── contexts/ # Theme, sidebar
└── next.config.ts # Webpack aliases, file tracing excludes
Try it live: https://web-gpu-studio.vercel.app/
- Node.js 20+
- Chrome 113+ or Edge 113+ (WebGPU support required for chat models)
- ~2–4 GB free disk/RAM for model weight downloads (varies by model)
git clone https://github.com/samarthshukla6/WebGPU-Studio.git
cd WebGPU-Studio
npm install
npm run devOpen http://localhost:3000.
npm run build
npm startAuth is currently disabled, but the codebase supports Auth0 v4 when re-enabled:
AUTH0_SECRET='use openssl rand -hex 32'
APP_BASE_URL='http://localhost:3000'
AUTH0_DOMAIN='your-tenant.auth0.com'
AUTH0_CLIENT_ID='your-client-id'
AUTH0_CLIENT_SECRET='your-client-secret'- Chat — Select a model from the dropdown, type a message, and watch tokens stream in. Use the welcome hero for quick starts.
- Vision — Navigate to Vision, upload an image, and ask questions about it.
- Structured JSON — Describe the JSON you need; get a typed object with title, summary, and steps.
- Embeddings — Paste multiple lines of text, embed them, compare similarity, or build a searchable library.
First model load downloads weights from Hugging Face / MLC — progress appears in the UI. Subsequent visits use cached weights.
| Project | Description |
|---|---|
| Robo Physics Simulator | Sibling repo — browser MuJoCo WASM simulator for the SO-101 robot arm (live demo) |
| WebLLM | MLC-LLM browser inference engine |
| Transformers.js | Hugging Face ONNX/WASM inference in JS |
| Vercel AI SDK | Unified streaming and structured output API |
- Built-in AI —
transformers-jsandweb-llmpackages - MLC — WebGPU LLM compilation and MLC model format
- Hugging Face — Model hosting and Transformers.js
- Vercel — Next.js and AI SDK
- Meta, Alibaba Qwen, Google Gemma — Open-weight model families
Built for the local-first AI community
Live Demo · Report an Issue · Robo Physics Simulator · WebGPU Spec