An invisible desktop AI overlay for Claude, GPT, Gemini, and local models.

Features · Quick start · Hotkeys · Providers · Architecture · Roadmap · Contributing

---

Why Penumbra

Penumbra is a translucent always-on-top desktop overlay that puts a multi-model AI assistant one keystroke away. Snap a screenshot, drop in an audio file, or just chat — and get an answer from Claude, GPT, Gemini, or a model running on your own machine. The window is invisible to screen sharing on macOS and Windows by default, so it stays out of your demos and recordings.

Inspired by free-cluely and rebuilt from scratch with a provider-agnostic architecture, a clean pnpm monorepo, encrypted key storage, and first-class Claude support.

---

✨ Features

• Truly invisible. Frameless translucent window, always on top, hidden from screen sharing (setContentProtection), no taskbar entry.
• Multi-provider, no lock-in. Anthropic Claude, OpenAI GPT, Google Gemini, Ollama, and any OpenAI-compatible endpoint — OpenRouter, Groq, Together, Fireworks, LM Studio, LocalAI, vLLM, llama.cpp.
• Vision built in. Cmd/Ctrl + H captures the screen, sends it to a vision-capable model, and streams the answer back.
• Persistent chat history. Every turn is saved to a local SQLite database. Quit, restart, pick up where you left off. Browse, search, rename, delete past conversations.
• Local-first by default. All app state lives in a single SQLite file at userData/penumbra.db. API keys are encrypted via OS keychain. Nothing leaves your machine except your prompts to the provider you configured.
• Pluggable models. Point it at Ollama or LM Studio and Penumbra never touches the cloud.
• Global hotkeys. Toggle, capture, focus, settings. All remappable.
• Browser dev mode. The Vite dev URL works in any browser for fast UI iteration — chat persists to localStorage there.

---

📸 Screenshots

Empty state — hotkey crib sheet

Settings — pick any provider

---

🚀 Quick start

git clone https://github.com/siri1410/penumbra.git
cd penumbra
corepack enable
pnpm install
pnpm dev

On first launch the overlay appears in the top-right of your primary display. Open Settings (Cmd/Ctrl + ,), pick a provider, paste an API key, and start asking.

Prefer the browser? pnpm dev also serves the UI at http://localhost:5180. Chat works there with localStorage-backed config. Screenshots and global hotkeys are Electron-only.

---

⌨️ Hotkeys

Action	macOS	Windows / Linux
Toggle window	⌘ B	Ctrl + B
Capture & ask	⌘ H	Ctrl + H
Focus chat	⌘ J	Ctrl + J
Open settings	⌘ ,	Ctrl + ,

Remap any of them in Settings.

---

🔌 Providers

Provider	Vision	Streaming	API key	Notes
Anthropic (Claude)	✅	✅	✅	claude-opus-4-7, claude-sonnet-4-6, claude-haiku-4-5
OpenAI	✅	✅	✅	gpt-4o, gpt-4.1, o4-mini
Google Gemini	✅	✅	✅	gemini-2.0-flash, gemini-2.0-flash-thinking-exp
Ollama	✅¹	✅	—	Default URL http://localhost:11434. Try llama3.2-vision, llava.
OpenAI-compatible	✅¹	✅	opt.	OpenRouter, Groq, Together, Fireworks, LM Studio, LocalAI, vLLM…

¹ Vision requires the underlying model to support image input.

Adding a new provider

1. Implement Provider in packages/providers/src/providers/<name>.ts.
2. Register a descriptor + createProvider case in packages/providers/src/registry.ts.
3. Open a PR. See anthropic.ts for the canonical example.

If the service speaks the OpenAI wire format (/v1/chat/completions), the existing OpenAI-compatible adapter probably handles it — point it at the base URL and you're done.

---

🏗 Architecture

Penumbra is a pnpm monorepo:

penumbra/
├── apps/
│   └── desktop/          Electron + React 18 + Vite + Tailwind overlay app
└── packages/
    ├── core/             Chat session, system prompts, default config
    ├── db/               SQLite schema + repos (better-sqlite3)
    ├── providers/        Unified Provider interface + 5 adapters
    └── types/            Shared TypeScript types (no runtime deps)

┌────────────────────┐   IPC (contextIsolation)   ┌────────────────────┐
│  Electron renderer │ ◄────────────────────────► │  Electron main     │
│  React + Tailwind  │                            │  hotkeys / capture │
│  ChatSession ──────┼─── streams ────────────────│  safeStorage       │
└──────────┬─────────┘                            └─────────┬──────────┘
           │ HTTPS                                          │
           ▼                                                ▼
   ┌──────────────────────────────────────────────────────────────┐
   │  Provider adapter (Anthropic / OpenAI / Gemini / Ollama /    │
   │  OpenAI-compatible) — streams ChatChunk back to ChatSession  │
   └──────────────────────────────────────────────────────────────┘

All adapters share one Provider interface:

interface Provider {
  readonly id: ProviderId;
  readonly supportsVision: boolean;
  chat(messages: ChatMessage[], opts?: ChatOptions): AsyncIterable<ChatChunk>;
}

That's the entire extension surface.

---

🔒 Privacy & local data

Penumbra is built so your conversations stay yours.

• Single local database. All conversations, messages, configuration, and provider settings live in one SQLite file at userData/penumbra.db. Quitting and re-launching restores the most recent conversation; the drawer (≡) lists everything.
• Encrypted API keys. Keys never go into the SQLite file. They live in userData/secrets.bin, encrypted via Electron safeStorage (macOS Keychain, Windows DPAPI, Linux libsecret where available).
• No telemetry, no phone-home. The only outbound traffic is your prompts going to the provider you configured. Penumbra itself calls nothing.
• Local models supported. Point at Ollama, LM Studio, vLLM, or llama.cpp and nothing leaves your machine.
• Delete on demand. Drawer → kebab → Delete removes a conversation and cascades to its messages. "Delete all" in the drawer footer nukes everything.
• Portable. The whole DB is one file — back it up, sync it, inspect it with any SQLite tool.

Userdata locations:

• macOS: ~/Library/Application Support/@penumbra/desktop/
• Windows: %APPDATA%\@penumbra\desktop\
• Linux: ~/.config/@penumbra/desktop/

🗺 Roadmap

• Whisper-based local audio transcription
• Region-select screenshot (don't grab the whole display)
• Plugin API for custom tools / actions
• Conversation history & search
• Multi-window workspaces
• Linux Wayland screenshot pipeline
• Tool use / function calling

See issues for what's actively being worked on.

---

🤝 Contributing

PRs welcome — especially new provider adapters. Read CONTRIBUTING.md and the Code of Conduct.

Found a security issue? Please report it privately, not as a public issue.

---

📜 License

MIT © Siri Y