Small, focused tools for building systems you can actually trust in production - verifiable retrieval, agent guardrails, drift and cost monitoring, rate limiting and consensus, a few CI gates that turn "we should really check that" into "the build fails if we don't," and - to tie it together - full-stack apps, a set of mobile and on-device-AI apps, and the odd GPU visualization.

Most of these started as a problem I hit at work and couldn't find a clean, small answer for, so I wrote one. Each project is real, tested code with a README that explains the why - and, where it earns one, a design doc on the trade-offs and benchmarks with actual numbers. Where something is a scaffold or a deliberate shortcut, I say so instead of pretending it's production-grade.

A lot of the cores are written three times - Python, C#, and Java. That's not for show: they're plain algorithms (a Merkle proof, a point-in-time join, a drift statistic), and porting them keeps me honest that the logic is the logic, not a trick of one language's libraries.

New to all this? llm-in-production is the field guide that ties these tools together - short notes on the seven things that will bite an LLM feature in production (cost, evals, prompt injection, drift, grounding, rate limits, agent guardrails), each with the pattern that fixes it, a runnable example, and a link to the tool below that implements it properly. If you only read one file, read its pre-launch checklist.

Grouped by what each thing is. Anything with a [live demo] runs in the browser - no install. When a new repo ships, it goes under the right heading below.

The LLM proposes and reasons; deterministic code validates and executes. Apps, services, and runtimes for building, running, watching, and governing agents.

• agent-canvas - a visual, type-safe platform for designing, running, debugging and evaluating ai-agent workflows - the llm proposes, deterministic code validates and executes
• AgentForge - register, monitor, and deeply compare ai agents - a seeded simulation engine, a comparison cockpit with radar + trade-off insights, and a spatial agent map, all in the browser (live demo)
• agent-trace - visual timeline and replay for agent runs: see where a run spent time, tokens, and money, then diff two runs (live demo)
• agent-run-dashboard - a small full-stack dashboard (fastapi + react) for recording ai agent runs and watching cost, tokens, and failures over time
• operations-agent - a multi-agent operations assistant - the planner proposes, deterministic code validates and executes, and dangerous actions wait for human approval (typescript, next.js, postgres)
• incident-commander - an autonomous incident-response agent in go - state machine, concurrent tools, a safety gate, mcp, and an eval harness, with the llm proposing and deterministic code executing
• infra-optimizer - an ai infrastructure optimizer in rust - rust computes and validates every plan and enforces every constraint; the llm only chooses among candidates already proven safe (property tests, criterion benchmarks, mcp)
• guardianforge - runtime governance for multi-agent systems, in go and c# - watch agent fleets, enforce policy, score anomalies and trust, intervene, and keep a hash-chained audit trail
• knowledge-workspace - an evidence-grounded knowledge-graph workspace - the llm plans the query, deterministic code traverses the graph and cites its sources (graph rag with per-claim citations, typescript/next.js)
• meeting-execution - turn meeting transcripts into decisions, owners, deadlines and executed actions - the llm extracts and proposes, deterministic code validates and executes, external actions wait for approval (typescript, next.js)
• mobile-assistant - an offline-first react native / expo ai life assistant - the ai proposes a plan, deterministic code fits it to your time window and reserves a travel buffer, with conflict-resolved sync
• repo-index - turn a git repo into a grounded, citable knowledge base - deterministic code-aware indexing + retrieval, the model answers only from retrieved chunks and every claim cites file:line; one engine drives a cli, a github action, and an @mention bot (live demo)

Small, drop-in libraries and CI gates for the unglamorous parts - grounding, cost, quotas, injection, drift, breaking changes - each solving one problem well.

• LedgerRAG - verifiable RAG with a tamper-evident cryptographic ledger - every answer ships a proof (python/c#/java, design doc + benchmarks)
• PromptShield - firewall for llm apps: block prompt injection inbound, redact pii/secrets outbound
• AgentGuard - zero-trust runtime sandbox for tool-calling ai agents: least-privilege policy + signed audit log
• TokenLens - attribute llm cost and latency to feature/tenant/model, with budget and anomaly gates (python/c#/java, design doc + benchmarks)
• QuotaGate - rate limiter for llm api traffic: per-model token & request budgets across sliding windows, per tenant/user scope, reserve-then-reconcile (python/c#/java, design doc + benchmarks)
• EvalForge - eval-driven ci gate for llm quality - fail the build when a prompt change regresses
• SchemaGuard - catch breaking api/schema changes at the pull request, not in production
• DriftWatch - data-drift detection (psi + kl-divergence) for ml monitoring, in three languages
• FeatureVault - a mini feature store with point-in-time-correct joins that never leak the future
• ChaosMeshLite - resilience testing as a ci slo gate: inject faults, assert the slo still holds
• DeployKit - one command to deploy an llm app into any cloud or on-prem, secure by default
• spec-decode - speculative decoding from scratch, with a numeric proof it's exact: a draft model proposes tokens, the target verifies them in one pass, and the emitted distribution provably equals sampling the target alone (python, honest speedup benchmarks)
• metamorph - metamorphic (property-based) testing for llms: assert the invariants a correct answer must hold under paraphrase, option-reorder, distraction, and negation, then shrink any violation to a minimal failing prompt
• honest-transformer - a tiny transformer forward pass with bit-identical logits in python, c#, and java - custom deterministic exp/tanh, explicit reduction order, and a conformance harness that diffs every 64-bit output; the arithmetic proven equal, not trusted
• batch-invariant - batch-invariant llm inference: shows the reduction-order bug that flips a decoded token depending on who else is in the batch, and a kernel that provably fixes it

On-device, privacy-first Flutter apps. The first six are interaction and UI concepts; the last four put small, deterministic AI on the device. Each has a tested pure-Dart core and runs live in the browser.

• IntentCanvas - an intent-first mobile home: express a goal in plain language and get a living workspace of modules instead of an app grid, from a deterministic on-device intent engine (live demo)
• MorphUI - a real-time generative interface engine: the ui continuously morphs its layout, density, and components between modes as context changes (live demo)
• DepthFlow - a spatial depth + soft-glass interface for flat screens: content on depth planes, pointer/tilt parallax, and depth-driven blur/scale/shadow - non-ar, 60fps (live demo)
• ThumbSphere - a thumb-zone-first design system: ambient content up top, all primary interaction in a curved bottom sphere along the natural thumb arc, with a tested reachability model (live demo)
• AuraSurface - an emotion-responsive visual language: the ui adapts tone, motion and density to on-device signals, subtly and transparently, with a tested affect engine (live demo)
• SenseGuard - an adaptive accessibility system: the ui transforms in real time across text, contrast, spacing, targets, motion and language, from a profile plus live conditions (live demo)
• AetherMesh - on-device personal intelligence: local specialist agents (calendar, focus, comms, finance, health) collaborate on your intent and fuse a unified response (live demo)
• VitalSwarm - an on-device continuous health companion: multi-signal features, anomaly detection against personal baselines, and calm fused insights - fully local (live demo)
• CrossForge - a privacy-respecting multi-device personal mesh: device discovery, secure pairing, and consent-gated context and task hand-off across your own devices (live demo)
• PolicyLens - a personal on-device ai governance layer: user-defined policies, allow/warn/block evaluation with explanations, and a readable audit trail (live demo)

Weekend attempts to really understand the systems I rely on, by building the smallest honest version of each - and, where the behavior is a plain algorithm, porting it across languages to keep the logic honest.

• coracle - a small, readable raft consensus implementation - leader election + log replication with a deterministic simulator
• flotilla - run thousands of independent raft consensus groups on one set of nodes - a shared scheduler instead of a thread per group, and cross-group rpc batching instead of an rpc per group
• ConsistentHash - a consistent-hash ring with virtual nodes in three languages - minimal remap on membership change
• deterministic-sim-testing - deterministic simulation testing for distributed code - replay any run from a single 64-bit seed, and shrink a failing fault schedule to a minimal reproducer (foundationdb / tigerbeetle style)
• durable-execution - durable execution - workflows written as ordinary code that survive crashes by replaying an append-only history, so a completed step never runs twice (temporal style)
• FlowForge - durable workflow execution engine that survives worker crashes, duplicate messages and retries; interchangeable workers in six languages

• gpu-flock - thousands of boids computed and rendered entirely on the gpu with webgpu compute shaders, with an automatic webgl2 fallback so it runs anywhere - zero dependencies, no build step (live demo)

• llm-in-production - field notes on shipping llm features you can trust: the seven problems that bite in production, each with a pattern, a runnable example, and the tool that fixes it - plus a pre-launch checklist

Also maintained, outside this org: flatwire - streaming serialization that keeps memory flat and time linear, with one identical API across Python, Node, .NET, Rust, Go, and Java (published to PyPI, npm, crates.io, NuGet, Maven Central, and Go).

• A README that explains the why, not just the how - the problem it solves and where it stops.
• Tests that actually assert the hard part - and, for the systems repos, chaos or stress suites that prove behavior under partitions, packet loss, memory pressure, and out-of-order input.
• A DESIGN.md where the design has trade-offs worth defending - constraints, decisions, and explicit non-goals.
• BENCHMARKS.md with real, reproducible numbers where performance is a claim - measured on a plain machine from a committed script, with graphs, compared against an honest baseline (e.g. consistent hashing vs hash % N, a bounded window vs an exact log).

Most repos follow the same shape, so the commands are predictable:

• Python libraries / cores: pip install -r requirements.txt (or pip install pytest) then pytest -q.
• Tri-language cores (Merkle proofs, rings, limiters, drift stats): the same behavior in three languages - pytest -q, dotnet test (.NET 10), and mvn test (JDK 17+).
• Web / UI (e.g. agent-trace): npm ci then npm run dev for the app, npm test for the suite, npm run build for a static bundle. Some ship a live demo on GitHub Pages.
• Go / Rust / single-language JVM & .NET services (e.g. incident-commander, infra-optimizer, durable-execution, flotilla): the usual per-toolchain commands - go test ./... and go run, cargo test and cargo run, mvn test (JDK 17+), and dotnet test (.NET 10).
• Flutter apps: flutter pub get, then flutter test for the suite and flutter run for the app (-d chrome for web). Each also ships a live demo on GitHub Pages.
• Benchmarks, where a repo has them: pip install -r bench/requirements.txt then python bench/benchmark.py, which writes graphs and a JSON summary under bench/results/.

Parag Sawant - linkedin.com/in/paragsawant