High-voltage synchronization for local-first apps.
Capacitor is a distributed, persistent, and sub-millisecond caching layer for the Cuprite Flux engine. It provides high availability and zero-latency local reads/writes by utilizing a local-first architecture synchronized via a background replication log and gossip-based discovery.
- Local-First Performance: All operations (
Get,Set,Increment) are performed against a sharded in-memory cache backed by local BadgerDB persistence. The network is never on the hot path. - Hybrid Logical Clocks (HLC): Ensures causality-preserving order for distributed updates without requiring perfect clock synchronization across nodes.
- Eventual Consistency via CRDTs (see the Conflict Resolution Guide):
- Registers: Uses Last-Write-Wins (LWW) based on HLC timestamps.
- Counters: Implements state-based PN-Counters for idempotent increments.
- Sliding Windows: Distributed windowed counters with automated pruning.
- Asynchronous Replication: A binary circular Delta Log ensures high-throughput, low-latency propagation of updates between peers.
- Secure by Default: Supports mTLS for replication streams and shared-secret authentication for the gossip layer.
- Observability: Built-in metrics tracking for end-to-end replication latency and operation performance.
Capacitor is designed for high-throughput environments where read/write latency is critical. For a complete deep-dive into internal modules, data flows, and subsystem diagrams, see the Architecture Guide:
- Write Path: When a write occurs, it is committed to the sharded in-memory cache, appended to an in-memory binary Delta Log, and asynchronously flushed to the local BadgerDB database.
- Discovery: Nodes use the SWIM protocol (via HashiCorp Memberlist) to discover peers and maintain cluster membership.
- Sync Path: Background replicators stream entries from the Delta Log to peers over TCP/TLS.
- Conflict Resolution: Received updates are merged into the local store using HLC-based conflict resolution, ensuring all nodes eventually converge to the same state.
| Operation | Latency (ns/op) | Latency (ms) |
|---|---|---|
| Set | ~226 | 0.00022 ms |
| Get | ~21 | 0.00002 ms |
| Get (Scan) | ~29 | 0.00003 ms |
| Increment | ~409 | 0.00041 ms |
| Exists | ~19 | 0.00002 ms |
| Delete | ~179 | 0.00018 ms |
| SetAdd | ~211 | 0.00021 ms |
| SetRemove | ~545 | 0.00055 ms |
| SetIsMember | ~220 | 0.00022 ms |
| SetMIsMember | ~812 | 0.00081 ms |
| SetCard | ~6536 (1k set size) | 0.00654 ms |
| SetMembers | ~293947 (1k set size) | 0.29395 ms |
| SetPop | ~11447 (1k set size) | 0.01145 ms |
| SetRandMember | ~10294 (1k set size) | 0.01029 ms |
| SetMove | ~238 | 0.00024 ms |
| SortedSetAdd | ~765 | 0.00077 ms |
| SortedSetRemove | ~480 | 0.00048 ms |
| SortedSetScore | ~227 | 0.00023 ms |
| SortedSetCard | ~16 | 0.00002 ms |
| SortedSetRank | ~280 (1k set size) | 0.00028 ms |
| SortedSetCount | ~1000 (1k set size) | 0.00100 ms |
| SortedSetIncrementBy | ~780 | 0.00078 ms |
| SortedSetRangeScan | ~285000 (1k set size) | 0.28500 ms |
| SortedSetRangeByScoreScan | ~110000 (1k set size) | 0.11000 ms |
| SortedSetRevRangeScan | ~305000 (1k set size) | 0.30500 ms |
| SortedSetRevRangeByScoreScan | ~111000 (1k set size) | 0.11100 ms |
| SortedSetPopScan | ~930 (1k set size) | 0.00093 ms |
| MapSet | ~720 | 0.00072 ms |
| MapGetScan | ~70 | 0.00007 ms |
| MapGetMScan | ~142 | 0.00014 ms |
| MapGetAllScan | ~538 | 0.00054 ms |
| MapIncrementBy | ~580 | 0.00058 ms |
| MapExists | ~35 | 0.00003 ms |
| MapLen | ~165 | 0.00016 ms |
| MapKeysScan | ~780 | 0.00078 ms |
| MapValuesScan | ~598 | 0.00059 ms |
| NMapSet | ~1057 | 0.00105 ms |
| NMapGetScan | ~448 | 0.00044 ms |
| NMapGetMScan | ~843 | 0.00084 ms |
| NMapGetAllScan | ~868 | 0.00086 ms |
| NMapIncrementBy | ~811 | 0.00081 ms |
| NMapExists | ~48 | 0.00004 ms |
| NMapLen | ~193 | 0.00019 ms |
| NMapKeysScan | ~1005 | 0.00100 ms |
| NMapValuesScan | ~700 | 0.00070 ms |
| ListLeftPush | ~2144 | 0.00214 ms |
| ListRightPush | ~1622492 | 1.62249 ms |
| ListLeftPopScan | ~1552419 | 1.55241 ms |
| ListRightPopScan | ~1514050 | 1.51405 ms |
| ListRangeScan | ~5039 | 0.00503 ms |
| ListLen | ~369829 | 0.36983 ms |
| HLLAdd | ~27 | 0.00003 ms |
| HLLCount | ~260 | 0.00026 ms |
| BloomAdd | ~32 | 0.00003 ms |
| BloomExists | ~7 | 0.000007 ms |
| CMSIncrement | ~111 | 0.00011 ms |
| CMSQuery | ~7 | 0.000007 ms |
| PubSub Propagation | ~8002 | 0.00800 ms |
import "github.com/cuprite-io/capacitor"
cfg := capacitor.Config{
NodeID: "node-1",
BindPort: 7946, // Gossip port
StreamPort: 7947, // Replication port
Peers: []string{"10.0.0.1:7946"},
DataPath: "/var/lib/capacitor",
AuthToken: "your-secure-token",
}
cp, _ := capacitor.New(cfg)
defer cp.Close()
// Standard Cache Operations
cp.Set(ctx, "session:123", "data", 10 * time.Minute)
val, _ := cp.Get(ctx, "session:123")
exists, _ := cp.Exists(ctx, "session:123")
_ = cp.Delete(ctx, "session:123")
// Structured Scanning (similar to json.Unmarshal)
var session UserSession
_ = cp.GetScan(ctx, "session:123", &session)
// Distributed Counters
count, _ := cp.Increment(ctx, "page_views")
// Sliding Windows
windowCount, _ := cp.IncrementSlidingWindow(ctx, "rate_limit:user_1", 1 * time.Minute)
// Replicated Conflict-Free Sets
_, _ = cp.SetAdd(ctx, "user:1:tags", "golang")
_, _ = cp.SetAdd(ctx, "user:1:tags", "crdt")
tags, _ := cp.SetMembers(ctx, "user:1:tags")
isMem, _ := cp.SetIsMember(ctx, "user:1:tags", "golang")
card, _ := cp.SetCard(ctx, "user:1:tags")
_, _ = cp.SetRemove(ctx, "user:1:tags", "golang")
// Storing and scanning non-string types in Sets (e.g. integers, custom structs)
type UserProfile struct {
Age int `msgpack:"age"`
Name string `msgpack:"name"`
}
_, _ = cp.SetAdd(ctx, "users", UserProfile{Age: 30, Name: "Alice"})
_, _ = cp.SetAdd(ctx, "users", UserProfile{Age: 25, Name: "Bob"})
var users []UserProfile
_ = cp.SetMembersScan(ctx, "users", &users)
var poppedUser UserProfile
_, _ = cp.SetPopScan(ctx, "users", &poppedUser)For production deployments, always configure a non-empty AuthToken. Setting an AuthToken automatically encrypts the gossip memberlist layer (via AES-256 using Memberlist's SecretKey derived via SHA-256), securing stream addresses and node metadata against plaintext eavesdropping. If AuthToken is left empty, metadata and replication ports are broadcast in plaintext.
Run the comprehensive test suite, including chaos and convergence tests:
go test -v .
go test -bench=. .