feat: pluggable storage backends (SQLite + remote control plane) - #32
Merged
Conversation
Recording wrote only to a local JSONL file. Deployed agents need durable and shared storage, so introduce a Store protocol with three backends. The default stays local JSONL (zero config); nothing downstream changes. - Store protocol in chronicle/envelope/backends.py. EnvelopeStore (JSONL) already satisfies it; exposed as JsonlStore too. - SqliteStore: durable, queryable, zero dependency (stdlib sqlite3). Thread-safe appends for concurrent async requests. Good for a single deployed instance. - RemoteStore: ships envelopes to a Chronicle control plane over HTTP (stdlib urllib). Never raises into the agent (a failed append is warned and dropped); reads return [] on failure. - open_store(target) picks a backend from a string: a path -> JsonlStore, sqlite:/// or *.db/*.sqlite -> SqliteStore, http(s):// -> RemoteStore. - chronicle.record(store=...) now accepts any Store, or a path/URL string routed through open_store, so store="sqlite:///runs.db" and control-plane URLs work. - examples/control_plane/server.py: a minimal reference control plane (stdlib http.server + SQLite) that RemoteStore talks to and TokenOps can share. Exports: Store, JsonlStore, SqliteStore, RemoteStore, open_store, EnvelopeStore. Tests: tests/test_stores.py (SQLite round-trip, open_store dispatch, record into SQLite and via a sqlite:// URL, RemoteStore end-to-end against the reference server, and that a dead control plane never crashes the agent). ruff 0.15.22 clean. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> Signed-off-by: Tisha Chawla <tisha.chawla2020@vitalum.ac.in>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Recording wrote only to a local JSONL file. Deployed agents need durable and shared storage. This adds a
Storeprotocol with three backends, keeping local JSONL as the zero-config default. Nothing downstream changes.Backends
JsonlStore(the defaultEnvelopeStore)SqliteStoresqlite3RemoteStoreurllibAll satisfy the
Storeprotocol, sochronicle.record(store=...)andsession.storeaccept any of them.Dev-simple selection
open_store(target)routes by the string; or pass aSqliteStore/RemoteStoreinstance.Safety
RemoteStorenever raises into the agent: a failed append is warned and dropped, reads return[]on failure. Recording can't take down production.SqliteStoreserializes concurrent appends with a lock and allows cross-thread use (async requests share one store).Control plane
examples/control_plane/server.pyis a minimal reference service (stdlibhttp.server+ SQLite) thatRemoteStoretalks to. TokenOps can point at the same host to co-locate its cost ledger with trace storage, which is the "Chronicle provides control-plane support to TokenOps" direction.Tests
tests/test_stores.py: SQLite round-trip,open_storedispatch, recording into SQLite and via asqlite://URL, RemoteStore end-to-end against the reference server, and that a dead control plane never crashes the agent. Full suite green; ruff 0.15.22 clean. No em/en dashes.Follow-up (not in this PR): Phase C, first-class OpenTelemetry spans per boundary.