Releases · maxpert/marmot

26 Apr 12:51

github-actions

v2.9.13-beta

da44d58

v2.9.13-beta Pre-release

Pre-release

What's Changed

🔧 Refactoring

make bootstrap and maintenance incremental
improve IVF maintenance and query execution

📦 Other Changes

Improve live vector encoding and CDC replication
Document vector search usage and tuning
Implement hierarchical IVF catch-up maintenance
Improve vector read benchmark coverage
Improve live IVF PQ indexing and read performance
Improve live IVF PQ scan and maintenance performance

Full Changelog: v2.9.12-beta...v2.9.13-beta

Assets 12

21 Apr 13:57

github-actions

v2.9.12-beta

0c70764

v2.9.12-beta Pre-release

Pre-release

What's Changed

🐛 Bug Fixes

correct arm64 dotprod target attribute

🧹 Maintenance

bump version to v2.9.12-beta

Full Changelog: v2.9.11-beta...v2.9.12-beta

Assets 12

21 Apr 13:50

github-actions

v2.9.11-beta

9a38d54

v2.9.11-beta Pre-release

Pre-release

What's Changed

🚀 New Features

add packed streaming ANN path and refresh docs

🔧 Refactoring

move vector serving to quantized segment store
remove legacy Go partition cache\n\nRemove the old Go-side partition cache and its config/session/benchmark knobs so the vector runtime has a single read architecture: packed stable partitions, resident delta, and SQLite fallback.\n\nThis commit:\n- deletes the legacy partition/vector cache implementations and cache-only tests\n- removes cache_bytes, @@marmot_vec_use_cache, and vec-bench cache flags\n- simplifies go-rank planning/execution to packed path + shared-scan/SQLite fallback\n- keeps resident delta maintenance and packed dirty-cluster invalidation after flush\n- updates docs to describe the packed-path runtime and current exposed knobs\n- drops the obsolete otter dependency from both Go modules\n\nVerification:\n- go build -tags "sqlite_preupdate_hook sqlite_fts5 sqlite_json sqlite_math_functions sqlite_foreign_keys sqlite_stat4 sqlite_vacuum_incr" -o vec-bench ./cmd/vec-bench\n- go test -tags "sqlite_preupdate_hook sqlite_fts5 sqlite_json sqlite_math_functions sqlite_foreign_keys sqlite_stat4 sqlite_vacuum_incr" ./coordinator -run 'TestPackedRankUsesResidentDeltaWithoutSQLite|TestTryDirectPKResult|TestMakeVecSharedScanKey|TestSharedScanRankFallsBackOnProbeRefreshMismatch|TestRewrite_GoRank' -count=1\n- go test -tags "sqlite_preupdate_hook sqlite_fts5 sqlite_json sqlite_math_functions sqlite_foreign_keys sqlite_stat4 sqlite_vacuum_incr" ./db/... -run 'TestRebuildPackedPartitionStore|Test(CreateVectorIndex|BulkPopulate_|Loader_|DeltaFlush|Reindex)' -count=1\n- (cd modules/vecindex && go test -tags "sqlite_preupdate_hook sqlite_fts5 sqlite_json sqlite_math_functions sqlite_foreign_keys sqlite_stat4 sqlite_vacuum_incr" . -run 'Test(IndexState_(StoreResidentDelta|ApplyDeltaFlushUpdates)|Engine_Unregister|DeltaBuffer_)' -count=1)
remove dead code, consolidate duplicates

🧪 Testing

skip timing-bound tests under -race

Full Changelog: v2.9.10-beta...v2.9.11-beta

Assets 10

16 Apr 14:09

github-actions

v2.9.10-beta

e0f3236

v2.9.10-beta Pre-release

Pre-release

What's Changed

🚀 New Features

SQLite-native vector search (v5.4) — replaces Pebble IVF
implement MetricDot via MIPS→L2 augmentation (Bachrach et al. 2014)
scalar int8 quantization (SQ8) for posting lists
lock-free concurrency rewrite with online MacQueen k-means
wire adapter, rewire main + db + e2e tests
implement engine + index + lifecycle (TDD phase 2)
implement k-means++ and centroid set (TDD phase 2)
implement store layer (TDD phase 2)
nuke HD-Index, scaffold IVF package skeleton
wire PebbleCacheMB, add snapshot/restore, E2E tests
add crash recovery, rebuild, and source table checks (Phase 5)
complete marmot integration wiring + gap fixes
add vec_knn() query interception for vector search (Phase 4)
wire vector DDL through coordinator handler (Phase 3)
add VectorIndexManager with CDC subscription (Phase 2)
add vector index DDL parsing (Phase 1 of marmot integration)
replace freshann with HD-Index vector indexing module
graceful shutdown and node decommission
add standalone Pebble-backed ANN module with benchmark harness and perf-focused query path
Signal-based CDC notification for replica streaming
Add write forwarding from read-only replicas to leader
Add -daemon flag for background process mode
Add file-based logging with rotation support
Wire up new admin endpoints in dashboard UI
Add new admin API endpoints for improved observability
Add auto-release workflow for tag-based releases
Comprehensive telemetry improvements and gap fixes
Add time-limited incremental vacuum after WAL checkpoint
Add zstd compression for gRPC cluster communication
Session-level transpilation toggle and SQLite extension loading
Add determinism detection for DML statement replication
Make CDC streaming threshold configurable
Defer CDC payload from PREPARE to COMMIT phase
add ReconnectWaitSeconds; tidy connect loop
configurable NATS connection retries

🐛 Bug Fixes

pebble cache Unref moved to store.New after Open (MR-04)
specMu guards idx.spec mutations against publishWorker race
Close waits for compactWorker; test aligned to nlist=64 tier
wire Score + PebbleCacheMB + msgpack spec + bloom (Phase 4 Group B)
atomic write batches + sync docID + bg compact (Phase 4 Group A)
safe seed fold + tar path traversal + remove unused MIPS transforms (Phase 4 Group D)
atomic split/merge + epoch-derived split seed + lock contracts (Phase 4 Group C)
Parameterized query extraction for binary data safety
MySQL/Drupal compatibility improvements
Strip MySQL column COLLATE and COMMENT in CREATE TABLE transpilation
Forward handler missing InTransaction and CommittedTxnId
Schema version not incremented on followers for CREATE/DROP DATABASE
Return LastInsertId in write forwarding response
CREATE/DROP DATABASE forwarding and cleanup dead code
Skip transpilation for forwarded queries from replica
Write forwarding improvements and stream client robustness
DDL streaming replication - persist DDL to CDC storage
CREATE/DROP DATABASE via write forwarding now available immediately
CREATE DATABASE via write forwarding fails with syntax error
Add missing logging options to config.toml
Release workflow version naming and Linux uploads
Replica snapshot verification and codebase cleanup
CDC Type override breaking UPSERT replication
Deadlock in wireGCCoordination during startup
GC interval must be >= AE interval to prevent stale watermark decisions
Wire up database discovery loop and system DB filtering
TxnID-ordered streaming for replica failover correctness
Critical safety fixes and dead code removal
Remove dead batch writer and fix CDC raw row cleanup
Schema version race condition causing Node 3 instability
Add reliable table replication wait in crash recovery tests
Critical cluster reliability and SWIM protocol issues
Automatic gRPC connection establishment for ALIVE nodes
Create __marmot__replication_state table during schema initialization
check conn status on client side
use structured logging & init connect_retries

⚡ Performance

W4-2 set Pebble SST block size to 64 KB for posting lists
W4-1 unsafe zero-copy vecBytes→float32 for SIMD distance
kill per-query centVec copies + pre-size seen map
parallel per-cluster scan bounded by GOMAXPROCS
zero-copy callback scan eliminates 91% of search allocations
tighten adaptive multi-probe to prevent cascade to nlist/2
HD-Index performance overhaul + real-data benchmark harness
add viterin/vek SIMD-optimized float32 distance functions
Optimize CleanupAfterCommit with in-memory intent tracking
Simplify commit path - remove validation, add DDL flush barrier
Flip intent storage schema to eliminate O(N) random seeks
Wrap applyDataChanges CDC entries in single SQLite transaction
Reduce network payload in 2PC COMMIT phase
Native memory allocation and hookDB connection optimization
Eliminate double serialization and merge cleanup operations
Eliminate read-modify-write anti-patterns in PebbleMetaStore

🔧 Refactoring

simplify pass — bug fix, validation, cleanup
simplify pass — bug fix, validation, cleanup
Consolidate Pebble key-building functions with generic helpers
Replace Pebble intent locks with in-memory RowLockStore
Unify streaming chunk size configuration
Update cluster-with-replicas script for dynamic config generation
Improve example scripts with auto-generated configs and PSK
Remove CDC pipeline buffering for streaming
Binary intent key encoding for performance
Replace external go-mimalloc with internal pkg/mimalloc
Unified TableSchema eliminates duplicate PRAGMA parsers
Unified CDC applier consolidating 4 duplicate implementations
Consolidate CDCEntry type and cleanup duplicates
Revert PebbleDB tuning and simplify preupdate hook
Use centralized encoding package for msgpack
Extract ReplicationEngine to eliminate code duplication
Extract INFORMATION_SCHEMA filters using AST instead of regex
address PR feedback
reduce configuration bloat

📖 Documentation

Add per-database snapshot configuration and feature documentation
Update README with new replica failover configuration
update NATS options in config.toml

🧹 Maintenance

correct beta version to v2.9.0-beta
Remove deprecated code and unused parameters
Remove dead code and unused config fields
Update outdated dependencies
upload pages artifact fix
action/configure-pages
update pages action
updates ci
remove comments
add docs site
Add nextjs github pages workflow

🧪 Testing

regression tests for CR-02 and HR-01
fix TestKMeansPlusPlus_K1 expected mean arithmetic
failing tests for index/engine/lifecycle/snapshot (TDD phase 1)
failing tests for store layer (TDD phase 1)
failing tests for kmeans + centroids (TDD phase 1)
Remove skeleton test TestBackgroundRetry_CancelOnShutdown

📦 Other Changes

bench(vecindex): add Update benchmark + p95/p99 on insert path
Revert "feat(vecindex): scalar int8 quantization (SQ8) for posting lists"
bench(vecindex): add insert + mixed + graduation benchmarks
bench(vecindex): persistent-dir warm/cold bench with p50/p95/p99 metrics
bench(vecindex): restore real-data bench harness + DBpedia-1536 100K
Fix bug in handler for admin routes for remove and allow nodes in cluster quorum.
freshann: align shared dependency versions with root module
freshann: cut to V2 format and harden disk-first ANN pipeline
freshann: checkpoint disk-first ANN optimizations + library adaptive budgeting
replication: complete P1 hardening for replay/stream/load-data and test stability
build: move to Go 1.26 + Green Tea GC and bump project to v2.1.0-beta
Add replicated LOAD DATA LOCAL INFILE with replica forwarding
transpiler/cdc: wire schema-aware upsert conflicts and remove forced WAL checkpoints
replica: make forwarded writes leader-transparent and idempotent
ci: release assets to GitHub Releases on workflow_dispatch events too
Pika: Add error categorization, verify command, and documentation
Correct rqlite comparison
Feature: Replica cluster discovery, partial failure handling, and performance optimizations
Phase 1: Add per-database snapshot support
Feature: Add correctness verification test suite and utf8mb4 support
Feature: Add cluster discovery and transparent failover for replicas
Updating README.md with new features
Feature: Add Unix socket support for MySQL server
Duh
Improvements improvements
Revert: Use mimalloc for native memory allocation
mimalloc packaging
Refactor WriteCoordinator with comprehensive unit tests
Optimize single-node example config for benchmarks
Optimize default config for production performance
Implement adaptive checkpoint strategy for SQLite batch committer
Add SQLite batch committer for improved write throughput
Add SQLite batch committer for improved write throughput
Fix code review issues: error handling and hot path optimization
Consolidate StatementCode in common package with strict wire boundary
Fix prepared statement string params for SQLite TEXT comparison
Rename StatementType to StatementCode for type safety
Fix SQL injection vulnerability by removing string interpolation
Add cluster integration test and schema cache error handling
Optimize hot path logging and use cached schema for CDC operations
Fix P1 issues in anti-entropy streams and PK-change UPDATE replay
Fix P1 issues in schema versioning and catch-up system
Fix P0 bugs in 2PC and CDC merge pipeline
Rename mvccDB variables and ExecuteMVCCRead method
Refactor schema cache system with unified API
Mod tidy
Add CDC Publisher system with Kafka/NATS support
Add reverse index for CDC locks to optimize cleanup from O(n) to O(m)
Add CDC pipeline for multi-row replication fix
Fixing mdx...

Assets 12

14 Feb 12:43

github-actions

v2.9.9-beta

271bcb6

v2.9.9-beta Pre-release

Pre-release

What's Changed

🐛 Bug Fixes

Parameterized query extraction for binary data safety
MySQL/Drupal compatibility improvements
Strip MySQL column COLLATE and COMMENT in CREATE TABLE transpilation

🧹 Maintenance

correct beta version to v2.9.0-beta

📦 Other Changes

build: move to Go 1.26 + Green Tea GC and bump project to v2.1.0-beta
Add replicated LOAD DATA LOCAL INFILE with replica forwarding
transpiler/cdc: wire schema-aware upsert conflicts and remove forced WAL checkpoints
replica: make forwarded writes leader-transparent and idempotent

Full Changelog: v2.9.8-beta...v2.9.9-beta

Assets 12