DistributedSQL

A distributed HTAP SQL engine with dual rowstore/columnstore, Raft consensus, and vectorized execution.

Architecture

┌─────────────────────────────────────────────────────────────────┐
│                      DistributedSQL Server                      │
├─────────────────────────────────────────────────────────────────┤
│  SQL Layer          │  Execution Layer      │  Storage Layer   │
│  ┌───────────────┐  │  ┌────────────────┐  │  ┌────────────┐  │
│  │ Parser        │  │  │ Vectorized     │  │  │ RowStore   │  │
│  │ Planner       │──┼──│ Operators      │──┼──│ (B+Tree)   │  │
│  │ Optimizer     │  │  │ - Filter       │  │  │ ColumnStore│  │
│  │ Executor      │  │  │ - Project      │  │  │ (Segment)  │  │
│  └───────────────┘  │  │ - HashJoin     │  │  │ BufferPool │  │
│                     │  │ - Aggregate    │  │  │ WAL/Checkpt│  │
│                     │  │ - Sort/Limit   │  │  └────────────┘  │
│                     │  │ - Exchange     │  │                  │
│                     │  └────────────────┘  │                  │
├─────────────────────────────────────────────────────────────────┤
│  Consensus Layer    │  Transaction Layer   │  Network Layer   │
│  ┌───────────────┐  │  ┌────────────────┐  │  ┌────────────┐  │
│  │ Raft          │  │  │ MVCC           │  │  │ RPC        │  │
│  │ - Leader      │  │  │ - HLC          │  │  │ io_uring   │  │
│  │ - Election    │  │  │ - Version Chain│  │  │ Connection │  │
│  │ - Log Repl.   │  │  │ - Snapshot     │  │  │ Pool       │  │
│  │ - Snapshots   │  │  │ 2PC            │  │  └────────────┘  │
│  │ - Joint Cons. │  │  │ Deadlock Det.  │  │                  │
│  └───────────────┘  │  └────────────────┘  │                  │
└─────────────────────────────────────────────────────────────────┘

Features

• SQL Engine: Full SQL parser, planner, optimizer with cost-based join reordering
• Dual Storage: RowStore (B+Tree) for OLTP, ColumnStore for OLAP
• Vectorized Execution: SIMD-optimized operators (AVX2/AVX-512)
• Raft Consensus: Leader election, log replication, joint consensus, snapshots
• MVCC: Hybrid Logical Clock, version chains, snapshot isolation
• Distributed Transactions: 2PC coordinator, deadlock detection
• High Performance: io_uring async I/O, lock-free data structures

Building

# Prerequisites: CMake 3.20+, GCC 11+, liburing, gtest, benchmark
./scripts/build.sh Release

# With AddressSanitizer
./scripts/build.sh Debug --asan

# With io_uring
./scripts/build.sh Release --io-uring

Running Tests

./scripts/run_tests.sh

# With filter
./scripts/run_tests.sh -f "RowStore*"

Running Benchmarks

./scripts/run_benchmarks.sh

# Output JSON
./scripts/run_benchmarks.sh --json results.json

Configuration

Environment variables:

• DATA_DIR - Data directory (default: /data)
• NODE_ID - Node identifier (default: 1)
• BIND_ADDRESS - Bind address (default: 0.0.0.0)
• SQL_PORT - SQL port (default: 5432)
• RAFT_PORT - Raft port (default: 9000)
• CLUSTER_NODES - Comma-separated node list

Docker

docker-compose up -d

# Or build and run manually
docker build -t distributedsql .
docker run -p 5432:5432 -p 9000:9000 distributedsql

Project Structure

distributedsql/
├── include/           # Public headers
│   ├── sql/          # Parser, planner, optimizer, executor
│   ├── storage/      # RowStore, ColumnStore, BufferPool, WAL
│   ├── raft/         # Consensus, log, state machine
│   ├── execution/    # Vectorized operators, joins, aggregates
│   ├── catalog/      # Schema, statistics
│   ├── transaction/  # MVCC, 2PC, deadlock
│   ├── network/      # RPC, io_uring
│   └── common/       # Types, memory, macros
├── src/              # Implementation
├── test/             # Unit tests
├── bench/            # Benchmarks
├── scripts/          # Build/test scripts
└── .github/          # CI/CD

License

Apache 2.0