diff --git a/.goreleaser.yml b/.goreleaser.yml index a551059..7d3e074 100644 --- a/.goreleaser.yml +++ b/.goreleaser.yml @@ -33,7 +33,7 @@ archives: files: - README.md - LICENSE* - - deploy/env.example + - deploy/systemd/env.example checksum: name_template: checksums.txt diff --git a/CHANGELOG.md b/CHANGELOG.md index 9af9349..ede1927 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -38,7 +38,7 @@ First open-source release. ### Verified - Zero lost jobs at up to 5,000 concurrent connections / 15,000 submissions - (see `docs/benchmark-results.md`). + (see `docs/benchmarks/benchmark-results.md`). ## [Unreleased] diff --git a/README.md b/README.md index 1baeeda..845db04 100644 --- a/README.md +++ b/README.md @@ -1,7 +1,7 @@ # AetherCode Execution Engine

- AetherCode Execution Engine + AetherCode Execution Engine

An enterprise-grade, high-throughput, on-premise distributed code execution and evaluation engine. AetherCode ingests untrusted code submissions over HTTP, safely runs them inside hardware-isolated Linux kernel sandboxes (cgroups v2, seccomp-bpf, network namespaces, and tmpfs overlays), and streams real-time execution lifecycle events and test verdicts back to clients via Server-Sent Events (SSE). @@ -115,7 +115,7 @@ go build -o aethercode-exec ./cmd/stj-exec # 2. Setup configuration file sudo mkdir -p /etc/aethercode-exec -sudo cp deploy/env.example /etc/aethercode-exec/env +sudo cp deploy/systemd/env.example /etc/aethercode-exec/env # 3. Configure mandatory environment variables (database credentials, judge auth token) sudo nano /etc/aethercode-exec/env @@ -130,9 +130,9 @@ For persistent host deployment under Linux systemd: ```bash # Copy systemd unit file and environment configuration -sudo cp deploy/aethercode-exec.service /etc/systemd/system/ +sudo cp deploy/systemd/aethercode-exec.service /etc/systemd/system/ sudo mkdir -p /etc/aethercode-exec -sudo cp deploy/env.example /etc/aethercode-exec/env +sudo cp deploy/systemd/env.example /etc/aethercode-exec/env # Reload systemd daemon and activate service sudo systemctl daemon-reload @@ -161,7 +161,7 @@ go build -o /tmp/loadgen ./cmd/loadgen ## System Architecture

- AetherCode Execution Engine System Architecture + AetherCode Execution Engine System Architecture

### Execution Flow @@ -207,7 +207,7 @@ Microbenchmarks demonstrate sustained 0 B/op and 0 allocs/op across `exact`, `to ### Environment Variables -Configuration parameters are loaded from environment variables on startup. Reference template: `deploy/env.example`. +Configuration parameters are loaded from environment variables on startup. Reference template: `deploy/systemd/env.example`. | Variable | Default Value | Description | |---|---|---| @@ -565,7 +565,7 @@ Conducted on a dual-socket, 256-thread Linux host with 512 GB RAM and NVMe stora | 800 concurrent clients | 2,400 jobs | 21.6 s | 111.1 jobs/sec | 0 (0.00%) | | **5,000 concurrent clients** | **15,000 jobs** | **124.0 s** | **121.0 jobs/sec** | **0 (0.00%)** | -Review `docs/benchmark-results.md` for in-depth percentiles (p50, p95, p99) and latency profiles. +Review `docs/benchmarks/benchmark-results.md` for in-depth percentiles (p50, p95, p99) and latency profiles. --- @@ -583,6 +583,22 @@ feature/* / fix/* ──> development (CI Gate) ──> production (Protecte --- +## Repository layout + +| Path | Contents | +|---|---| +| `cmd/stj-exec`, `cmd/loadgen` | The engine binary and the load generator | +| `internal/` | The gateway, queue, worker, sandbox client, database layer and supporting packages | +| `api/openapi.yaml` | The HTTP API contract | +| `migrations/` | Embedded PostgreSQL migrations | +| `deploy/systemd/` | The systemd unit and the environment template | +| `deploy/k8s/` | Kustomize manifests for Kubernetes | +| `deploy/monitoring/` | Prometheus alert rules | +| `docker/go-judge/` | The sandbox image (go-judge and the language toolchains) | +| `scripts/` | Smoke and end-to-end shell tests | +| `spike/` | The go-judge spike this design started from | +| `docs/` | Architecture, `benchmarks/` and `assets/` (README images) | + ## Contributing Contributions are welcome. Please consult: diff --git a/code-rabbit.yml b/code-rabbit.yml deleted file mode 100644 index 4f3b0ed..0000000 --- a/code-rabbit.yml +++ /dev/null @@ -1,46 +0,0 @@ -# CodeRabbit configuration -# Docs: https://docs.coderabbit.ai/configuration-options -# -# This file pre-configures CodeRabbit so that, once the CodeRabbit GitHub App -# is installed on the repo, reviews are focused on the engine's load-bearing -# invariants rather than generic style. -version: 1 - -reviews: - auto_review: - enabled: true - base_branches: - - development - - testing - - staging - - production - ignore_empty_commits: true - - instructions: | - You are reviewing AetherCode, a distributed code-execution engine that - runs untrusted user code inside a go-judge sandbox. Prioritize - correctness and the reliability invariants below. Be concise and - concrete; cite file:line and show the fix. - - ## Load-bearing invariants (flag any change that weakens them) - - Insert before enqueue: the PostgreSQL row must exist before the Redis - XADD. The orphan reaper and "zero lost jobs" guarantee rely on this. - - Subscribe before replay: the SSE handler must Subscribe before - GetEventLog, or a VERDICT published in between is silently dropped. - - writeCtx / XAck use context.WithoutCancel so in-flight jobs survive a - SIGTERM shutdown. - - CAS verdict writes: UPDATE ... WHERE status NOT IN ('completed','failed') - prevents double-writes on redelivery. - - Hidden test inputs never enter the sandbox, and tests never run inside - the sandbox. - - Redis Streams is the only queue (no Kafka). - - ## Focus areas - - Correctness, race conditions, and error handling on the hot path - (execute -> enqueue -> worker -> verdict -> SSE). - - The sandbox boundary: nothing untrusted should escape it. - - Resource cleanup (goroutines, connections, contexts, timers). - - ## Style - - Keep feedback actionable; skip nitpicks that don't affect correctness - or reliability. diff --git a/deploy/alerts.yml b/deploy/monitoring/alerts.yml similarity index 100% rename from deploy/alerts.yml rename to deploy/monitoring/alerts.yml diff --git a/deploy/aethercode-exec.service b/deploy/systemd/aethercode-exec.service similarity index 100% rename from deploy/aethercode-exec.service rename to deploy/systemd/aethercode-exec.service diff --git a/deploy/env.example b/deploy/systemd/env.example similarity index 100% rename from deploy/env.example rename to deploy/systemd/env.example diff --git a/docker-compose.yml b/docker-compose.yml index 9f94398..bd21f3e 100644 --- a/docker-compose.yml +++ b/docker-compose.yml @@ -7,7 +7,7 @@ # The go-judge service runs with `privileged: true` because its sandbox must # create cgroups + namespaces. That is acceptable for a disposable dev stack on # a machine you control; for production use the systemd deployment -# (deploy/aethercode-exec.service) with explicit resource limits instead. +# (deploy/systemd/aethercode-exec.service) with explicit resource limits instead. name: aethercode diff --git a/docs/ARCHITECTURE.md b/docs/ARCHITECTURE.md index 9cfe8c6..58d9d3f 100644 --- a/docs/ARCHITECTURE.md +++ b/docs/ARCHITECTURE.md @@ -136,7 +136,7 @@ sequenceDiagram → [postgres :5432] → [redis :6379] ``` -See `deploy/aethercode-exec.service` and `deploy/env.example`. +See `deploy/systemd/aethercode-exec.service` and `deploy/systemd/env.example`. ### Kubernetes (multi-replica) ``` diff --git a/AetherCode.png b/docs/assets/AetherCode.png similarity index 100% rename from AetherCode.png rename to docs/assets/AetherCode.png diff --git a/Architecture.png b/docs/assets/Architecture.png similarity index 100% rename from Architecture.png rename to docs/assets/Architecture.png diff --git a/docs/benchmark-results.md b/docs/benchmarks/benchmark-results.md similarity index 100% rename from docs/benchmark-results.md rename to docs/benchmarks/benchmark-results.md diff --git a/docs/spike-results.md b/docs/benchmarks/spike-results.md similarity index 100% rename from docs/spike-results.md rename to docs/benchmarks/spike-results.md