Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
6 changes: 5 additions & 1 deletion .env.example
Original file line number Diff line number Diff line change
Expand Up @@ -2,9 +2,13 @@ POSTGRES_PASSWORD=replace-with-a-long-random-password
APP_DATABASE_PASSWORD=replace-with-a-different-long-random-password
POSTGRES_PORT=54329
POSTGRES_BIND_ADDRESS=127.0.0.1
AGENTIC_DATA_IMAGE=ghcr.io/jason-doyle/agentic-data-kernel:0.3.0-alpha.2
AGENTIC_DATA_IMAGE=ghcr.io/jason-doyle/agentic-data-kernel:0.3.0-alpha.4
DATABASE_URL=postgresql://agentic_app:replace-with-a-different-long-random-password@127.0.0.1:54329/agentic_data
MIGRATION_DATABASE_URL=postgresql://postgres:replace-with-a-long-random-password@127.0.0.1:54329/agentic_data
DATABASE_SSL=disable

# Optional: base64-encoded PEM trust bundle for managed PostgreSQL.
# DATABASE_CA_CERT_BASE64=<base64-pem-bundle>

# At least 32 random characters. Used only to hash high-entropy API tokens.
AUTH_PEPPER=replace-with-at-least-32-random-characters
Expand Down
9 changes: 9 additions & 0 deletions .github/workflows/ci.yml
Original file line number Diff line number Diff line change
Expand Up @@ -47,6 +47,15 @@ jobs:
- name: Check production CLI
run: docker run --rm agentic-data-kernel:test node dist/production/cli.js --help

deployments:
name: Deployment templates
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v7
- name: Validate Helm, Bicep, and OpenTofu
shell: pwsh
run: ./scripts/validate-deployments.ps1

postgres:
name: PostgreSQL integration
runs-on: ubuntu-latest
Expand Down
5 changes: 5 additions & 0 deletions .gitignore
Original file line number Diff line number Diff line change
Expand Up @@ -8,3 +8,8 @@ dist/
*.db-wal
*.log
.DS_Store
**/.terraform/
*.tfstate
*.tfstate.*
*.tfvars
!*.tfvars.example
4 changes: 4 additions & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -2,6 +2,10 @@

## Unreleased

- Added validated Helm, Azure Bicep, AWS OpenTofu, and GCP OpenTofu deployment
templates with one shared security and runtime contract.
- Added an idempotent `bootstrap-role` production command for cloud migration
workflows.
- Rebuilt the README around the agent-first thesis, SRE proof, causal trace,
comparative evidence, measured costs, and explicit fit boundaries.

Expand Down
16 changes: 16 additions & 0 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -322,6 +322,22 @@ agent-facing data model, invariants, and interfaces. See
[Production Profile](docs/PRODUCTION.md) and
[Threat Model](docs/THREAT_MODEL.md).

### Cloud deployment templates

The package includes validated reference workloads for:

| Platform | Template |
| --- | --- |
| Kubernetes | Helm |
| Azure Container Apps | Bicep |
| AWS ECS Fargate | OpenTofu |
| Google Kubernetes Engine | OpenTofu plus Helm |

The templates require existing private PostgreSQL, secret stores, network
controls, TLS, and shared filesystems. They intentionally do not place
generated credentials in Bicep parameters or OpenTofu state. See
[Deployment Templates](deploy/README.md).

## Architecture

```text
Expand Down
10 changes: 5 additions & 5 deletions benchmarks/sre/results/report.md
Original file line number Diff line number Diff line change
Expand Up @@ -2,9 +2,9 @@

Generated from `summary.json`.

Source revision: `b3a12a6a066c4de68adad8f376ff235af58b61c7`
Source revision: `35d010cf263bbaf60a3c1d9f70af8daa9cd51b18`

Source hash: `db5052d97f02920144fbdc1a0871048bef4ad0ac6f742b8ff7013dc7af871545`
Source hash: `d4b468552af75056e88deb65104f535d0e60c4b5b0b8a1333e6e3a9458fc1f95`

## Correctness

Expand All @@ -24,7 +24,7 @@ Both variants must resolve every run with one delivery and one reconciliation.

The adapter delegates to the shipped SRE scenario, which contains
929 nonblank TypeScript source lines inside the
dependency. The full kernel dependency contains 13750
dependency. The full kernel dependency contains 14181
nonblank TypeScript source lines.

The benchmark runner and engine-specific audit verification contain
Expand All @@ -44,8 +44,8 @@ operated, or upgraded.

| Variant | Median milliseconds |
| --- | ---: |
| Conventional PostgreSQL | 58.12 |
| Agentic Data Kernel | 876.17 |
| Conventional PostgreSQL | 99.08 |
| Agentic Data Kernel | 1132.31 |

Runtime is not a headline metric. The variants perform different work and this
deterministic smoke benchmark is not a latency study.
Expand Down
22 changes: 11 additions & 11 deletions benchmarks/sre/results/summary.json
Original file line number Diff line number Diff line change
Expand Up @@ -4,8 +4,8 @@
"environment": {
"node": "v22.22.2",
"postgres": "18.6 (Debian 18.6-1.pgdg12+2)",
"commit": "b3a12a6a066c4de68adad8f376ff235af58b61c7",
"sourceHash": "db5052d97f02920144fbdc1a0871048bef4ad0ac6f742b8ff7013dc7af871545"
"commit": "35d010cf263bbaf60a3c1d9f70af8daa9cd51b18",
"sourceHash": "d4b468552af75056e88deb65104f535d0e60c4b5b0b8a1333e6e3a9458fc1f95"
},
"runs": [
{
Expand All @@ -31,7 +31,7 @@
"provider reconciliation": true,
"verification and terminal state": true
},
"durationMs": 63.28479999999996
"durationMs": 207.70100000000002
},
"operatedTables": 8,
"databaseBytes": 540672
Expand Down Expand Up @@ -59,7 +59,7 @@
"provider reconciliation": true,
"verification and terminal state": true
},
"durationMs": 891.0822000000001
"durationMs": 1001.6923999999999
},
"operatedTables": 18,
"databaseBytes": 1572864
Expand Down Expand Up @@ -87,7 +87,7 @@
"provider reconciliation": true,
"verification and terminal state": true
},
"durationMs": 58.11660000000006
"durationMs": 61.16439999999966
},
"operatedTables": 8,
"databaseBytes": 540672
Expand Down Expand Up @@ -115,7 +115,7 @@
"provider reconciliation": true,
"verification and terminal state": true
},
"durationMs": 876.1727000000001
"durationMs": 1132.3120000000004
},
"operatedTables": 18,
"databaseBytes": 1572864
Expand Down Expand Up @@ -143,7 +143,7 @@
"provider reconciliation": true,
"verification and terminal state": true
},
"durationMs": 53.971000000000004
"durationMs": 99.08329999999933
},
"operatedTables": 8,
"databaseBytes": 540672
Expand Down Expand Up @@ -171,7 +171,7 @@
"provider reconciliation": true,
"verification and terminal state": true
},
"durationMs": 830.5592000000001
"durationMs": 1252.8352999999997
},
"operatedTables": 18,
"databaseBytes": 1572864
Expand Down Expand Up @@ -266,7 +266,7 @@
"authoredTables": 0,
"operatedTables": 18,
"scenarioSourceLines": 929,
"dependencySourceLines": 13750
"dependencySourceLines": 14181
}
},
"benchmarkHarness": {
Expand All @@ -277,8 +277,8 @@
"agenticDataKernelMedian": 1572864
},
"runtimeMillisecondsInformational": {
"conventionalPostgresMedian": 58.11660000000006,
"agenticDataKernelMedian": 876.1727000000001
"conventionalPostgresMedian": 99.08329999999933,
"agenticDataKernelMedian": 1132.3120000000004
},
"explanationQuestions": 9,
"claims": {
Expand Down
127 changes: 127 additions & 0 deletions deploy/CONTRACT.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,127 @@
# Deployment Contract

Every deployment template must preserve the same runtime and security
invariants.

## Workloads

Run four independent workloads from the same immutable image:

| Workload | Command | Lifecycle |
| --- | --- | --- |
| Runtime role bootstrap | `node dist/production/cli.js bootstrap-role` | One shot before migrations |
| Database migration | `node dist/production/cli.js migrate` | One shot before API and worker rollout |
| API | `node dist/production/cli.js serve` | Long running, port 4318 |
| Effect worker | `node dist/production/cli.js worker` | Long running, no ingress |

The bootstrap and migration workloads are idempotent. Migrations use a
PostgreSQL advisory lock and checksum every applied migration.

## PostgreSQL

The database must provide:

- PostgreSQL 18;
- pgvector 0.8 or newer;
- `pgcrypto`;
- an administrative migration identity with `CREATEROLE` and permission to
install the required extensions;
- a fixed `agentic_app` login created by `bootstrap-role`;
- encrypted connections and private network access;
- backups and point-in-time recovery appropriate to the environment.

The runtime role is always configured as:

```text
LOGIN NOSUPERUSER NOCREATEDB NOCREATEROLE NOINHERIT NOBYPASSRLS
```

When `agentic_app` already exists, a non-superuser bootstrap identity must also
hold `ADMIN OPTION` on that role. A managed-service administrator or
superuser-equivalent identity is the simplest bootstrap and migration
credential.

## Secrets

Cloud secret stores or externally managed Kubernetes Secrets must provide:

| Name | Consumer |
| --- | --- |
| `DATABASE_URL` | API and worker, using `agentic_app` |
| `MIGRATION_DATABASE_URL` | Bootstrap and migration, using the administrative identity |
| `APP_DATABASE_PASSWORD` | Bootstrap job |
| `DATABASE_CA_CERT_BASE64` | Optional PEM trust bundle for managed PostgreSQL |
| `AUTH_PEPPER` | API and worker |
| `ARTIFACT_KEYRING` | API and worker |
| `EMBEDDING_API_KEY` | API and worker |

API and worker identities must not be able to read
`MIGRATION_DATABASE_URL` or `APP_DATABASE_PASSWORD`. Use separate runtime and
administrative secrets and cloud identities.

If a non-superuser bootstrap identity manages `agentic_app`, its PostgreSQL 18
membership must use `ADMIN TRUE, INHERIT FALSE, SET FALSE`. Privilege-bearing
memberships are rejected.

`APP_DATABASE_PASSWORD` must contain 16 to 256 printable ASCII characters
without spaces. The bootstrap command derives a SCRAM-SHA-256 verifier
client-side so the plaintext password is not sent as a SQL bind value.

`ARTIFACT_CURRENT_KEY_ID` and `ARTIFACT_KEYRING` must identify a 32-byte
base64-encoded encryption key. Secret values must not be committed to values
files, Bicep parameter files, OpenTofu variables, logs, or outputs.
Every template exposes the current key ID separately so rotations can add a
new key to the keyring before switching new writes.

Set `DATABASE_SSL=require`. When the managed PostgreSQL certificate chain is
not present in the container's system trust store, provide its PEM CA bundle
as base64 through `DATABASE_CA_CERT_BASE64`.
Do not add `sslmode`, `sslrootcert`, or other SSL query parameters to database
URLs; the runtime rejects URL-level SSL settings so they cannot weaken the
configured verification policy.

An authenticated database proxy running in the same pod is the exception. In
that profile, the application may use `DATABASE_SSL=disable` only for a
loopback connection while the proxy performs encrypted, authenticated
upstream transport.

## Artifact filesystem

The encrypted artifact store uses:

- exclusive temporary-file creation;
- file `fsync`;
- atomic hard-link creation;
- concurrent reads and writes;
- recursive listing and deletion.

All API and worker replicas must mount the same filesystem at
`ARTIFACT_DIR`, normally `/var/lib/agentic-data/artifacts`. The filesystem must
support hard links within one mount and be writable by UID and GID `10001`.

Compatible examples include Azure Files NFS, Amazon EFS, Google Cloud
Filestore, and suitable Kubernetes ReadWriteMany volumes. Object-storage FUSE
drivers are not supported unless they document equivalent hard-link and
durability semantics.

## Networking

- Expose only the API.
- Terminate TLS at the managed ingress or load balancer.
- Keep PostgreSQL and artifact storage private.
- Set `HOST=0.0.0.0` and `PORT=4318` for the API container.
- Apply platform-specific egress controls for PostgreSQL, DNS, the embedding
endpoint, and effect destinations. Standard Kubernetes NetworkPolicy cannot
filter HTTPS by hostname.
- Configure `EFFECT_ALLOWED_HOSTS` explicitly.

## Health and rollout

- Liveness: `GET /health/live`
- Readiness: `GET /health/ready`
- Run bootstrap and migrations before increasing API or worker replicas.
- Use immutable image tags or digests.
- Restart workloads after rotating environment-injected secrets.

The API rate limiter is process-local. Horizontal replicas multiply the
effective aggregate request allowance.
38 changes: 38 additions & 0 deletions deploy/README.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,38 @@
# Deployment Templates

These templates deploy the same production workload contract across multiple
platforms:

| Platform | Template | Scope |
| --- | --- | --- |
| Kubernetes | [Helm chart](kubernetes/helm/agentic-data-kernel) | Portable API, worker, bootstrap, migration, service, ingress, and artifact PVC |
| Azure | [Bicep](azure) | Azure Container Apps workloads and manual jobs |
| AWS | [OpenTofu](aws) | ECS Fargate services and one-shot task definitions |
| Google Cloud | [OpenTofu](gcp) | Helm deployment into an existing GKE cluster |

Read [CONTRACT.md](CONTRACT.md) before using any template.

Every template requires an explicit immutable application image version or
digest. No runnable default points at an older release.

The templates intentionally consume existing cloud networks, PostgreSQL
servers, secret stores, and persistent storage. Landing zones and credentials
vary substantially between organizations, and placing generated database or
provider credentials in Bicep parameters or OpenTofu state would create an
unsafe default.

## Support level

The templates are reference deployments for the repository's bounded
single-primary production profile. Static validation runs in CI. Cloud applies
require an account, billable resources, provider-specific policy decisions,
and operator verification.

They do not claim:

- multi-region database failover;
- zero-downtime schema changes;
- automatic secret rotation;
- cloud object-storage support;
- compatibility with filesystems that lack POSIX hard links;
- a complete organizational landing zone.
25 changes: 25 additions & 0 deletions deploy/aws/.terraform.lock.hcl

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

Loading