Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion .github/workflows/ci.yml
Original file line number Diff line number Diff line change
Expand Up @@ -169,7 +169,7 @@ jobs:
run: |
modelforge deployments --environment ci-demo > cli-deployments.json
modelforge predict ci-demo 10 > cli-prediction.json
- name: Run recruiter deployment proof
- name: Validate deployment lifecycle
run: >-
python demo/proof_scenario.py
--environment ci-proof
Expand Down
2 changes: 1 addition & 1 deletion .github/workflows/demo-proof.yml
Original file line number Diff line number Diff line change
Expand Up @@ -6,7 +6,7 @@ on:
environment:
description: "ModelForge serving environment name used for the proof."
required: false
default: "recruiter-demo"
default: "validation-demo"

permissions:
contents: read
Expand Down
2 changes: 1 addition & 1 deletion .gitignore
Original file line number Diff line number Diff line change
Expand Up @@ -49,5 +49,5 @@ override.tf.json
*_override.tf
*_override.tf.json

# Generated demo proof evidence
# Generated deployment validation evidence
demo-proof-evidence/
2 changes: 1 addition & 1 deletion Makefile
Original file line number Diff line number Diff line change
Expand Up @@ -26,7 +26,7 @@ demo:
python demo/bootstrap_demo.py --environment demo --weight 20

proof:
python demo/proof_scenario.py --environment recruiter-demo --output-dir demo-proof-evidence
python demo/proof_scenario.py --environment validation-demo --output-dir demo-proof-evidence

benchmark:
python benchmarks/bootstrap_external.py --environment benchmark-go
Expand Down
6 changes: 3 additions & 3 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -391,7 +391,7 @@ be captured on controlled hardware.

The latest validation also smoke-tests the installed CLI, public landing page,
multi-tenant SaaS console, auth configuration, Docker Compose stack, external Go
runtime, and the full recruiter deployment proof. In that proof, v2 received
runtime, and the full deployment validation scenario. In that scenario, v2 received
real weighted canary traffic and was promoted, v1 was restored through rollback,
and an intentionally regressed v3 produced 185.7% output drift against a 50%
threshold, was recorded FAILED, and left v1 as the authoritative final target.
Expand All @@ -410,8 +410,8 @@ This creates the foundation for measuring reliability and performance rather tha
- [External runtime contract](docs/runtime-protocol.md)
- [Public-cloud deployment contract](docs/cloud-deployment.md)
- [AWS ECS/Fargate reference deployment](infra/aws/README.md)
- [Recruiter deployment proof](docs/demo-proof.md)
- [Recruiter and interview brief](docs/recruiting.md)
- [Deployment validation scenario](docs/demo-proof.md)
- [Platform overview](docs/platform-overview.md)

Common workflows are available through the root `Makefile`:

Expand Down
6 changes: 3 additions & 3 deletions demo/proof_scenario.py
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
"""Run and capture a recruiter-visible ModelForge deployment proof."""
"""Run and capture a ModelForge deployment validation scenario."""

from __future__ import annotations

Expand Down Expand Up @@ -256,7 +256,7 @@ def run_proof(
json={
"name": model_name,
"description": (
"Recruiter-proof model for canary promotion, rollback, "
"Validation model for canary promotion, rollback, "
"and regression abort."
),
},
Expand Down Expand Up @@ -575,7 +575,7 @@ def write_evidence(
summary: dict[str, Any],
samples: list[dict[str, Any]],
) -> None:
"""Write machine-readable and recruiter-readable proof artifacts."""
"""Write machine-readable evidence and a human-readable report."""

output_dir.mkdir(parents=True, exist_ok=True)
(output_dir / "proof-summary.json").write_text(
Expand Down
9 changes: 4 additions & 5 deletions docs/demo-proof.md
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
# Recruiter Demo Proof
# Deployment Validation Scenario

This scenario is the shortest way to demonstrate ModelForge as a working ML
deployment platform rather than a collection of API endpoints.
Expand Down Expand Up @@ -31,7 +31,7 @@ docker compose up --build --wait --detach
Run the proof:

~~~bash
python demo/proof_scenario.py --environment recruiter-demo --output-dir demo-proof-evidence
python demo/proof_scenario.py --environment validation-demo --output-dir demo-proof-evidence
~~~

The command exits non-zero if any lifecycle assertion fails.
Expand All @@ -45,7 +45,7 @@ export MODELFORGE_URL="https://YOUR_MODELFORGE_HOST"
export MODELFORGE_API_KEY="mf_live_..."
export MODELFORGE_WORKSPACE="YOUR_WORKSPACE_SLUG"

python demo/proof_scenario.py --environment recruiter-demo --output-dir demo-proof-evidence
python demo/proof_scenario.py --environment validation-demo --output-dir demo-proof-evidence
~~~

The raw API key is used only as an HTTP Authorization header. It is never
Expand All @@ -57,8 +57,7 @@ The output directory contains:

- proof-summary.json — compact machine-readable lifecycle evidence;
- traffic-samples.jsonl — request-level model-version and traffic-lane proof;
- proof-report.md — recruiter-readable summary suitable for a GitHub Actions
artifact or README screenshot.
- proof-report.md — human-readable summary of the deployment lifecycle checks.

The proof deliberately includes both a successful and unsuccessful release:

Expand Down
39 changes: 39 additions & 0 deletions docs/platform-overview.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,39 @@
# Platform Overview

ModelForge manages the lifecycle of machine learning models from artifact
registration through deployment and inference. It combines a FastAPI control
plane, MySQL-backed model and deployment state, a browser console, an
authenticated CLI, and a standalone Go inference runtime.

## Model lifecycle

Model versions reference immutable artifacts with checksums. Deployments have
explicit lifecycle states, and each serving environment has an authoritative
target. Weighted canaries can receive live traffic before promotion. Operators
can abort a canary or roll back to a previous stable version.

## Runtime integration

The control plane supports in-process PyTorch and ONNX runtimes and an external
runtime protocol. The Go service implements that protocol in a separate
process. Other model frameworks can integrate through the same contract
without adding framework-specific inference logic to the control plane.

## Reliability and access

The serving path includes bounded retries, circuit breakers, health checks,
automatic stable fallback, and Prometheus metrics. Workspaces scope data and
API keys. Browser access uses OIDC Authorization Code with PKCE, opaque
sessions, CSRF protection, and a strict Content Security Policy.

## Validation and deployment

The repository includes a reproducible [deployment validation
scenario](demo-proof.md) covering stable serving, weighted canary traffic,
promotion, rollback, regression detection, failed-canary abort, and final
stable-target preservation. Benchmark results in the README were measured in
CI and should not be treated as production capacity estimates.

The [AWS reference deployment](../infra/aws/README.md) uses Terraform for
ECS/Fargate, RDS MySQL, S3, and supporting infrastructure. It is a deployment
reference; the public service has not yet been launched.
84 changes: 0 additions & 84 deletions docs/recruiting.md

This file was deleted.

2 changes: 1 addition & 1 deletion tests/test_demo_proof.py
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
"""Unit tests for the recruiter-visible deployment proof harness."""
"""Unit tests for the deployment validation scenario."""

from __future__ import annotations

Expand Down
Loading