ShortsEngine is a production-beta, production-hardening project for turning source media or structured ideas into reviewable vertical videos. It combines media analysis, editorial planning, deterministic rendering, computer-vision-assisted framing and final-output verification in one local-first system.
The central engineering idea is simple: an AI pipeline should not report success because its metadata looks correct; the rendered video must support the claim.
Built by Anastasis Chatzedakis, an undergraduate student in the Department of Management Science and Technology at the Athens University of Economics and Business.
These specific outputs are used as evidence for ShortsEngine's documented 2026 workflows. Evidence depth differs by example, so the public showcase separates live-page verification from repository-backed render and provenance records.
Verified public motivational Short from the Budget Friendly channel. The preview above is the video's actual public thumbnail, stored in the repository so the output is visible directly inside this README.
Verified local narrated-animation technical final: 41.1 seconds, 1080×1920, 30 fps, H.264/AAC. Both previews were extracted from the SHA-256-verified final render; public upload is pending.
This is an inline animated render—not a link to an external player. It exercises the new promise-first overview, persistent visual thread, original line-art recipes, semantic typography, focus transitions and narration-bound microbeats. The profile remains opt-in while the ten-chain production calibration gate is completed.
The full MP4 is embedded above rather than represented by an external player link. This 45-second Wow Signal explainer is 1080×1920 at 30 fps with H.264/AAC, rights-cleared local Kokoro narration, exact transformed word anchors, engine-generated SFX, integrated semantic typography and a WebVTT transcript sidecar. Perceptual QA and all ten rendered-video technical gates passed. It remains a calibration candidate—not a publishable release—until the required human creative review is recorded.
The supplied football Short was unavailable during the latest verification, so no screenshot is shown until a verifiable output artifact is available.
See SHOWCASE.md for output metadata, provenance boundaries, quality evidence and publication status.
| Problem | Producing good short-form video repeatedly requires more than finding a loud moment and applying a center crop. |
| Approach | Separate evidence collection, editorial decisions, rendering and final visual proof behind explicit contracts. |
| Primary research track | Football highlights with scoreboard-aware event discovery, full-phase reconstruction and action-safe reframing. |
| Additional workflows | Motivational edits and original narrated animation. |
| Engineering focus | Validation, deterministic jobs, safe artifact handling, observability, evaluation and fail-closed release gates. |
| Current stage | Integration-proven production beta; real managed staging and external OIDC proof are still pending. |
The football pipeline is designed to find candidate match events, build a stable score timeline, reject disallowed or weakly supported events, reconstruct the live phase before a goal and verify the final render. Scoreboard OCR is treated as an anchor, while visual and audio evidence help locate the actual action.
The active research goal is not merely “detect a goal.” It is to show the buildup, finish, payoff and confirmation while keeping the ball and relevant players visible in a vertical frame.
The editorial pipeline ranks candidate moments, protects sentence boundaries, generates kinetic captions and records controlled quality experiments against a saved baseline.
The narrated pipeline turns approved scripts and claims into a frame-addressable
AnimationIR, synthesizes narration through an optional local TTS runtime and
renders continuous vector scenes without depending on broadcast or stock footage.
Its opt-in educational-explainer-v1 path adds ReferenceStyleSpec,
NarrativeBeatGraph, DirectorPlan, AnimationIR v4, AudioIR and a
rights-bound AssetManifest v2; see
the architecture contract.
- Evidence before claims. A score change, caption or JSON label is not enough to prove that a rendered video visibly contains a goal.
- Fail-closed output gates. Missing artifacts, unsafe crops, incomplete goal coverage and failed renders block export instead of producing misleading success.
- Domain-specific framing. Football reframing can follow action evidence and fall back to a wider view when tracking confidence is not strong enough.
- Local-first AI tooling. Tests and demos use deterministic providers by default; optional OCR, transcription, enhancement and TTS runtimes stay behind adapters.
- Quality is measured. Evaluation fixtures, visual QA, browser smoke tests and a baseline-driven research loop turn subjective editing changes into reviewable evidence.
flowchart LR
U["Authenticated user"] --> A["Rights-cleared media or content brief"]
U <--> O["OIDC session boundary"]
A --> B["Multipart ingest and validation"]
B <--> P[("PostgreSQL")]
B <--> S[("S3 / R2 artifacts")]
B --> C{"Pipeline router"}
C --> D["Football event analysis"]
C --> E["Editorial moment selection"]
C --> F["Script and AnimationIR"]
D --> G["Evidence-backed edit plan"]
E --> G
F --> H["Animation renderer"]
G --> I["FFmpeg renderer"]
H --> I
I --> J["Rendered video QA"]
J --> K{"Release gate"}
K -->|Pass| L["Export or private publish"]
K -->|Fail| M["Structured review evidence"]
W1["Worker A"] --> P
W2["Worker B"] --> P
W1 <--> S
W2 <--> S
The HTTP layer stays thin. Domain logic, providers, repositories, artifact storage, jobs and renderers are isolated behind testable boundaries. This keeps external tools replaceable and prevents API routes from becoming the workflow engine.
| Area | Implementation |
|---|---|
| Media safety | Extension, MIME, signature, size, duration and FFprobe validation before pipeline entry |
| Job lifecycle | Durable state transitions, cancellation, leases, retries, recovery and terminal-state protection |
| Production persistence | PostgreSQL repositories, scoped idempotency, immutable checksummed migrations and owner isolation |
| Distributed work | PostgreSQL SKIP LOCKED claims, fenced leases, heartbeat, retry, cancellation and dead-letter recovery |
| Storage | Local development plus S3/R2 multipart artifacts, validation, cleanup operations and session-bound delivery |
| Identity | OIDC Authorization Code + PKCE, hashed sessions, CSRF checks and non-enumerating owner boundaries |
| Football truth | Score-change timeline, evidence fusion, no-false-goal guards and chronological event binding |
| Rendering | FFmpeg/FFprobe adapters, bounded execution, edit-plan validation and export gating |
| Auto-framing | Ball/player/action tracking contracts with conservative wide-safe fallback |
| Enhancement | Managed Python Real-ESRGAN adapter with Apple MPS support and validated output frame counts |
| Transcription | Local Faster-Whisper adapter with word timestamps and deterministic fallback |
| Original animation | Frame-accurate AnimationIR v4, continuous vector rendering, narration alignment, persistent entities and semantic typography |
| Observability | Structured traces, bounded metrics, durable usage/cost events and sanitized readiness reports |
| Verification | Node tests, deterministic evals, Playwright browser checks, visual proofs and release reports |
ShortsEngine is explicit about what is stable and what is still being improved.
| Level | Capabilities |
|---|---|
| Implemented | PostgreSQL persistence and queue, OIDC sessions, multipart S3/R2 artifacts, owner isolation, football previews/approval, FFmpeg rendering, usage ledger and fail-closed production configuration |
| Integration-tested | Disposable PostgreSQL/S3 multi-worker claims, lease recovery, retries/cancellation, ownership isolation, shared storage, review transactions and durable cost schema |
| External proof pending | Real managed staging, a real external OIDC provider, private R2 delivery, deployed restart/recovery and live telemetry export |
| Active product research | Consistent full-goal recall on varied broadcasts, per-frame ball visibility, scorer tracking, reference-style pacing and larger rights-cleared evaluation sets |
- Node.js 18 or newer
- npm
- FFmpeg and FFprobe on
PATH - Playwright Chromium for browser proof checks
Optional capabilities include yt-dlp, OCR, Faster-Whisper, Real-ESRGAN and
Kokoro TTS. The default test path does not require cloud API keys.
git clone https://github.com/anaschatz/Shorts-Engine.git
cd Shorts-Engine
npm ci
npm run demo:fixture
npm run devOpen http://localhost:4175. The port can be changed with
PORT.
Production-beta deployments use the separate web, worker and migration entrypoints
from render.yaml. Validate the strict
postgres/postgres/r2/oidc/postgres profile before deploying:
npm run staging:production:check
npm run integration:productionSee the staging contract. These commands do not replace exact-commit proof against real managed services.
Run the core local release checks:
npm run lint
npm run build
npm test
npm run eval
npm run eval:reference
npm run demo:browser:ci
npm run release:checkThe repository currently contains 119 focused test files covering validation, media safety, persistence, jobs, provider contracts, football evidence, rendering, visual behavior, publishing guards and release workflows.
ShortsEngine uses a small-experiment loop for quality changes. Evaluation fixtures and rubrics remain fixed so an experiment cannot improve its score by changing the measurement.
npm run research:short:baseline
npm run research:short -- --description="one scoped quality experiment"Each run reports keep, discard or crash, together with the quality score,
delta, hard-gate failures and guardrail regressions.
- Live YouTube ingest is disabled until an operator explicitly enables it and confirms processing rights.
- Tests and the local demo do not require cloud API keys.
- External providers, FFmpeg, OCR, tracking and enhancement stay behind adapters.
- Public errors and reports exclude secrets, raw provider output, storage keys and absolute local paths.
- Temporary and partial artifacts are cleaned only inside managed staging areas.
- Exports remain unavailable until rendering and output validation complete.
- Ambiguous content can be routed to human review instead of being guessed.
server/ API, domain services, jobs, providers, storage and repositories
renderer/ Continuous animation and narrated renderers
tests/ Unit, integration, contract and visual-behavior tests
eval/ Deterministic fixtures, scoring and reference rubrics
demo/ Local proofs, browser checks and human-review tools
tools/ Research, environment, publishing and release utilities
docs/ Architecture, operations, staging and product decisions
shortresearch/ Saved baseline and experiment reports
Selected technical documents:
- Narrated visual shorts architecture
- Continuous animation architecture
- Growth pipeline architecture
- Production beta plan
- Environment reference
- Release process
- YouTube publishing guide
ShortsEngine is an integration-proven production beta and applied AI research project, not a finished commercial product and not yet externally proven for production traffic.
The engineering platform is broad and well tested, but live football broadcasts remain a difficult open problem: scorebugs vary, camera direction changes rapidly, the ball is small, and a correct data record does not guarantee a human-visible result. The project therefore keeps strict visual gates and records failures rather than claiming universal highlight accuracy.
Next milestones:
- Evaluate goal recall and visible phase coverage on a larger rights-cleared set.
- Improve broadcast-independent scorebug localization and temporal OCR stability.
- Strengthen ball/scorer tracking without aggressive or distracting crop motion.
- Prove the exact release commit on managed PostgreSQL, private R2, external OIDC, two deployed workers and durable telemetry.
- Measure edit-free pass rate, render reliability, latency and cost per video.
- Designing backend boundaries for unreliable AI and media tools.
- Building asynchronous workflows with recovery, idempotency and safe failure.
- Combining OCR, audio, vision and timeline evidence without overclaiming certainty.
- Testing subjective visual output with deterministic metrics and human review.
- Turning product risks into explicit contracts, observability and release gates.
- Balancing technical ambition with rights, provenance and operational constraints.
Anastasis Chatzidakis
Undergraduate student, Department of Management Science and Technology
Athens University of Economics and Business
- GitHub: @anaschatz
- Email: t8240165@aueb.gr
Built in Athens as a student project about AI systems, media operations and product engineering.