FOVEA

Forensic Open-generator Video Evidence Acquisition

Evidence-first analysis for AI-generated video

FOVEA combines multimodal-model reasoning with targeted computer-vision measurements, routes each video to relevant specialist agents, and fuses their outputs through an explicit decision protocol—without a task-specific training stage.

5 specialist analysts · 12 CV sub-skills · 3-stage evidence fusion

Overview · Architecture · Quick start · Output · Configuration · CLI reference

---

Overview

FOVEA is a CLI research framework for assessing whether a video is authentic or AI-generated. Instead of asking one model for a single opaque answer, it separates the process into evidence extraction, adaptive routing, domain-specific inspection, and explicit decision fusion.

Why FOVEA?

• Evidence-first workflow. FFmpeg/ffprobe metadata, sampled frames, hashes, and computer-vision measurements are prepared before model reasoning.
• Adaptive analysis. A planner can select only the analyst domains relevant to the current video.
• Hybrid evidence. Multimodal observations are paired with FFT, optical flow, geometry, ELA, boundary, edge, face, blur, phase, and feature-tracking signals.
• Parallel specialists. Selected style, physics, spatial, temporal, and watermark analysts run through LangGraph.
• Explicit fusion. A configurable judge supports strong-evidence override, confidence-filtered voting, and disagreement-aware weighted fusion.
• Reusable run records. Evidence caches and SQLite records make individual runs inspectable and experiments easier to resume or compare.

Note

“Training-free” means that FOVEA does not train or fine-tune a detector for each generator. It still relies on pretrained multimodal models at inference time.

Architecture

The diagram below shows the path enabled by the recommended configuration in this README. Human-Eyes screening, planner routing, CV skills, and the deterministic three-stage judge are individually configurable.

flowchart TD
    V[Input video] --> E[Evidence extraction and cache<br/>FFmpeg · ffprobe · SHA-256]
    E --> HE{Human-Eyes enabled?}
    HE -->|Yes| H[Three key frames<br/>spatial · watermark · physics · style]
    HE -->|No| P
    H --> X{High-confidence<br/>fake signal?}
    X -->|Yes| O[Verdict and run record]
    X -->|No| P{Planner routing enabled?}

    P -->|Yes| R[Select a relevant analyst subset]
    P -->|No| C[Use the configured analyst set]

    subgraph A[Selected specialists run in parallel]
        S[Style<br/>FFT]
        PH[Physics<br/>Optical flow · Geometry · NSG-Lite]
        SP[Spatial<br/>ELA · Patch · Boundary · Edge · Face · Blur]
        T[Temporal<br/>Local phase · Feature tracking]
        W[Watermark<br/>Visual text and logo reasoning]
    end

    R --> S
    R --> PH
    R --> SP
    R --> T
    R --> W
    C --> S
    C --> PH
    C --> SP
    C --> T
    C --> W

    S --> J{Judge}
    PH --> J
    SP --> J
    T --> J
    W --> J
    J --> O
    O --> DB[(SQLite persistence)]

Loading

If workflow_analysts is empty, the workflow falls back to a direct visual judge.

Analysis domains

Analyst	Primary question	Supporting signals
Style	Does the rendering exhibit synthetic-style characteristics?	FFT, texture and frequency observations
Physics	Are motion, support, geometry, and interactions plausible?	Optical flow, geometry stability, NSG-Lite
Spatial	Are local structures and boundaries internally consistent?	ELA, patch anomalies, boundaries, edges, face structure, blur uniformity
Temporal	Do identity, geometry, texture, and motion persist over time?	Local phase coherence, feature-point tracking
Watermark	Are generator, editor, or platform markers visible?	Multimodal visual/text reasoning

After analyst selection, each non-watermark analyst can perform a second, content-aware routing step to choose only the CV sub-skills relevant to the current video. The recommended configuration exposes up to four preview frames to this router.

Each analyst returns a score_fake, three confidence components, a rationale, and structured evidence items. FOVEA uses a conservative weakest-link aggregation:

confidence = min(clarity, model–CV agreement, input quality)

These values are decision signals, not calibrated probabilities.

Decision protocol

With judge_dynamic.enabled: true, the default cascade is:

Stage	Default rule	Outcome
1. Strong evidence	Any analyst has score_fake >= 0.85 and confidence >= 0.60	Return fake from that signal
2. Confident vote	Keep analysts with confidence >= 0.60; require at least two agreeing voters and a strict majority	Return majority label
3. Weighted fusion	Confidence-weighted score with a disagreement penalty (lambda = 0.15)	Threshold final score at 0.50

Weighted-fusion definition

s_w     = Σ(c_a · s_a) / Σ(c_a)
sigma_w = sqrt(Σ(c_a · (s_a - s_w)^2) / Σ(c_a))
s_final = clip(s_w - lambda · sigma_w, 0, 1)
c_final = mean(c_a) · (1 - sigma_w)

When Human-Eyes screening is enabled, it inspects the first, middle, and last valid frames. An early exit occurs when any of its four dimensions reaches both score >= 0.80 and confidence >= 0.80.

Quick start

1. Prerequisites

• Python 3.10+ (3.11 recommended)
• ffmpeg and ffprobe available on PATH
• A vision-capable model endpoint that supports image inputs and structured output

Install FFmpeg with your platform package manager:

# macOS
brew install ffmpeg

# Ubuntu / Debian
sudo apt-get update && sudo apt-get install -y ffmpeg

# Windows with Scoop
scoop install ffmpeg

Verify the system dependency:

ffmpeg -version
ffprobe -version

2. Clone and install

git clone https://github.com/IatomicreactorI/FOVEA.git
cd FOVEA

python -m venv .venv

Activate the environment and install dependencies:

# macOS / Linux
source .venv/bin/activate

# Windows PowerShell
.venv\Scripts\Activate.ps1

python -m pip install --upgrade pip
python -m pip install -r requirements.txt

3. Configure environment variables

Copy the environment template:

# macOS / Linux
cp env.example .env

# Windows PowerShell
Copy-Item env.example .env

At minimum, set a writable SQLite path and the API key for the selected provider:

DB_PATH=DB/database.db
OPENAI_API_KEY=your_api_key_here

DB_PATH is required by the current implementation. FOVEA creates the parent directory and initializes the database on first use.

4. Create the runtime configuration

Runtime configurations live under src/config/. This directory is not tracked in the current repository, so create it before running any CLI command:

# macOS / Linux
mkdir -p src/config

# Windows PowerShell
New-Item -ItemType Directory -Force src/config

Create src/config/config.yaml:

experiment_name: fovea_default

human_eyes_enabled: true
planner_mode: true
enable_skills: true
workflow_analysts:
  - style
  - physics
  - spatial
  - temporal
  - watermark

judge_dynamic:
  enabled: true
  tau_ovr: 0.85
  gamma_ovr: 0.60
  gamma: 0.60
  lambda_penalty: 0.15

llm:
  provider: openai
  model: YOUR_VISION_CAPABLE_MODEL
  temperature: 0
  max_retries: 3
  max_images: 40
  skill_router_max_preview_frames: 4

video:
  frame_ext: jpg
  frame_quality: 2
  frame_mime: image/jpeg

concurrency:
  lock_wait_sec: 8.0
  lock_poll_sec: 0.25

batch:
  delay_sec: 0.1

Replace YOUR_VISION_CAPABLE_MODEL with a model available through your chosen provider. The endpoint/model must support both image inputs and structured output (function calling).

5. Add video data

Videos must be stored below DATA_ROOT. If DATA_ROOT is unset, the default is the repository’s data/ directory.

data/
├── Real/
│   └── example_real.mp4
└── Fake/
    ├── Sora/
    │   └── example_sora.mp4
    └── OtherGenerator/
        └── example_fake.mp4

Supported media discovery is recursive for MP4, MOV, AVI, MKV, WebM, FLV, and GIF.

For external datasets, point DATA_ROOT and CACHE_ROOT to writable directories:

DATA_ROOT=/absolute/path/to/videos
CACHE_ROOT=/absolute/path/to/fovea-cache

6. Extract and cache evidence

Conversion is required before analysis:

# Small smoke run
python -X utf8 src/convert.py --label all --limit 5

# Full conversion
python -X utf8 src/convert.py --label all

Conversion is resumable by default and skips videos with a ready evidence cache.

7. Run FOVEA

Use a path relative to DATA_ROOT:

# One video, human-readable summary
python -X utf8 src/main.py \
  --label Real/example_real.mp4 \
  --config config.yaml

# One video, JSON summary
python -X utf8 src/main.py \
  --label Fake/Sora/example_sora.mp4 \
  --config config.yaml \
  --json

# One category
python -X utf8 src/main.py --label Fake --config config.yaml

# One generator directory
python -X utf8 src/main.py --label Fake/Sora --config config.yaml

# Entire data root
python -X utf8 src/main.py --label all --config config.yaml

Important

Pass only the configuration filename to --config; configuration paths are resolved inside src/config/. For example, use --config config.yaml, not --config src/config/config.yaml.

Output

For a single video, --json prints a compact run summary shaped like this:

{
  "run_id": "abc12345",
  "case_id": "UmVhbC9leGFtcGxlX3JlYWwubXA0",
  "case": {
    "case_id": "UmVhbC9leGFtcGxlX3JlYWwubXA0",
    "video_path": "/path/to/FOVEA/data/Real/example_real.mp4",
    "label": null
  },
  "results": {
    "style": {
      "agent": "style",
      "status": "ok",
      "score_fake": 0.0,
      "confidence": 0.0,
      "evidence_count": 0,
      "error": null
    }
  },
  "verdict": {
    "label": "real",
    "score_fake": 0.0,
    "confidence": 0.0,
    "rationale": "...",
    "evidence_count": 0
  }
}

The numeric values above illustrate the schema only. Detailed agent evidence, configuration, timing, and verdict records are persisted in SQLite. Export a complete run—including evidence arrays—with:

# Latest run to stdout
python -X utf8 src/export_result.py

# Specific run to a file
python -X utf8 src/export_result.py abc12345 --output result.json

Configuration

Behavior-changing keys

Key	Purpose
experiment_name	Separates repeated experiments and controls batch skip behavior
human_eyes_enabled	Enables three-frame early-exit screening
planner_mode	Lets the planner choose a subset from workflow_analysts
workflow_analysts	Defines the allowed analyst pool; empty means direct visual judge
enable_skills	Enables computer-vision sub-skills inside supported analysts
judge_dynamic.enabled	Uses the explicit three-stage judge instead of the LLM judge
llm.max_images	Caps frames sent in one multimodal request
batch.delay_sec	Delay between videos in batch mode

Provider adapters

The code contains adapters for the following endpoints. An adapter entry does not guarantee that every model on that endpoint is multimodal or supports structured output.

llm.provider	Environment variable	Adapter
openai	OPENAI_API_KEY	OpenAI
deepseek	DEEPSEEK_API_KEY	DeepSeek
alibaba	QWEN_API_KEY	DashScope OpenAI-compatible endpoint
kimi	KIMI_API_KEY	Moonshot OpenAI-compatible endpoint
aihubmix	AIHUBMIX_API_KEY	AIHubMix OpenAI-compatible endpoint
yizhan	YIZHAN_API_KEY	Yi-Zhan OpenAI-compatible endpoint

Environment overrides recognized by the configuration loader:

Variable	Overrides
DATA_ROOT	paths.data_root
CACHE_ROOT	paths.cache_root
LLM_PROVIDER	llm.provider
OPENAI_MODEL	llm.model

Evidence, persistence, and experiments

Evidence cache

src/convert.py creates an evidence bundle for each video under CACHE_ROOT/evidence/. The bundle includes sampled frames and probe metadata. Case IDs are derived from the video path relative to DATA_ROOT; metadata also records a SHA-256 digest.

Video duration	Extraction density
<= 60 s	4 frames/second
> 60 s	2 frames/second

Frames supplied to a multimodal call are uniformly downsampled when they exceed llm.max_images.

SQLite run records

The database stores:

• analysis runs and the full configuration used;
• selected analysts and per-agent scores;
• evidence items and agent errors;
• final verdicts and rationales;
• run and agent timing fields.

Experiment isolation

Always set an explicit, stable experiment_name. Batch analysis skips a video only when the database already contains a verdict under the same experiment name. Change the name for a new configuration or ablation.

experiment_name: fovea_no_cv_ablation
enable_skills: false

Summarize stored results with:

python -X utf8 src/statistics.py --list-experiments
python -X utf8 src/statistics.py --config config.yaml

Reported statistics include a confusion matrix, accuracy, per-class precision/recall/F1, confidence summaries, misclassified cases, and timing summaries when timing data exists.

CLI reference

Evidence conversion

python -X utf8 src/convert.py --label LABEL [options]

Option	Description
--label	Category, nested generator path, or all
--generator	Optional generator filter for fake videos
--limit	Maximum videos to convert
--delay	Delay between items
--quiet	Reduce progress output
--no-skip	Rebuild already converted videos
--no-resume	Disable resume behavior
--retry-failed-only	Process only previously failed conversions

Analysis

python -X utf8 src/main.py --label LABEL [--generator NAME] [--delay SECONDS] [--config FILE] [--json]

--json affects single-video output. Batch runs print progress and aggregate counts.

Result inspection

python -X utf8 src/statistics.py [--config FILE] [--list-experiments]
python -X utf8 src/export_result.py [RUN_ID] [--output FILE]

Repository layout

FOVEA/
├── src/
│   ├── main.py                    # Analysis CLI
│   ├── convert.py                 # Evidence extraction and caching
│   ├── statistics.py              # Experiment metrics
│   ├── export_result.py           # Full JSON export
│   ├── agents/
│   │   ├── human_eyes.py          # Early-exit screener
│   │   ├── planner.py             # Content-aware routing
│   │   ├── judge.py               # Decision fusion
│   │   ├── analysts/              # Five analysis domains
│   │   └── routing/               # CV sub-skill routers
│   ├── skill/                     # Computer-vision measurements
│   ├── graph/                     # LangGraph workflow and state
│   ├── pipeline/                  # Analysis and evidence entry points
│   ├── llm/                       # Provider adapters and structured inference
│   ├── database/                  # SQLite schema and persistence
│   ├── apis/video_io/             # FFmpeg/ffprobe integration
│   ├── util/                      # Paths, config, logging, sampling
│   └── config/                    # Local runtime YAML files (create locally)
├── env.example
├── requirements.txt
├── QUICKSTART.md
└── OUTPUT_FORMAT.md

Troubleshooting

Configuration file not found

Create src/config/config.yaml before running any command that imports FOVEA. The current repository does not ship runtime YAML files.

DB_PATH is missing

Copy env.example to .env and keep DB_PATH=DB/database.db, or set another writable path. The current database setup requires this value.

Evidence cache not found

Run python -X utf8 src/convert.py --label ... for the same DATA_ROOT and CACHE_ROOT before analysis.

The selected model rejects images or structured output

Choose a provider/model combination that accepts image data URLs and supports structured output through function calling.

Windows reports a UnicodeEncodeError

Use the documented python -X utf8 ... commands or enable UTF-8 mode with PYTHONUTF8=1.

A batch run skips completed videos

FOVEA found a stored verdict under the same experiment_name. Use a new experiment name for a new run, or use the experiment-deletion utility deliberately.

Project status

FOVEA is research code. Its heuristic scores depend on the configured multimodal model, decision thresholds, and input quality and should not be interpreted as calibrated probabilities. Video frames are sent to the configured model provider; review that provider’s data-handling terms before processing sensitive material.

The repository currently provides a CLI workflow. It does not include a web dashboard, packaged benchmark results, or a root license declaration.

Contributing

Issues and focused pull requests are welcome. For reproducible bug reports, include:

1. operating system and Python version;
2. the exact command;
3. a redacted configuration showing behavior-changing keys;
4. the relevant log excerpt and stack trace;
5. a minimal non-sensitive video sample or a precise reproduction description.

Never include API keys, private videos, or unredacted database files in an issue.

Citation

Canonical paper and citation metadata will be added when publicly available.

---

FOVEA turns a video verdict into an inspectable analysis process.