Skip to content

Add production runtime standard runner - #422

Open
jarcherNV wants to merge 7 commits into
dev/jarcher/inference-runtime-apifrom
dev/jarcher/api-runtime
Open

Add production runtime standard runner#422
jarcherNV wants to merge 7 commits into
dev/jarcher/inference-runtime-apifrom
dev/jarcher/api-runtime

Conversation

@jarcherNV

Copy link
Copy Markdown
Collaborator

Promote the test-only inference loop into flashdreams.runtime as run_inference_session.

The runner validates mapping compatibility before runtime creation, maps global conditioning, drives a synchronous sequential session, writes step outputs, records timing metrics, and reliably closes output/session/runtime/metrics.

Add focused production runner tests for success, validation ordering, cleanup, and declared mapping compatibility failures. Update runtime docs to mark T4 complete and point at the new runner entry point.

@greptile-apps

greptile-apps Bot commented Aug 6, 2026

Copy link
Copy Markdown
Contributor

Greptile Summary

The PR promotes the inference loop into a production runtime runner and introduces shared replay, MP4-output, WebRTC-demo, and OmniDreams adapter surfaces.

  • Validates mapping compatibility before creating runtime resources.
  • Maps global and per-step inputs while driving a synchronous sequential session.
  • Records timing metrics and closes output, session, runtime, and metrics resources.
  • Adds shared demo specifications, replay and WebRTC entry points, MP4 output support, and focused tests.

Confidence Score: 5/5

The PR appears safe to merge.

The previously reported input-loss issue is fixed: the runner now constructs its per-step base from initial_inputs.step and metadata, and the regression coverage verifies that a pass-through mapping delivers those values to every session step. No blocking failure remains.

Important Files Changed

Filename Overview
flashdreams/flashdreams/runtime/runner.py Adds the production synchronous runner, including pre-runtime validation, preserved pass-through step inputs, timing collection, and ordered cleanup.
flashdreams/tests/test_runtime_runner.py Covers successful multi-step execution, preservation of initial step inputs, validation ordering, mapping compatibility, metrics, and cleanup paths.
flashdreams/flashdreams/runtime/video_output.py Adds an MP4 output target that collects runtime video chunks, validates layouts, and emits an output artifact.
flashdreams/flashdreams/runtime/demo/replay.py Adds shared replay orchestration that prepares adapter scenarios and delegates execution to the production runner.
flashdreams/flashdreams/runtime/demo/webrtc.py Adds shared WebRTC runtime, session-manager, application, and server construction.
integrations/omnidreams/omnidreams/demo/adapter.py Adapts OmniDreams scenarios to the shared demo/runtime interfaces using global scenario conditioning.

Sequence Diagram

sequenceDiagram
  participant Demo as Demo/Caller
  participant Runner as run_inference_session
  participant Mapping as InputMapping
  participant Runtime as InferenceRuntime
  participant Session as InferenceSession
  participant Output as OutputTarget
  participant Metrics as MetricsRecorder

  Demo->>Runner: config, initial inputs, mapping, output
  Runner->>Mapping: validate compatibility
  Runner->>Mapping: map global conditioning
  Runner->>Runtime: create_runtime(config)
  Runtime-->>Runner: runtime
  Runner->>Runtime: start_session(mapped initial inputs)
  Runtime-->>Runner: session
  Runner->>Output: open()
  loop while next_step_request exists
    Runner->>Session: next_step_request()
    Runner->>Mapping: map_step_inputs(canonical inputs, preserved step base, request)
    Mapping-->>Runner: step inputs
    Runner->>Session: step(step inputs)
    Session-->>Runner: StepResult
    Runner->>Output: write(result)
    Runner->>Metrics: record timing samples
  end
  Runner->>Output: close()
  Runner->>Session: close()
  Runner->>Runtime: close()
  Runner->>Metrics: close()
Loading

Reviews (8): Last reviewed commit: "omnidreams: add shared demo API adapter" | Re-trigger Greptile

Comment thread flashdreams/flashdreams/runtime/runner.py Outdated
Promote the test-only inference loop into flashdreams.runtime as
run_inference_session.

The runner validates mapping compatibility before runtime creation, maps global
conditioning, drives a synchronous sequential session, writes step outputs,
records timing metrics, and reliably closes output/session/runtime/metrics.

Add focused production runner tests for success, validation ordering, cleanup,
and declared mapping compatibility failures. Update runtime docs to mark T4
complete and point at the new runner entry point.
Pass the caller-provided initial step payload into per-step input mapping
instead of always supplying an empty InferenceInput.

This keeps IdentityInputMapping and other pass-through mappings compatible with
sessions that require fixed per-step inputs. Add a regression test covering that
path while still avoiding global conditioning on steady-state step calls.
Apply the import sorting changes required by the CPU pre-commit job for the
runtime runner and its focused tests.
Apply the formatting changes produced by the CPU pre-commit job.

This updates runtime mapping code and input-mapping tests to match Ruff format,
resolving the ruff-format hook failure from PR CI.
Add the Phase 1 shared demo skeleton under flashdreams.runtime.demo with
DemoSpec, output specs, PreparedScenario, and DemoAdapter shapes. Route replay
demos through run_inference_session(), add shared output target construction,
and shape WebRTC demo construction around the existing BaseWebRTCSessionManager.

Cover the new boundary with fake-model tests, and update the demo API plan to
port OmniDreams before LingBot.
@jarcherNV
jarcherNV force-pushed the dev/jarcher/api-runtime branch from 3499fe0 to 5632e12 Compare August 6, 2026 05:45
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant