[Prudctual]

Overview:

Preserve response_format=b64_json on vLLM-Omni video generation (T2V and I2V) so the formatter returns inline base64 instead of a pod-local file:// URL.

This is part A of ai-dynamo#14667 only. The independent FFmpeg color-corruption issue (part B) is out of scope.

Upstream PR could not be opened (403). Open against ai-dynamo/dynamo from:

https://github.com/ai-dynamo/dynamo/compare/main...Prudctual:dynamo:cursor/omni-video-response-format-fbb4?expand=1

Details:

_engine_inputs_from_video() constructed EngineInputs without response_format=req.response_format. The output formatter then defaulted to url, so an explicit b64_json request completed with only a worker-local file URL.

Sibling builders were checked for the same omission (ai-dynamo#14668 shared root cause):

• Image (_engine_inputs_from_image): already forwards req.response_format.
• Audio (AudioGenerationHandler.build_engine_inputs): already maps data_source to EngineInputs.response_format (OpenAI audio uses response_format for the codec).
• I2V: shares _engine_inputs_from_video, so the one-line assignment covers it.

A unit test now asserts T2V and I2V forward b64_json/url, plus an image regression check.

Where should the reviewer start?

• components/src/dynamo/vllm/omni/omni_handler.py (_engine_inputs_from_video)
• components/src/dynamo/vllm/tests/omni/test_omni_handler.py (test_video_and_i2v_forward_response_format)

Related Issues

🔗 This PR is linked to an issue:

• Closes [BUG]: vllm-omni-video-agg on 1.4.2 drops base64 delivery and corrupts video colors ai-dynamo/dynamo#14667

Summary

Video omni EngineInputs now carry req.response_format, matching the image path, so b64_json is not dropped.

Validation

• Sibling builders inspected: image and audio already forward delivery format; I2V uses the patched video builder.
• Added unit tests in test_omni_handler.py for image, T2V, and I2V response_format forwarding.
• Source-level check: both image and video EngineInputs constructors pass response_format=req.response_format.
• Full vllm-omni unit suite is intended to run in CI (pytest markers: unit, vllm, gpu_0, multimodal, pre_merge). This cloud-agent environment does not have vllm-omni installed.

  

---

Summary by cubic

Fixes vLLM-Omni video generation (T2V and I2V) so response_format=b64_json returns inline base64 instead of a pod-local file URL. _engine_inputs_from_video now forwards req.response_format to EngineInputs, matching the image and audio builders. Adds unit tests verifying b64_json and url forwarding for T2V and I2V. Closes ai-dynamo#14667.

^{Written for commit 4ffdc68. Summary will update on new commits.}