Detailed description of the requested feature
The llm_inference command currently supports VLM image inputs only when the image is provided via a local file path. This prevents input files from referencing images as base64-encoded data URLs.
I would like to add support for base64-encoded images, like data:image/jpeg;base64,... or data:image/png;base64,....
This would make llm_inference easier to use across same filesystem. It would also simplify integration with evaluation frameworks such as LMMs-Eval, where images may already be available as base64-encoded data.
My work-around implementation is available at: https://github.com/FABallemand/TensorRT-Edge-LLM/tree/feat/llm_inference_base64_support
Timeline
No hard deadline.
Describe alternatives you've considered
The current workaround is to save the image to a file and update the image path in the input file. This adds unnecessary filesystem I/O.
Adding native base64 image support to llm_inference would provide a more interoperable solution while preserving the existing file-path behavior.
Target hardware/use case
Jetson Thor (SM_110, JetPack 7.1, TensorRT 10.13.3.9).
Detailed description of the requested feature
The
llm_inferencecommand currently supports VLM image inputs only when the image is provided via a local file path. This prevents input files from referencing images as base64-encoded data URLs.I would like to add support for base64-encoded images, like
data:image/jpeg;base64,...ordata:image/png;base64,....This would make
llm_inferenceeasier to use across same filesystem. It would also simplify integration with evaluation frameworks such as LMMs-Eval, where images may already be available as base64-encoded data.My work-around implementation is available at: https://github.com/FABallemand/TensorRT-Edge-LLM/tree/feat/llm_inference_base64_support
Timeline
No hard deadline.
Describe alternatives you've considered
The current workaround is to save the image to a file and update the image path in the input file. This adds unnecessary filesystem I/O.
Adding native base64 image support to
llm_inferencewould provide a more interoperable solution while preserving the existing file-path behavior.Target hardware/use case
Jetson Thor (SM_110, JetPack 7.1, TensorRT 10.13.3.9).