Generative Scaling of Long-Tail Driving Data

Lulin Liu^* · Nuo Chen^* · Yan Wang · Bangya Liu · Wenyan Cong · Hezhen Hu
Boris Ivanovic · Hao Wang · Ziyao Zeng · Xinyu Gong · Yang Zhou · Zixiang Xiong
Dilin Wang · Zhangyang Wang · Weisong Shi · Ruohan Zhang · Marco Pavone · Zhiwen Fan^†

Texas A&M University · NVIDIA · University of Wisconsin–Madison · UT Austin · Yale University · Adobe · Meta · University of Delaware · Stanford University

^*Equal contribution ^†Corresponding author

TL;DR — From a single front-camera driving video, OpenLongTail synthesizes the five missing rig cameras at the same timestamps — turning monocular long-tail clips into pose-grounded, multi-view training data for robust VLA driving policies.

Release

• Checkpoint
• Demo
• Inference code
• Preprocessing & training code

Table of Contents

• Installation
• Demo
• Inference
• Building Training Data
• Training
• Citation

Installation

# 1. Clone
git clone https://github.com/phai-lab/OpenLongTail.git
cd OpenLongTail

# 2. Create the environment (Python 3.11, CUDA 12.8, bf16)
conda create -n openlongtail python=3.11 -y
conda activate openlongtail

# 3. Install a CUDA-matched PyTorch, then the rest of the deps
pip install torch==2.8.0 torchvision==0.23.0 --index-url https://download.pytorch.org/whl/cu128
pip install -r requirements.txt

Download the base backbone, the OpenLongTail checkpoint, and the demo data from Hugging Face in one step:

bash scripts/download.sh

Demo

Generate the 5 surround views for the bundled demo clips and build a PRED-vs-GT comparison gallery:

bash scripts/demo.sh                  # single GPU
bash scripts/demo.sh --gpus 0,1,2     # shard clips across GPUs
open demo/outputs/gallery/index.html

Or run end-to-end on your own front-camera video (depth and ego-pose are estimated automatically):

bash scripts/demo.sh --raw path/to/my_front.mp4

Inference

Generate the 5 synchronized non-front views from a single front-camera clip. Each clip produces the 5 generated views, the front input, and a preview grid.

# raw dash-cam clips (front video only)
bash scripts/infer.sh --input path/to/front_clips --output outputs/openlongtail_infer

# already-preprocessed clips
bash scripts/infer.sh --input path/to/cache --output outputs/openlongtail_infer --cached

The rear views reuse the front camera's earlier view of the same scene and are steered by an automatically generated scene caption, so the synthesized surround stays consistent with the input. Model and sampler settings can be overridden via environment variables (see the top of scripts/infer.sh).

Building Training Data

Preprocessing turns raw front-camera clips into a ready-to-train multi-view cache in one step: it recovers a metric ego-trajectory for each clip and reprojects the front-view evidence into every target camera (same-time for the side views, temporal look-back for the rear views).

bash scripts/preprocess.sh --input path/to/raw_clips --output data/openlongtail_cache

# shard across GPUs (run i = 0..N-1 in parallel)
bash scripts/preprocess.sh --input path/to/raw_clips --output data/openlongtail_cache --shards 8 --shard 0

# if your source already provides camera poses, skip pose recovery
bash scripts/preprocess.sh --input path/to/raw_clips --output data/openlongtail_cache --have-poses

The resulting cache is consumed directly by training.

Training

# Single node (torchrun), 8 GPUs → outputs/openlongtail_train
LATENT_CACHE_ROOT=data/openlongtail_cache OUTPUT_DIR=outputs/openlongtail_train bash scripts/train.sh

# short run on fewer GPUs
NUM_GPUS=4 NUM_STEPS=5000 OUTPUT_DIR=outputs/smoke bash scripts/train.sh

# multi-node (SLURM)
sbatch scripts/train.slurm

train.sh auto-resumes from the latest checkpoint under the output directory.

Citation

@misc{liu2026openlongtailgenerativescalinglongtail,
      title={OpenLongTail: Generative Scaling of Long-Tail Driving Data}, 
      author={Lulin Liu and Nuo Chen and Yan Wang and Bangya Liu and Wenyan Cong and Hezhen Hu and Boris Ivanovic and Hao Wang and Ziyao Zeng and Xinyu Gong and Yang Zhou and Zixiang Xiong and Dilin Wang and Zhangyang Wang and Weisong Shi and Ruohan Zhang and Marco Pavone and Zhiwen Fan},
      year={2026},
      eprint={2607.09655},
      archivePrefix={arXiv},
      primaryClass={cs.CV},
      url={https://arxiv.org/abs/2607.09655}, 
}

Acknowledgements

OpenLongTail builds on the following open-source projects:

• Wan2.1-VACE for the video-diffusion backbone.
• DepthCrafter for video-coherent depth used by the depth warp.
• MapAnything for camera-pose recovery on in-the-wild clips.
• Qwen2.5-VL for front-view scene captioning.

We thank the authors of these projects for releasing their code and models.

License

Released under the MIT License; the released weights are Apache-2.0, and base models remain under their respective licenses.