Download models on the fly

[TimPietrusky]

Is your feature request related to a problem? Please describe.
The Docker images are very huge and users would like to just use the base image (which doesn't contain any models). This would make it more convenient for them to work on the image and add things, without the need to upload a huge image all the time when they make changes.

Describe the solution you'd like

• Specify the models they want via env variables
• Make sure that private models can also be handled
• Models are downloaded on first request to the worker or when the worker starts
• Models are downloaded onto the network-storage. If this doesn't exist, we will not download anything, as this would totally screw up the cold start time otherwise

Describe alternatives you've considered

• Use the base image
• Download the models manually onto the network storage
• Use the network storage with the deployed endpoint

Additional context
This idea came to life because of https://discord.com/channels/912829806415085598/1273963578369642557 and https://discord.com/channels/912829806415085598/1270792081580753047.

[jelling]

It's been awhile, but I recall trying this via a custom image I built. And the long initial download times caused the Runpod serverless process to restart the worker repeatedly... which then tried to download the image. Perhaps that's been changed, dunno.

A simpler, and more reliable way to accomplish your goal of a smaller image: mount your network volume using a CPU-only / cheap GPU pod, download the models to it, and then use the models from this image. But keep in mind that loading models from the NAS is much slower than on-disk.

[TimPietruskyRunPod]

There are now three supported ways to get models into the worker, ordered from most-to-least integrated:

1. Bake into image (recommended for production, fixed model set): add `wget`/`curl` lines in your derived Dockerfile. See `Dockerfile` lines 95-142 for the patterns we use for the official model variants.
2. Network volume (recommended for shared models across endpoints): pre-load to a Runpod network volume, attach to the endpoint. See https://github.com/runpod-workers/worker-comfyui/blob/main/docs/network-volumes.md (and PR Docs: Network volume path guidance, S3 upload, and test workaround #193 which is improving these docs further).
3. ComfyUI-to-API (recommended for rapidly iterating on workflows): https://comfy.getrunpod.io exports a ComfyUI workflow, detects all required models, and generates a custom worker image with everything pre-downloaded.

"On the fly" runtime model downloads from the worker itself are intentionally not supported — they would block job execution on a slow download, mask network failures as workflow validation errors, and create unpredictable cold-start times. ComfyUI-to-API gives you the developer ergonomics ("I just added a new model to my workflow, redeploy with it included") without those production downsides.

Closing as answered by existing supported flows.