Many GPUs, particularly consumer ones, might not have enough VRAM to load everything all at once.
I have done an experiment that offloads UMT5 and only loads DiT and VAE separately. The result of running Wan 2.1 shows a maximum VRAM usage of about 5.63 GB, and it is tested and working on my 3060 Ti.
I know the goal of FlashDreams is focused on speed, but I wonder if people are interested in adding low VRAM support. I am currently adding it behind a --low-vram flag similar to ComfyUI, and I will try to benchmark the performance impact on a remote system with more VRAM.
Many GPUs, particularly consumer ones, might not have enough VRAM to load everything all at once.
I have done an experiment that offloads UMT5 and only loads DiT and VAE separately. The result of running Wan 2.1 shows a maximum VRAM usage of about 5.63 GB, and it is tested and working on my 3060 Ti.
I know the goal of FlashDreams is focused on speed, but I wonder if people are interested in adding low VRAM support. I am currently adding it behind a
--low-vramflag similar to ComfyUI, and I will try to benchmark the performance impact on a remote system with more VRAM.