large image inference

[Galaxy8172]

Your work is excellent, and the test cases demonstrate the model's strong performance. However, when I tested it on a very large image (6k*8k), I set the tile size to 512 and the tile overlap to 16, using a 1.4B model. Whether I used single-step or multi-step inference, the results showed obvious tiling artifacts, and some tiles simply produced random colorful noise. Have you encountered this issue?

[cswry]

Thanks for reporting this. I’ve fixed the bug. Please try again and let me know if the issue still persists.

[Galaxy8172]

Thank you very much! I've tried the latest engineering, and the previous issue was indeed perfectly resolved. However, I've discovered another problem: I noticed that you moved the tile-related operations into the diffusion process. This means my VAE Encoder now needs to process the full image as input. When I tested with a resolution of 1472×1984 and an upscale factor of 1, it already consumed approximately 40GB of GPU memory. When I increased the upscale factor to 2, an OOM error occurred. Here is the code I used:
CUDA_VISIBLE_DEVICES=0 python inference_vosr.py
-c preset/ckpts/VOSR_1.4B_ms
-i preset/datasets/inp_data/
-o preset/results
-u 1 --tile_size 512

Could you please help me optimize this or suggest a way to handle full-image encoding with higher resolutions? I'm very much looking forward to your answer.

[minh-nguyenhoang]

Thank you very much! I've tried the latest engineering, and the previous issue was indeed perfectly resolved. However, I've discovered another problem: I noticed that you moved the tile-related operations into the diffusion process. This means my VAE Encoder now needs to process the full image as input. When I tested with a resolution of 1472×1984 and an upscale factor of 1, it already consumed approximately 40GB of GPU memory. When I increased the upscale factor to 2, an OOM error occurred. Here is the code I used: CUDA_VISIBLE_DEVICES=0 python inference_vosr.py -c preset/ckpts/VOSR_1.4B_ms -i preset/datasets/inp_data/ -o preset/results -u 1 --tile_size 512

Could you please help me optimize this or suggest a way to handle full-image encoding with higher resolutions? I'm very much looking forward to your answer.

You can enable vae tiling if using diffusers VAE or refer to here for tiled encode/inference/decode.