fix(diffusers): explicitly set image save format for .tmp paths#10730
Open
treilhes wants to merge 5 commits into
Open
fix(diffusers): explicitly set image save format for .tmp paths#10730treilhes wants to merge 5 commits into
treilhes wants to merge 5 commits into
Conversation
Signed-off-by: treil <p.treilhes@free.fr>
Allow llama.cpp model configs to select the backend devices used for offload, matching upstream --device behavior so users can exclude a display or debug GPU. Signed-off-by: rvmzes <rvmzes@rvmzess-MacBook-Pro.local> Co-authored-by: rvmzes <rvmzes@rvmzess-MacBook-Pro.local> Signed-off-by: treil <p.treilhes@free.fr>
…udler#10725) The nvidia-l4t-cuda-13-arm64 vLLM backend left `vllm` unpinned, so the prebuilt image drifted onto whatever aarch64 wheel was latest at build time (0.23.x). On GB10 / DGX Spark (Grace Blackwell, unified memory), 0.23 crashes deterministically during cold model loads with an empty "Engine core initialization failed" set and pins GPU memory until a host reboot. vLLM 0.24.0 carries vllm-project/vllm#45179 ("release cached device memory under pressure on UMA GPUs during weight loading"), which the reporter verified fixes the crash on GB10. Pin the L4T requirements to 0.24.0 to match the already-pinned cublas13 build (requirements-cublas13-after.txt) and keep the image deterministic. Editing this file also re-triggers the single-arch L4T image build via the path filter, republishing the gallery image with 0.24.0 (the single-arch matrix builds again after mudler#10703). Closes mudler#10722 Assisted-by: Claude:claude-opus-4-8 [Claude Code] Signed-off-by: Ettore Di Giacinto <mudler@localai.io> Co-authored-by: Ettore Di Giacinto <mudler@localai.io> Signed-off-by: treil <p.treilhes@free.fr>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
3 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
This PR fixes #10727. This PR resolves an unhandled exception where the Python diffusers backend crashes when trying to save images to an engine-allocated temporary file. The core Go server generates an extensionless absolute file path ending in .tmp (e.g., /staging/localai-output-xxx.tmp) and passes it down via gRPC as request.dst.
Because image.save(request.dst) is called without an explicit format override parameter, the underlying Pillow library attempts to guess the image encoder strictly by slicing the file extension. Seeing .tmp, it panics with an "unknown file extension: .tmp" error, instantly crashing the image generation process right after inference completes.
Changes introduced:
Updated backend/python/diffusers/backend.py to intercept .tmp string paths.
Added validation to check if a valid "tmp_file_format" exists inside the incoming kwargs dictionary.
Safely forced Pillow to use the designated format encoder (normalized to uppercase) rather than relying on extension-guessing when saving a .tmp tracking asset.
Notes for Reviewers
I am completely new to this ecosystem (AI, Diffusers, LocalAI, etc.), so please review this PR with caution as I lack the technical depth to anticipate potential side effects. The parameter is currently passed through the "options" section, as it was directly accessible via the kwargs object, but I am not certain if this is the architecturally correct location for it.
PS: The documentation doesn't explicitly state which section parameter overrides should be added to (e.g., root level vs. the diffusers block), so I appended my parameter documentation to the existing paragraph for consistency.
Scope: The change is highly isolated and completely contained within the Python diffusers wrapper layer (backend.py). It has zero side-effects on the core Go binary or other unrelated non-image pipelines.
Fallback Safety: The logic safely falls back to standard image.save(request.dst) if the destination path does not end in .tmp or if tmp_file_format evaluates to None / empty string, preserving all current vanilla runtime behaviors.
Normalization: Added str(...).upper() wrapper to prevent Pillow runtime capitalization mismatches if lowercase configuration flags are supplied by upstream client layers.