Skip to content

arch(vulkan): record image transfers in caller command buffers - #125

Merged
chuzcjoe merged 1 commit into
masterfrom
daily/arch-vulkan-record-image-transfers-20261023
Oct 23, 2026
Merged

chuzcjoe merged 1 commit into
masterfrom
daily/arch-vulkan-record-image-transfers-20261023

Conversation

@chuzcjoe

Copy link
Copy Markdown
Collaborator

Problem

VulkanImage::TransitionImageLayout, VulkanBuffer::CopyToImage and VulkanImage::GenerateMipmaps each create, submit and synchronously complete their own command buffer. The existing DrawTexture, CubeMap and Mipmap loaders therefore require three submission/completion lifecycles for one logical upload. DrawTexture and Mipmap already identify this limitation in TODOs. Callers cannot compose these operations with each other or choose the completion boundary.

This PR addresses that concrete Vulkan execution boundary. It changes no shared dependency or independent graphics/compute backend and does not overlap open #124 (push constants).

Solution

  • Add VkCommandBuffer overloads for color layout transitions, buffer-to-image copies and mipmap generation. They only record commands; the existing synchronous signatures delegate to the same implementation and keep their blocking behavior.
  • Update DrawTexture, CubeMap and Mipmap to own one command buffer around the complete upload. Preserve the existing barriers, copy regions, supported layout pairs and mip blits. Reject a zero layer count before division or recording.
  • Keep submission, completion and GPU resource lifetime with the caller. No hidden layout tracker, global upload cache or additional owning upload/session abstraction is introduced.
Responsibility Before After in the three migrated loaders
Recording and completion Each individual operation One upload caller
Command pools/buffers, fences, queue submissions and waits per upload 3 of each 1 of each
Staging lifetime Each blocking call completes before returning Staging stays alive through the caller's final wait

The resource/submission counts follow directly from the call paths, rather than timing measurements. This is an architecture change; no latency, bandwidth, frame-rate or measured RSS improvement is claimed.

For a single-layer mipmapped texture, the existing Mipmap loader now uses:

const vulkan::VulkanCommandBuffer upload =
    vulkan::VulkanCommandBuffer::BeginOneTimeCommands(context_);
texture_image_.TransitionImageLayout(upload.buffer(), VK_IMAGE_LAYOUT_UNDEFINED,
                                    VK_IMAGE_LAYOUT_TRANSFER_DST_OPTIMAL,
                                    VK_FORMAT_R8G8B8A8_SRGB, mip_levels);
staging_buffer.CopyToImage(upload.buffer(), texture_image_, width, height);
texture_image_.GenerateMipmaps(upload.buffer());
upload.EndOneTimeCommands();

For a non-mipmapped texture, record the existing transfer-destination to shader-read transition instead of GenerateMipmaps; CubeMap supplies six layers to the transition and copy overloads. Callers can also record multiple uploads and submit with their own fence, as the integration test does.

A dedicated uploader would combine staging ownership and submission policy, increasing the lifetime/API surface for these existing callers. Native command-buffer composition matches the existing swapchain recording API and keeps one implementation for synchronous and caller-controlled paths. Pitched/partial image uploads were also evaluated but need a concrete streaming caller plus a format/block/pitch contract. Gaussian-blur optimization has a real 3000x4000 compute caller, but representative nonconstant workloads and repeated GPU baselines are still needed before choosing tiled or separable algorithms. Neither alternative was replaced with a guard-only PR.

Validation

Both complete builds and default test runs used the final source tree committed as c2572ca. scripts/run.sh requests Debug; top-level CMake actually selects Release with -O3 -DNDEBUG; the core warning flags include -Werror. All pinned submodules were initialized in this new Orca worktree. The existing ignored PNG fixture was copied read-only from the main checkout with matching SHA-256. No build ran in the user's checkout.

Platform Actual command Required test executables Passed / failed / skipped
macOS native ./scripts/run.sh -t macos -r tests 6: mat 3, threadpool 4, stb 4, opencl 8, vulkan 18, metal 1 38 / 0 / 0
Android physical device ./scripts/run.sh -t arm64-v8a -r tests 7: mat 3, threadpool 4, stb 4, opencl 8, vulkan 18, gles 2, ahardwarebuffer 1 40 / 0 / 0

Both commands exited 0. Checked each expected executable's actual invocation, GoogleTest start/completion/pass counts, and every RUN/OK pair. The Android runner uses set -e; reaching the next executable and final pull with exit 0 confirms successful adb shell process exit propagation, not just compilation.

macOS environment: Apple M6, MoltenVK 1.4.1, Apple Clang 21.0.0, macOS SDK 27.0, CMake 4.4.4, Vulkan SDK 1.4.350.0. Android environment: sole authorized USB physical 2210132C / nuwa, serial 1dda2807, Android 13 / API 33, Qualcomm hardware, empty ro.kernel.qemu; ANDROID_SERIAL fixed throughout. Existing NDK 28.2.13676358 / r28c, Android build API 34, JDK 17.0.20.1 and ADB 37.0.1. Preflight ran adb devices -l before device validation. Shell-local SDK setup explicitly supplied CMake, adb, glslc, slangc and xxd. No installation, license acceptance, shell-profile edits, APK installation or device-setting changes.

Three new GPU integration tests executed on both platforms:

  • Compare synchronous and recorded uploads byte-for-byte for 1x1, odd rectangular textures and six distinct cube faces. Verify base pixels against deterministic source data.
  • Read every generated mip and compare both paths for rectangular, odd and one-dimensional images, including the one-mip boundary; independently check constant-color mip chains.
  • Record two image uploads, change staging contents after recording and before explicit submission, then verify the GPU consumed the final contents. Repeat using the same command buffer after completion/reset. Unsupported layout and zero-layer rejection leave the recording usable.

The tests check completion fence results and require a queue family supporting graphics for the inherited fragment-stage barriers. Native Vulkan validation was requested; macOS lists the Khronos layer, and no validation errors were observed. Android layer availability was not independently established. The three graphical examples compiled in the default macOS build and are macOS-only CMake targets; windowed rendering was not visually exercised. Their upload operations were exercised headlessly by the integration tests on both platforms. No benchmark was required or run for this architecture acceptance.

An initial macOS run failed because this fresh worktree lacked tests/data/core.png; after copying and hashing the existing fixture, the entire macOS command was rerun successfully. The failed log is retained and excluded from passing counts. No source changes followed the two successful final runs. Separate complete logs, exit statuses, preserved macOS tmp outputs, device properties, verified test summaries and tested tree ID are retained under build/daily-image-recording-20261023/ in this worktree. git diff --check passed and the complete diff was reviewed.

Changes

File Added Deleted Category
examples/vulkan/CubeMapDemo/RenderCubeMap.cpp 6 3 implementation/API/examples
examples/vulkan/DrawTextureDemo/RenderTexture.cpp 7 4 implementation/API/examples
examples/vulkan/MipmapDemo/RenderModel.cpp 6 4 implementation/API/examples
tests/vulkan/VulkanImageRecordingTest.cpp 213 0 tests/benchmark
vulkan/include/VulkanBuffer.h 3 0 implementation/API/examples
vulkan/include/VulkanImage.h 6 0 implementation/API/examples
vulkan/src/VulkanBuffer.cpp 15 8 implementation/API/examples
vulkan/src/VulkanImage.cpp 24 16 implementation/API/examples

Implementation/API/examples: +67 / -35. Tests: +213 / -0. Benchmark/build/docs: 0. Total: 8 files, +280 / -35 = 315 changed lines, relative to freshly fetched origin/master f7070c0d9648b2dc7e583fe07e4868b80a57f609, below the 1000-line limit. No project Markdown or README edits, new dependency, generated binary or image asset is included.

Risk

Existing calls and blocking semantics remain available; class layouts and virtual interfaces are unchanged. Adding overloads means an untyped address-of-member expression must now specify its intended signature. Code choosing the new overloads must use an actively recording command buffer from the same device and an appropriate queue family, outside rendering, and keep the context, images, staging memory and command pool alive until GPU completion. Queue/command access requires external synchronization. Layout transitions are recorded intent, not a host-side state change; abandoning recording does not transition an image.

The inherited upload contract still requires correct usage flags, image sizes/formats, packed source/layer spacing and valid old layouts; this PR does not add general format/range validation. Color transitions retain their existing fragment-shader destination scope, so dedicated transfer/compute-only queues are not newly supported. Mipmap generation retains its existing single-layer, linear-blittable color-image contract and is not a cubemap mip generator. Cross-queue ownership transfer and asynchronous resource ownership are outside this change. The migrated callers explicitly wait before local staging destruction.

@chuzcjoe
chuzcjoe marked this pull request as ready for review October 23, 2026 17:32
@chuzcjoe
chuzcjoe merged commit 03119ea into master Oct 23, 2026
2 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant