Skip to content

arch(vulkan): reuse descriptor slots when rebinding resources - #136

Draft
chuzcjoe wants to merge 1 commit into
masterfrom
daily/arch-vulkan-rebind-descriptors-20261105
Draft

chuzcjoe wants to merge 1 commit into
masterfrom
daily/arch-vulkan-rebind-descriptors-20261105

Conversation

@chuzcjoe

@chuzcjoe chuzcjoe commented Nov 5, 2026

Copy link
Copy Markdown
Collaborator

Problem

VulkanBase's descriptor preparation owns backing data by call order: every buffer/image helper advances a counter, while writes_ is indexed directly by the shader binding number. Calling a helper again can overrun its backing vector; sparse binding numbers also index outside writes_. The lifetime of descriptor metadata is therefore coupled to a single dense, one-shot initialization sequence.

The existing Roofline ComputeCopy consumer fixes its buffer pair at initialization. Reusing its shader/pipeline for another completed copy requires a safe replacement path rather than rebuilding the pipeline or bypassing the helpers.

Three Vulkan candidates were compared:

Candidate Evidence and decision
Descriptor metadata ownership and rebinding (architecture, selected) Concrete call-order coupling in VulkanBase and an existing ComputeCopy consumer. Stable per-binding storage plus a completed-submission update contract enables an immediately usable reuse path and direct GPU verification.
Partial/pitched texture upload (feature) Existing texture loaders upload complete images; a real incremental consumer and format/pitch contract are still needed. Deferred rather than inventing a placeholder API.
Release CPU model geometry (architecture) Model renderers retain CPU vectors used for draw counts. Requires a reinitialization/count contract and is adjacent to open #133. Deferred pending that review.

Checked recent commits, open issues, open #132–#135 and closed #117/#129. This does not revive the closed submission-session/mapping proposals or modify the open changes. Scope is Vulkan and its existing tests; no shared dependency or other backend implementation changes.

Solution

  • Allocate descriptor metadata once from the declared layout during Init. Each declaration has a stable write slot and stable buffer/image backing slots for the pipeline object's lifetime. Resolve helper calls by binding number and descriptor type; repeated preparation replaces that binding's metadata. Declaration order and sparse binding numbers are supported independently of preparation order.
  • Reject duplicate layout bindings before driver allocation. Reject preparation before initialization, undeclared bindings and mismatched helper types before changing metadata. Keep all three existing helper signatures and full-buffer descriptor semantics.
  • Remove append counters and lazy resize helpers. The old mutex protected only lazy write-vector sizing, not descriptor/queue operations; these remain externally synchronized. Explicitly delete copying to preserve the noncopyable handle ownership previously imposed by that mutex.
  • Add ComputeCopy::BindBuffers(src, dst) and use it from the existing Init path. It updates the storage pair and submits the prepared writes without recreating the pipeline, layout, pool or descriptor set. The existing constructor, Init and Run calls continue to work. Reject undersized/aliased buffers and nonpositive copy counts before changing the pair.

After an earlier submission has completed, the concrete reuse sequence is:

VK_CHECK(vkWaitForFences(context.logical_device, 1, &fence.fence, VK_TRUE, UINT64_MAX));
command.Reset();
copy.BindBuffers(next_input, next_output);
fence.Reset();
command.BeginCommandBuffer();
copy.Run(command.buffer());
command.Submit(fence.fence, core::vulkan::SubmitSyncInfo{});

Retain the caller's required memory barriers and resource lifetimes. Init runs once. The buffers must be distinct, belong to the same context/device, have storage usage and sufficient capacity for the constructor's fixed vec4 count. Binding does not own the resources or synchronize GPU work.

Wait for every use of the descriptor set to complete and reset/re-record affected command buffers before further submission. Updating ordinary descriptors invalidates command buffers that already reference them, and updating pending uses is forbidden by Vulkan's descriptor update contract. Multiple dispatches recorded against one set do not capture different buffer snapshots; this is sequential reuse, not update-after-bind or multiple simultaneous frames.

A separate descriptor-set manager or per-frame sets would add ownership/API scope needed for in-flight replacement. Dynamic offsets address buffer subranges rather than this replacement problem. Rebuilding pipelines keeps the one-shot coupling. Stable slots address the current design with the existing single-set model. Linear lookup is bounded by the declared binding count; no new dependency or optional device feature is required.

Validation

Both full runs used the final staged source tree, with no source changes afterward. All default enabled libraries, examples and tests built. The runner requests Debug; actual core flags are C++20, -O3 -DNDEBUG, warnings as errors.

Platform Actual command Executed tests Result
macOS native ./scripts/run.sh -t macos -r tests mat 3, threadpool 4, stb 4, opencl 8, vulkan 43, metal 1 6 executables / 24 GoogleTest suites; 63 passed, 0 failed, 0 skipped; exit 0
Android arm64-v8a physical device ./scripts/run.sh -t arm64-v8a -r tests mat 3, threadpool 4, stb 4, opencl 8, vulkan 43, gles 2, ahardwarebuffer 1 7 device executables / 26 suites; 65 passed, 0 failed, 0 skipped; exit 0

macOS: Apple M6 / MoltenVK 1.4.1, macOS 27.0.1, installed Xcode SDK 27.2 selected through shell-local SDKROOT, Apple Clang 21.0.0 (clang-2100.4.1.1), CMake 4.4.4, Vulkan SDK 1.4.350.0, shaderc 2026.2, Slang 2026.8. Khronos validation 1.4.350 was available/requested; no validation diagnostics observed.

Android: adb devices -l before device runs confirmed exactly one authorized USB physical 2210132C / nuwa, serial 1dda2807, fixed with ANDROID_SERIAL; Android 13 / API 33, qcom hardware and empty emulator property. Reused installed NDK 28.2.13676358 (r28c), Android Clang 19.0.1, build target API 34, JDK 17 and ADB 37.0.1. Loaded the existing SDK setup and explicitly supplied PATH entries for CMake, adb, glslc, slangc and xxd. No dependency installation, license acceptance, shell-profile changes, APK installation or device-setting changes. Android validation-layer availability was not independently established.

Verified each actual local/adb-shell test invocation, GoogleTest start/end/pass totals, individual RUN/OK names and the set-e runner's successful process exits. Separate platform logs and macOS tmp outputs preserved in the worktree. New shader independently compiled and passed spirv-val --target-env vulkan1.1.

Three new GPU integration tests establish the design benefit:

  • Actual ComputeCopy: one pipeline/layout reused for six completed copies rotating among three buffers; 513 vec4 elements exercise a partial workgroup. Every copied word matches, seven guard words remain untouched, and every source remains unchanged. Rejected short/aliased replacements leave the selected pair usable for the subsequent checked dispatch.
  • Sparse mixed layout: declaration order 21/7/3/11 differs from preparation order. Two independent uniform/input/output/image sets alternate through 20 replacements per round for four completed rounds, covering all three helpers. GPU output matches the independent integer/texel oracle; unused outputs and trailing words remain untouched. Unknown/type-mismatched attempts leave valid metadata usable, with a stable write vector and unchanged pipeline handle.
  • Duplicate layout rejection happens before allocation; pre-init rejection and compile-time noncopyable ownership checks also pass. Existing dense-layout compute/render consumers retain their behavior in the full suites.

Architecture acceptance uses resource identity, metadata lifetime and complete GPU output evidence. No latency, throughput, FPS, allocation-byte or RSS improvement is claimed; no performance benchmark was needed. The early exploratory build overlapped development and is not treated as an unmodified baseline. Full final runs followed all source/formatting changes. Windowed examples were built, not visually exercised; additional vendors, descriptor arrays, in-flight replacement and device-loss injection were not tested.

Changes

File Added Deleted Category
tests/vulkan/DescriptorRebinding.comp 13 0 tests/shader
tests/vulkan/Roofline/ComputeCopy.cpp 18 3 implementation/caller
tests/vulkan/Roofline/ComputeCopy.h 1 0 implementation/caller
tests/vulkan/VulkanDescriptorRebindingTest.cpp 298 0 tests/shader
vulkan/include/VulkanBase.h 4 55 implementation/caller
vulkan/src/VulkanBase.cpp 57 45 implementation/caller

Implementation/caller +80/-103; tests/shader +311/-0; benchmark/build/documentation +0/-0. Total 6 files, +391/-103 = 494 changed lines, relative to freshly fetched origin/master 0dc12da8ee3f6b69ed18a0e6c817e9e436e85906, below the 1000-line cap. No project Markdown/README changes, added code comments or generated assets/binaries.

Risk

Existing public/helper signatures and the single-set model remain. VulkanBase object layout changes, so binary clients must rebuild. Protected backing vectors and writes now use declaration order rather than packed-by-type/numeric-binding indices; external subclasses that manipulate these internals directly must use the helpers or adapt their indexing. All repository consumers use helpers and whole-vector updates.

Each declaration reserves both a buffer-info and image-info slot, trading a small amount of extra CPU metadata for stable simple indexing. Lookup is linear. Descriptor metadata is retained until pipeline destruction; no unbounded update history accumulates. Pipeline objects remain noncopyable and nonmovable. Init is not a reinitialization API.

The context must outlive the pipeline and resources; callers retain descriptor resource lifetimes, device/usage/format validity, host/queue synchronization and required memory barriers. Populate every declaration before a whole-vector descriptor update. This change does not make pending descriptor updates or cross-device resources valid and does not add automatic synchronization, reflection, arrays, dynamic offsets or resource ownership. Invalid replacement inputs do not update the selected pair; device/driver failures are not recovered here.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant