Skip to content

GOU-02 memory + GpuPack marshal substrate #20

Closed

Description

@K-T0BIAS

Goal

Ship the permanent device-memory and pack layer used by every later launch:

  • memory:: helpers for buffer lifetimes and copies
  • Transfer engine (command pool + fence): staging <-> device-local copies
  • GpuPack (option 5): scalar SSBO + N list SSBOs
  • Marshal round-trip: Python scalars/list -> pack -> writeback into the same objects

No public buffer type. No SPIR-V yet. No host-visible-only “temporary” SSBO path that later gets deleted, shader-facing buffers are device-local from day one; staging is the host mirror used for copies.

Todos

  • Extend Context with all buffer / memory / command-pool / fence entry points needed for transfers (load once in init, clear in shutdown).
  • memory::Buffer: owns VkBuffer + VkDeviceMemory + size; distinguish usage kind (device-local storage vs host-visible staging) via create parameters; one API, two property sets, not two throwaway designs.
  • find_memory_type; create_buffer / destroy_buffer.
  • Staging: host-visible + coherent; may stay mapped for memcpy into staging only.
  • Device-local: VK_MEMORY_PROPERTY_DEVICE_LOCAL_BIT; STORAGE_BUFFER + TRANSFER_SRC + TRANSFER_DST; not persistently mapped for upload.
  • TransferEngine (or equivalent on context): one-time command pool on the compute queue family; record vkCmdCopyBuffer; submit; wait fence. Used by upload_buffer / download_buffer for device-local targets.
  • upload_buffer / download_buffer: final semantics, host bytes <-> device-local SSBO via staging + GPU copy (bounds checks, clear errors).
  • GpuPack: create/destroy; scalar region = one device-local SSBO with fixed std430-compatible host mirror struct; each list[float|int] = one device-local SSBO; upload/download/writeback helpers.
  • Empty-list rule (document and implement once): e.g. no VkBuffer / zero-size rejected, pick one permanent rule.
  • Internal / test-only bindings (not product API): round-trip scalars + lists through GpuPack.
  • Typed errors: OOM, bad size/dtype, use-after-destroy, not built / not available.
  • Pytest: round-trips; empty list; skip if not available().
  • CMake: compile all new GPU sources under CTHREADS_GPU.
  • Migrate any WIP host-visible-mapped-as-SSBO code to this model in the same issue (do not leave a parallel path).

Notes

  • Interface: users never see Vulkan or DeviceBuffer.
  • Arch: Context = device connection; memory:: = stateless buffer ops; transfer engine = reusable copy path; GpuPack = one launch’s scalar + list SSBOs.
  • Reject: BDA / pointers inside the scalar struct; one monolithic pack buffer; public device buffer type; “map the SSBO and memcpy forever” as the production list path.

Metadata

Metadata

Assignees

No one assigned

    Labels

    sub-featurePiece of a larger enchancement

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions