Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
6 changes: 3 additions & 3 deletions docs/PORT-AMPERE.md
Original file line number Diff line number Diff line change
Expand Up @@ -44,10 +44,10 @@ Blackwell header.
| Blackwell-only instruction semantics | BLOCKED | Blackwell FP8 conversion / QMMA forms have no same-semantics sm_86 encoding. The sm_86 encoder refuses them rather than inventing bytes or silently executing a different operation. Existing Blackwell-specialized model/tensor paths are not made Ampere-capable by changing an ELF flag. A real Ampere algorithm lowering is required before those paths are enabled. |
| Cubin/ELF target | DONE | `om/cubin/cubin_elf.om:26` selects the parameter base; `:269` selects CUDA ELF OSABI/ABI and e_version; `:286` sets the sm_86 flags. `omega_cubin_pack_target` is the target-explicit host boundary in `src/omega_cubin.c`. The old sm_121 entry remains explicit/default-compatible. `mk/ampere-cubin.mk` checks ELF fields and parameter metadata, differential behavior, and offline nvdisasm loading. |
| Capability-mercury sections | DONE | The sm_86 writer emits no `.nv.capmerc.*` / `.nv.merc.*` sections. The Blackwell driver-loader requirement observed for sm_121 must not be generalized to Ampere. Successful offline ELF decoding is not evidence of successful `cuModuleLoadData` or direct RM launch; both RTX loader/device paths remain NOT_RUN. |
| Compute/FIFO classes and QMD | DONE | `src/omega_ampere_qmd.h:5` records C6C0 versus C7C0 and Ampere FIFO constants. `om/encoder/amp_runtime.om` implements target admission, QMD3.0 layout and bounded command admission. `src/omega_gpu_session.c:562` emits the Ampere QMD and `:599` its PCAS launch instead of Blackwell QMD5/inline packets. `nvrm/nvrm.c:69` requires the queried class list to contain the selected target's classes. Raw Blackwell packets on an Ampere device are refused before ring submission (`nvrm/nvrm.c:1055`). |
| Discrete allocation and copies | DONE | `nvrm/nvrm.c:631` selects local framebuffer memory on sm_86, RM maps it into GPU VA and maps its CPU BAR alias through real RM_MAP_MEMORY/mmap. Queue and host-polled completion storage remain system memory. `nvrm/nvrm.c:746` / `:755` provide live-identity/bounds-checked CPU→BAR and BAR→CPU transfers with ordering fences. The session may write/read that uncached BAR view directly; platform sync (`src/omega_gpu_platform_linux.c:358`) orders visibility rather than pretending to copy coherent host RAM. No copy engine is claimed. |
| Compute/FIFO classes and QMD | DONE | `src/omega_ampere_qmd.h:5` records C6C0 versus C7C0 and Ampere FIFO constants. `om/encoder/amp_runtime.om` implements target admission, QMD3.0 layout and bounded command admission. `src/omega_gpu_session.c:562` emits the Ampere QMD and `:599` its PCAS launch instead of Blackwell QMD5/inline packets. `nvrm/nvrm.c:71` requires the queried class list to contain the selected target's classes. sm_121 is admitted for the GR SM-version field 12_01, or for 10_04 (0xa04, what the GB10 reports at driver 580.173.02; `ctrl2080gr.h` keeps 10_04 until its 12_* names replace it) only when the PCI identity is 0x10de:0x2e12, and always with the complete Blackwell-B class set; `tests/test_ampere_runtime.c` refuses 10_04 with another vendor or device id. Raw Blackwell packets on an Ampere device are refused before ring submission (`nvrm/nvrm.c:1059`). |
| Discrete allocation and copies | DONE | `nvrm/nvrm.c:634` selects local framebuffer memory on sm_86, RM maps it into GPU VA and maps its CPU BAR alias through real RM_MAP_MEMORY/mmap. Queue and host-polled completion storage remain system memory. `nvrm/nvrm.c:739` / `:748` provide live-identity/bounds-checked CPU→BAR and BAR→CPU transfers with ordering fences. The session may write/read that uncached BAR view directly; platform sync (`src/omega_gpu_platform_linux.c:358`) orders visibility rather than pretending to copy coherent host RAM. No copy engine is claimed. |
| Cached local mappings / large BAR availability | BLOCKED | Cached Ampere mappings are explicitly refused until their cache-maintenance contract is implemented. CPU BAR mapping availability and large-allocation behavior need the real driver/card configuration; failure is surfaced, not replaced by a fake host allocation. First-run tests use bounded allocations. |
| VRAM-aware D1 reserve | DONE | `nvrm/nvrm.c:637` queries the selected framebuffer heap. `src/omega_gpu_platform_linux.c:233` reads reserve policy and `:248` uses free VRAM, never host MemAvailable, for local allocations. `CUDAPP_GPU_MIN_FREE_BYTES` supports a precise safe refusal; the default MiB reserve remains documented by the runtime. The refusal precedes the driver allocation call. |
| VRAM-aware D1 reserve | DONE | `nvrm/nvrm.c:640` queries the selected framebuffer heap. `src/omega_gpu_platform_linux.c:233` reads reserve policy and `:248` uses free VRAM, never host MemAvailable, for local allocations. `CUDAPP_GPU_MIN_FREE_BYTES` supports a precise safe refusal; the default MiB reserve remains documented by the runtime. The refusal precedes the driver allocation call. |
| Safe reserve refusal proof | DONE | `tests/ampere/memory_refusal.c:39` sets reserve above total VRAM (therefore above free VRAM even if other processes release memory), then requests only a small allocation. It requires refusal, zero intervening RM allocation ioctls and unchanged free VRAM. Simulated-driver coverage is separate from this real-device test. It never pressure-allocates VRAM. |
| Open-module and device selection doctor | DONE | `tests/ampere/doctor.py:79` verifies Linux x86_64, the exact pinned open module, requested PCI IDs, device nodes and an unambiguous BDF. `CUDAPP_GPU_BDF` selects that exact GPU in the driver client. `test_doctor.py` uses a fake filesystem and is always SIMULATED. No driver is opened by the doctor. |
| Portable Omega build provenance | IMPLEMENTED | `om/units.sh` covers all canonical hosted roots plus Ampere encoding/runtime and the separate revision/epoch/f16 freestanding profile. Checked portable generated translation units replace executable-code trampolines. `om/omega-bridge.lock`, per-unit identities and generated declarations retain exact composed-source SHA → C SHA provenance. Missing/mismatched converter, profile or seed refuses before installation. Final host acceptance is recorded separately below. |
Expand Down
9 changes: 6 additions & 3 deletions nvrm/nvrm.c
Original file line number Diff line number Diff line change
Expand Up @@ -56,13 +56,15 @@ static int fail(Nvrm *rm, const char *fmt, ...);

/* Pure class admission lives in seeded Omega. Class IDs are vendor constants:
* open-gpu-doc clc7c0.h, clc56f.h, and vendored clc361.h / clc661.h. */
static int select_classes(Nvrm *rm, const uint32_t *classes, uint32_t count) {
static int select_classes(Nvrm *rm, const uint32_t *classes, uint32_t count,
uint16_t vendor_id, uint16_t device_id) {
rm->compute_class = rm->gpfifo_class = rm->usermode_class = 0;
rm->target_sm = 0; rm->discrete = 0;
uint64_t m[16] = {1, rm->sm_version,
NV2080_CTRL_GR_INFO_SM_VERSION_8_06, NV2080_CTRL_GR_INFO_SM_VERSION_12_01,
OMEGA_AMPERE_COMPUTE_B, OMEGA_AMPERE_GPFIFO_A, VOLTA_USERMODE_A,
BLACKWELL_COMPUTE_B, BLACKWELL_CHANNEL_GPFIFO_A, HOPPER_USERMODE_A};
BLACKWELL_COMPUTE_B, BLACKWELL_CHANNEL_GPFIFO_A, HOPPER_USERMODE_A,
0, 0, NV2080_CTRL_GR_INFO_SM_VERSION_10_04, vendor_id, device_id};
/* Marshal the vendor's class-list presence, without selecting a target. */
for (uint32_t i = 0; i < count; i++) {
for (uint32_t k = 0; k < 3; k++) {
Expand Down Expand Up @@ -427,7 +429,8 @@ int nvrm_open(Nvrm *rm) {
if (cl.numClasses > 512) return fail(rm, "too many classes");
cl.classList = NV_PTR_TO_NvP64(classes);
if (rm_control(rm, rm->device, NV0080_CTRL_CMD_GPU_GET_CLASSLIST, &cl, sizeof cl)) return -1;
if (cl.numClasses > 512 || select_classes(rm, classes, cl.numClasses)) return -1;
if (cl.numClasses > 512 || select_classes(rm, classes, cl.numClasses,
cards[found].pci_info.vendor_id, cards[found].pci_info.device_id)) return -1;
if (rm_alloc(rm, rm->subdevice, rm->usermode_class, NULL, 0, &rm->usermode)) return -1;
void *mmio = map_to_cpu(rm, rm->usermode, 0x10000, NULL, 0, 0);
if (mmio == MAP_FAILED) return -1;
Expand Down
7 changes: 6 additions & 1 deletion om/encoder/amp_runtime.om
Original file line number Diff line number Diff line change
Expand Up @@ -3,6 +3,7 @@ note Pure Ampere runtime decisions and QMD serialization. Host ioctls stay in nv
note Vendor fields: NVIDIA open-gpu-doc classes/compute/clc7c0qmd.h QMDV03_00.
note m[0] operation: 1 class-set admission, 2 byte reserve admission, 3 QMD.
note Class selection m[1]=rawSM,m[2..3]=vendorSM86/121,m[4..9]=Ampere then Blackwell classes,m[10..11]=presence masks.
note m[12]=vendorSM10.04,m[13..14]=PCI vendor/device. GB10 identity follows the existing qualified chip row.
note Reserve m[1]=available bytes, m[2]=request bytes, m[3]=reserve bytes.
note QMD m[1..9]=code,cbank,semaphore,threadsX,threadsY,gridX,gridY,registers,sharedBytes.
note q is 64 zero-extended uint words, not a native struct. Return 1 accepted or 0 refused.
Expand Down Expand Up @@ -30,7 +31,11 @@ put q 3 get m 5
put q 4 get m 6
ret 1
end
if eq get m 1 get m 3
gb10 = false
if and ne get m 12 0 eq get m 1 get m 12
gb10 = and eq get m 13 4318 eq get m 14 11794
end
if or eq get m 1 get m 3 gb10
if ne get m 11 7
ret 0
end
Expand Down
6 changes: 5 additions & 1 deletion src/cudapp_om_amp_runtime.c
Original file line number Diff line number Diff line change
Expand Up @@ -79,7 +79,11 @@ return o_leave(0ULL);
{uint64_t _i=4ULL;uint64_t _v=o_get(v0, v0_n, 6ULL, 5);o_put(v1, v1_n, _i,_v,5);}
return o_leave(1ULL);
}
if(({uint64_t _a=o_get(v0, v0_n, 1ULL, 5);uint64_t _b=o_get(v0, v0_n, 3ULL, 5);o_op(6, 5, _a,_b);})){
uint64_t __attribute__((unused)) v2=0ULL;
if(({uint64_t _a=({uint64_t _a=o_get(v0, v0_n, 12ULL, 5);uint64_t _b=0ULL;o_op(7, 5, _a,_b);});uint64_t _b=({uint64_t _a=o_get(v0, v0_n, 1ULL, 5);uint64_t _b=o_get(v0, v0_n, 12ULL, 5);o_op(6, 5, _a,_b);});o_op(12, 1, _a,_b);})){
v2=({uint64_t _a=({uint64_t _a=o_get(v0, v0_n, 13ULL, 5);uint64_t _b=4318ULL;o_op(6, 5, _a,_b);});uint64_t _b=({uint64_t _a=o_get(v0, v0_n, 14ULL, 5);uint64_t _b=11794ULL;o_op(6, 5, _a,_b);});o_op(12, 1, _a,_b);});
}
if(({uint64_t _a=({uint64_t _a=o_get(v0, v0_n, 1ULL, 5);uint64_t _b=o_get(v0, v0_n, 3ULL, 5);o_op(6, 5, _a,_b);});uint64_t _b=v2;o_op(13, 1, _a,_b);})){
if(({uint64_t _a=o_get(v0, v0_n, 11ULL, 5);uint64_t _b=7ULL;o_op(7, 5, _a,_b);})){
return o_leave(0ULL);
}
Expand Down
2 changes: 1 addition & 1 deletion src/cudapp_om_amp_runtime.ident
Original file line number Diff line number Diff line change
@@ -1 +1 @@
omega-ident 2 unit=amp_runtime abi=bridge-seeded src=841f6da18d2ea43d42f4d7434b4ca84e5168b5d96d3527628f79e2f77cbb265c c=6f3cf40c3f759ac380e6e2e46fbeed75cb2206c26bcf32db33ade6dc0dae7476 omc=2c82480596bbdc50cda9626866209ef662d5834f278f839ff4ce0250c495029f converter=3a416aea14770bd3fab4880337e89020c10476214d7a3c139d8cd1a8ba1423e6 frontend=23ff7a2356bb5d980060a4685a27ac46587013069b7d24b37c2873d7ee659b37 path=om/encoder/amp_runtime.om
omega-ident 2 unit=amp_runtime abi=bridge-seeded src=452f52245a62cc7f50b4a14cf8475aa3ca829cef52a71342632190d926226cc5 c=54302a88d9a9a9efbdf0c401fde6fc88d6518a0105c83d86916e7b4a73b1bbda omc=2c82480596bbdc50cda9626866209ef662d5834f278f839ff4ce0250c495029f converter=3a416aea14770bd3fab4880337e89020c10476214d7a3c139d8cd1a8ba1423e6 frontend=23ff7a2356bb5d980060a4685a27ac46587013069b7d24b37c2873d7ee659b37 path=om/encoder/amp_runtime.om
2 changes: 1 addition & 1 deletion src/cudapp_om_units.ident
Original file line number Diff line number Diff line change
@@ -1,7 +1,7 @@
omega-ident 2 unit=bw_enc_sm121 abi=bridge-seeded src=901e9dc7921663fa5e91d2df2267910c85d7933acc9421e55828f55fbba6c946 c=f0270d0a3742b310301233954fdd5d28f216e8c4ade64eb7770b5db9b99c1c5a omc=2c82480596bbdc50cda9626866209ef662d5834f278f839ff4ce0250c495029f converter=3a416aea14770bd3fab4880337e89020c10476214d7a3c139d8cd1a8ba1423e6 frontend=23ff7a2356bb5d980060a4685a27ac46587013069b7d24b37c2873d7ee659b37 path=om/encoder/bw_enc_sm121.om
omega-ident 2 unit=cubin_elf abi=bridge-seeded src=7c9bacf500bc116b110bbf884041005f1fb957ca3f34495e3e01a1d94bc1683e c=da7a5521b9f7953eb165cd26d4d8a38a0ffa49e5ca25009d0a4306aa6185f42a omc=2c82480596bbdc50cda9626866209ef662d5834f278f839ff4ce0250c495029f converter=3a416aea14770bd3fab4880337e89020c10476214d7a3c139d8cd1a8ba1423e6 frontend=23ff7a2356bb5d980060a4685a27ac46587013069b7d24b37c2873d7ee659b37 path=om/cubin/cubin_elf.om
omega-ident 2 unit=ampere_enc_sm86 abi=bridge-seeded src=fe3b212baa4afe9eb18b2a8c11a174fcf52a9a247a66d859bf32cabe0cd1ea64 c=a9db496848193bdcad72f9e4676a398466bc717233da8f22db50b3ab54e92166 omc=2c82480596bbdc50cda9626866209ef662d5834f278f839ff4ce0250c495029f converter=3a416aea14770bd3fab4880337e89020c10476214d7a3c139d8cd1a8ba1423e6 frontend=23ff7a2356bb5d980060a4685a27ac46587013069b7d24b37c2873d7ee659b37 path=om/encoder/ampere_enc_sm86.om
omega-ident 2 unit=amp_runtime abi=bridge-seeded src=841f6da18d2ea43d42f4d7434b4ca84e5168b5d96d3527628f79e2f77cbb265c c=6f3cf40c3f759ac380e6e2e46fbeed75cb2206c26bcf32db33ade6dc0dae7476 omc=2c82480596bbdc50cda9626866209ef662d5834f278f839ff4ce0250c495029f converter=3a416aea14770bd3fab4880337e89020c10476214d7a3c139d8cd1a8ba1423e6 frontend=23ff7a2356bb5d980060a4685a27ac46587013069b7d24b37c2873d7ee659b37 path=om/encoder/amp_runtime.om
omega-ident 2 unit=amp_runtime abi=bridge-seeded src=452f52245a62cc7f50b4a14cf8475aa3ca829cef52a71342632190d926226cc5 c=54302a88d9a9a9efbdf0c401fde6fc88d6518a0105c83d86916e7b4a73b1bbda omc=2c82480596bbdc50cda9626866209ef662d5834f278f839ff4ce0250c495029f converter=3a416aea14770bd3fab4880337e89020c10476214d7a3c139d8cd1a8ba1423e6 frontend=23ff7a2356bb5d980060a4685a27ac46587013069b7d24b37c2873d7ee659b37 path=om/encoder/amp_runtime.om
omega-ident 2 unit=repack abi=bridge-seeded src=ab03c810d7517ef3c2db9083f5073851cfda77f0a7f7bd83269dcc43e6be64ae c=c4037b20a46345b86c404be57defe318da8d6a9695349eb5ff3f1bc517dd3d5f omc=2c82480596bbdc50cda9626866209ef662d5834f278f839ff4ce0250c495029f converter=3a416aea14770bd3fab4880337e89020c10476214d7a3c139d8cd1a8ba1423e6 frontend=23ff7a2356bb5d980060a4685a27ac46587013069b7d24b37c2873d7ee659b37 path=repack=om/quant/quant.om,om/quant/repack.om
omega-ident 2 unit=quant_wire abi=bridge-seeded src=657e765de09ad4962165b11a18d7e8fb19b650aa2d7bb321ac11d27b8a63a3c5 c=97ca9ef6470cd8d1bbb33781c4e09c64d7a977339553f7f8d404be612a98b1c0 omc=2c82480596bbdc50cda9626866209ef662d5834f278f839ff4ce0250c495029f converter=3a416aea14770bd3fab4880337e89020c10476214d7a3c139d8cd1a8ba1423e6 frontend=23ff7a2356bb5d980060a4685a27ac46587013069b7d24b37c2873d7ee659b37 path=quant_wire=om/quant/quant.om,om/quant/runtime.om
omega-ident 2 unit=compose abi=bridge-seeded src=91cfa4c561ee04a8db7cb70e43829eeb19df2e4aa1030573a762242c788195e0 c=3ff9e84cea668f9ad53693e10fe6af67654d8c9918ba27f6c05be22c059c4dbe omc=2c82480596bbdc50cda9626866209ef662d5834f278f839ff4ce0250c495029f converter=3a416aea14770bd3fab4880337e89020c10476214d7a3c139d8cd1a8ba1423e6 frontend=23ff7a2356bb5d980060a4685a27ac46587013069b7d24b37c2873d7ee659b37 path=compose=om/components/bw_alloc.om,om/components/bw_effect.om,om/components/bw_check.om,om/components/bw_project.om,om/components/rowpipe.om,om/components/staged.om,om/components/compose.om,om/components/runtime.om
Expand Down
20 changes: 15 additions & 5 deletions tests/test_ampere_runtime.c
Original file line number Diff line number Diff line change
Expand Up @@ -103,14 +103,24 @@ static void class_and_bdf(void) {
const uint32_t amp[] = {OMEGA_AMPERE_COMPUTE_B, OMEGA_AMPERE_GPFIFO_A, VOLTA_USERMODE_A};
const uint32_t bw[] = {BLACKWELL_COMPUTE_B, BLACKWELL_CHANNEL_GPFIFO_A, HOPPER_USERMODE_A};
rm.sm_version = NV2080_CTRL_GR_INFO_SM_VERSION_8_06;
assert(select_classes(&rm, amp, 3) == 0 && rm.target_sm == 86 && rm.discrete);
assert(select_classes(&rm, amp, 3, 0, 0) == 0 && rm.target_sm == 86 && rm.discrete);
assert(rm.compute_class == OMEGA_AMPERE_COMPUTE_B && rm.gpfifo_class == OMEGA_AMPERE_GPFIFO_A);
assert(select_classes(&rm, amp, 2) != 0); /* missing usermode: no stale class reuse */
assert(select_classes(&rm, bw, 3) != 0); /* a Blackwell class set cannot authorize SM86 */
assert(select_classes(&rm, amp, 2, 0, 0) != 0); /* missing usermode: no stale class reuse */
assert(select_classes(&rm, bw, 3, 0, 0) != 0); /* a Blackwell class set cannot authorize SM86 */
rm.sm_version = NV2080_CTRL_GR_INFO_SM_VERSION_12_01;
assert(select_classes(&rm, bw, 3) == 0 && rm.target_sm == 121 && !rm.discrete);
assert(select_classes(&rm, bw, 3, 0, 0) == 0 && rm.target_sm == 121 && !rm.discrete);
/* Physical Spark reported raw RM SM 10.04 with PCI 10de:2e12. Its
* execution target is SM121. Do not normalize an unrelated device. */
rm.sm_version = NV2080_CTRL_GR_INFO_SM_VERSION_10_04;
assert(select_classes(&rm, bw, 3, 0x10de, 0x2e12) == 0 && rm.target_sm == 121 && !rm.discrete);
assert(select_classes(&rm, bw, 3, 0x10de, 0x2e13) != 0);
assert(select_classes(&rm, bw, 3, 0x10df, 0x2e12) != 0);
assert(select_classes(&rm, bw, 2, 0x10de, 0x2e12) != 0);
assert(select_classes(&rm, amp, 3, 0x10de, 0x2e12) != 0);
rm.sm_version = NV2080_CTRL_GR_INFO_SM_VERSION_10_03;
assert(select_classes(&rm, bw, 3, 0x10de, 0x2e12) != 0);
rm.sm_version = NV2080_CTRL_GR_INFO_SM_VERSION_8_09;
assert(select_classes(&rm, amp, 3) != 0); /* Ada is not silently treated as GA10x */
assert(select_classes(&rm, amp, 3, 0, 0) != 0); /* Ada is not silently treated as GA10x */
nv_ioctl_card_info_t cards[64] = {0};
cards[0].valid = cards[1].valid = 1;
cards[1].pci_info.bus = 1; cards[1].pci_info.slot = 2;
Expand Down
Loading