diff --git a/docs/PORT-AMPERE.md b/docs/PORT-AMPERE.md index 54295c57..cef27a6e 100644 --- a/docs/PORT-AMPERE.md +++ b/docs/PORT-AMPERE.md @@ -44,10 +44,10 @@ Blackwell header. | Blackwell-only instruction semantics | BLOCKED | Blackwell FP8 conversion / QMMA forms have no same-semantics sm_86 encoding. The sm_86 encoder refuses them rather than inventing bytes or silently executing a different operation. Existing Blackwell-specialized model/tensor paths are not made Ampere-capable by changing an ELF flag. A real Ampere algorithm lowering is required before those paths are enabled. | | Cubin/ELF target | DONE | `om/cubin/cubin_elf.om:26` selects the parameter base; `:269` selects CUDA ELF OSABI/ABI and e_version; `:286` sets the sm_86 flags. `omega_cubin_pack_target` is the target-explicit host boundary in `src/omega_cubin.c`. The old sm_121 entry remains explicit/default-compatible. `mk/ampere-cubin.mk` checks ELF fields and parameter metadata, differential behavior, and offline nvdisasm loading. | | Capability-mercury sections | DONE | The sm_86 writer emits no `.nv.capmerc.*` / `.nv.merc.*` sections. The Blackwell driver-loader requirement observed for sm_121 must not be generalized to Ampere. Successful offline ELF decoding is not evidence of successful `cuModuleLoadData` or direct RM launch; both RTX loader/device paths remain NOT_RUN. | -| Compute/FIFO classes and QMD | DONE | `src/omega_ampere_qmd.h:5` records C6C0 versus C7C0 and Ampere FIFO constants. `om/encoder/amp_runtime.om` implements target admission, QMD3.0 layout and bounded command admission. `src/omega_gpu_session.c:562` emits the Ampere QMD and `:599` its PCAS launch instead of Blackwell QMD5/inline packets. `nvrm/nvrm.c:69` requires the queried class list to contain the selected target's classes. Raw Blackwell packets on an Ampere device are refused before ring submission (`nvrm/nvrm.c:1055`). | -| Discrete allocation and copies | DONE | `nvrm/nvrm.c:631` selects local framebuffer memory on sm_86, RM maps it into GPU VA and maps its CPU BAR alias through real RM_MAP_MEMORY/mmap. Queue and host-polled completion storage remain system memory. `nvrm/nvrm.c:746` / `:755` provide live-identity/bounds-checked CPU→BAR and BAR→CPU transfers with ordering fences. The session may write/read that uncached BAR view directly; platform sync (`src/omega_gpu_platform_linux.c:358`) orders visibility rather than pretending to copy coherent host RAM. No copy engine is claimed. | +| Compute/FIFO classes and QMD | DONE | `src/omega_ampere_qmd.h:5` records C6C0 versus C7C0 and Ampere FIFO constants. `om/encoder/amp_runtime.om` implements target admission, QMD3.0 layout and bounded command admission. `src/omega_gpu_session.c:562` emits the Ampere QMD and `:599` its PCAS launch instead of Blackwell QMD5/inline packets. `nvrm/nvrm.c:71` requires the queried class list to contain the selected target's classes. sm_121 is admitted for the GR SM-version field 12_01, or for 10_04 (0xa04, what the GB10 reports at driver 580.173.02; `ctrl2080gr.h` keeps 10_04 until its 12_* names replace it) only when the PCI identity is 0x10de:0x2e12, and always with the complete Blackwell-B class set; `tests/test_ampere_runtime.c` refuses 10_04 with another vendor or device id. Raw Blackwell packets on an Ampere device are refused before ring submission (`nvrm/nvrm.c:1059`). | +| Discrete allocation and copies | DONE | `nvrm/nvrm.c:634` selects local framebuffer memory on sm_86, RM maps it into GPU VA and maps its CPU BAR alias through real RM_MAP_MEMORY/mmap. Queue and host-polled completion storage remain system memory. `nvrm/nvrm.c:739` / `:748` provide live-identity/bounds-checked CPU→BAR and BAR→CPU transfers with ordering fences. The session may write/read that uncached BAR view directly; platform sync (`src/omega_gpu_platform_linux.c:358`) orders visibility rather than pretending to copy coherent host RAM. No copy engine is claimed. | | Cached local mappings / large BAR availability | BLOCKED | Cached Ampere mappings are explicitly refused until their cache-maintenance contract is implemented. CPU BAR mapping availability and large-allocation behavior need the real driver/card configuration; failure is surfaced, not replaced by a fake host allocation. First-run tests use bounded allocations. | -| VRAM-aware D1 reserve | DONE | `nvrm/nvrm.c:637` queries the selected framebuffer heap. `src/omega_gpu_platform_linux.c:233` reads reserve policy and `:248` uses free VRAM, never host MemAvailable, for local allocations. `CUDAPP_GPU_MIN_FREE_BYTES` supports a precise safe refusal; the default MiB reserve remains documented by the runtime. The refusal precedes the driver allocation call. | +| VRAM-aware D1 reserve | DONE | `nvrm/nvrm.c:640` queries the selected framebuffer heap. `src/omega_gpu_platform_linux.c:233` reads reserve policy and `:248` uses free VRAM, never host MemAvailable, for local allocations. `CUDAPP_GPU_MIN_FREE_BYTES` supports a precise safe refusal; the default MiB reserve remains documented by the runtime. The refusal precedes the driver allocation call. | | Safe reserve refusal proof | DONE | `tests/ampere/memory_refusal.c:39` sets reserve above total VRAM (therefore above free VRAM even if other processes release memory), then requests only a small allocation. It requires refusal, zero intervening RM allocation ioctls and unchanged free VRAM. Simulated-driver coverage is separate from this real-device test. It never pressure-allocates VRAM. | | Open-module and device selection doctor | DONE | `tests/ampere/doctor.py:79` verifies Linux x86_64, the exact pinned open module, requested PCI IDs, device nodes and an unambiguous BDF. `CUDAPP_GPU_BDF` selects that exact GPU in the driver client. `test_doctor.py` uses a fake filesystem and is always SIMULATED. No driver is opened by the doctor. | | Portable Omega build provenance | IMPLEMENTED | `om/units.sh` covers all canonical hosted roots plus Ampere encoding/runtime and the separate revision/epoch/f16 freestanding profile. Checked portable generated translation units replace executable-code trampolines. `om/omega-bridge.lock`, per-unit identities and generated declarations retain exact composed-source SHA → C SHA provenance. Missing/mismatched converter, profile or seed refuses before installation. Final host acceptance is recorded separately below. | diff --git a/nvrm/nvrm.c b/nvrm/nvrm.c index 663e5b70..127435b6 100644 --- a/nvrm/nvrm.c +++ b/nvrm/nvrm.c @@ -56,13 +56,15 @@ static int fail(Nvrm *rm, const char *fmt, ...); /* Pure class admission lives in seeded Omega. Class IDs are vendor constants: * open-gpu-doc clc7c0.h, clc56f.h, and vendored clc361.h / clc661.h. */ -static int select_classes(Nvrm *rm, const uint32_t *classes, uint32_t count) { +static int select_classes(Nvrm *rm, const uint32_t *classes, uint32_t count, + uint16_t vendor_id, uint16_t device_id) { rm->compute_class = rm->gpfifo_class = rm->usermode_class = 0; rm->target_sm = 0; rm->discrete = 0; uint64_t m[16] = {1, rm->sm_version, NV2080_CTRL_GR_INFO_SM_VERSION_8_06, NV2080_CTRL_GR_INFO_SM_VERSION_12_01, OMEGA_AMPERE_COMPUTE_B, OMEGA_AMPERE_GPFIFO_A, VOLTA_USERMODE_A, - BLACKWELL_COMPUTE_B, BLACKWELL_CHANNEL_GPFIFO_A, HOPPER_USERMODE_A}; + BLACKWELL_COMPUTE_B, BLACKWELL_CHANNEL_GPFIFO_A, HOPPER_USERMODE_A, + 0, 0, NV2080_CTRL_GR_INFO_SM_VERSION_10_04, vendor_id, device_id}; /* Marshal the vendor's class-list presence, without selecting a target. */ for (uint32_t i = 0; i < count; i++) { for (uint32_t k = 0; k < 3; k++) { @@ -427,7 +429,8 @@ int nvrm_open(Nvrm *rm) { if (cl.numClasses > 512) return fail(rm, "too many classes"); cl.classList = NV_PTR_TO_NvP64(classes); if (rm_control(rm, rm->device, NV0080_CTRL_CMD_GPU_GET_CLASSLIST, &cl, sizeof cl)) return -1; - if (cl.numClasses > 512 || select_classes(rm, classes, cl.numClasses)) return -1; + if (cl.numClasses > 512 || select_classes(rm, classes, cl.numClasses, + cards[found].pci_info.vendor_id, cards[found].pci_info.device_id)) return -1; if (rm_alloc(rm, rm->subdevice, rm->usermode_class, NULL, 0, &rm->usermode)) return -1; void *mmio = map_to_cpu(rm, rm->usermode, 0x10000, NULL, 0, 0); if (mmio == MAP_FAILED) return -1; diff --git a/om/encoder/amp_runtime.om b/om/encoder/amp_runtime.om index 35ae9514..afc19fca 100644 --- a/om/encoder/amp_runtime.om +++ b/om/encoder/amp_runtime.om @@ -3,6 +3,7 @@ note Pure Ampere runtime decisions and QMD serialization. Host ioctls stay in nv note Vendor fields: NVIDIA open-gpu-doc classes/compute/clc7c0qmd.h QMDV03_00. note m[0] operation: 1 class-set admission, 2 byte reserve admission, 3 QMD. note Class selection m[1]=rawSM,m[2..3]=vendorSM86/121,m[4..9]=Ampere then Blackwell classes,m[10..11]=presence masks. +note m[12]=vendorSM10.04,m[13..14]=PCI vendor/device. GB10 identity follows the existing qualified chip row. note Reserve m[1]=available bytes, m[2]=request bytes, m[3]=reserve bytes. note QMD m[1..9]=code,cbank,semaphore,threadsX,threadsY,gridX,gridY,registers,sharedBytes. note q is 64 zero-extended uint words, not a native struct. Return 1 accepted or 0 refused. @@ -30,7 +31,11 @@ put q 3 get m 5 put q 4 get m 6 ret 1 end -if eq get m 1 get m 3 +gb10 = false +if and ne get m 12 0 eq get m 1 get m 12 +gb10 = and eq get m 13 4318 eq get m 14 11794 +end +if or eq get m 1 get m 3 gb10 if ne get m 11 7 ret 0 end diff --git a/src/cudapp_om_amp_runtime.c b/src/cudapp_om_amp_runtime.c index ed7ba223..c7c7e7af 100644 --- a/src/cudapp_om_amp_runtime.c +++ b/src/cudapp_om_amp_runtime.c @@ -79,7 +79,11 @@ return o_leave(0ULL); {uint64_t _i=4ULL;uint64_t _v=o_get(v0, v0_n, 6ULL, 5);o_put(v1, v1_n, _i,_v,5);} return o_leave(1ULL); } -if(({uint64_t _a=o_get(v0, v0_n, 1ULL, 5);uint64_t _b=o_get(v0, v0_n, 3ULL, 5);o_op(6, 5, _a,_b);})){ +uint64_t __attribute__((unused)) v2=0ULL; +if(({uint64_t _a=({uint64_t _a=o_get(v0, v0_n, 12ULL, 5);uint64_t _b=0ULL;o_op(7, 5, _a,_b);});uint64_t _b=({uint64_t _a=o_get(v0, v0_n, 1ULL, 5);uint64_t _b=o_get(v0, v0_n, 12ULL, 5);o_op(6, 5, _a,_b);});o_op(12, 1, _a,_b);})){ +v2=({uint64_t _a=({uint64_t _a=o_get(v0, v0_n, 13ULL, 5);uint64_t _b=4318ULL;o_op(6, 5, _a,_b);});uint64_t _b=({uint64_t _a=o_get(v0, v0_n, 14ULL, 5);uint64_t _b=11794ULL;o_op(6, 5, _a,_b);});o_op(12, 1, _a,_b);}); +} +if(({uint64_t _a=({uint64_t _a=o_get(v0, v0_n, 1ULL, 5);uint64_t _b=o_get(v0, v0_n, 3ULL, 5);o_op(6, 5, _a,_b);});uint64_t _b=v2;o_op(13, 1, _a,_b);})){ if(({uint64_t _a=o_get(v0, v0_n, 11ULL, 5);uint64_t _b=7ULL;o_op(7, 5, _a,_b);})){ return o_leave(0ULL); } diff --git a/src/cudapp_om_amp_runtime.ident b/src/cudapp_om_amp_runtime.ident index fbae1608..07072cb1 100644 --- a/src/cudapp_om_amp_runtime.ident +++ b/src/cudapp_om_amp_runtime.ident @@ -1 +1 @@ -omega-ident 2 unit=amp_runtime abi=bridge-seeded src=841f6da18d2ea43d42f4d7434b4ca84e5168b5d96d3527628f79e2f77cbb265c c=6f3cf40c3f759ac380e6e2e46fbeed75cb2206c26bcf32db33ade6dc0dae7476 omc=2c82480596bbdc50cda9626866209ef662d5834f278f839ff4ce0250c495029f converter=3a416aea14770bd3fab4880337e89020c10476214d7a3c139d8cd1a8ba1423e6 frontend=23ff7a2356bb5d980060a4685a27ac46587013069b7d24b37c2873d7ee659b37 path=om/encoder/amp_runtime.om +omega-ident 2 unit=amp_runtime abi=bridge-seeded src=452f52245a62cc7f50b4a14cf8475aa3ca829cef52a71342632190d926226cc5 c=54302a88d9a9a9efbdf0c401fde6fc88d6518a0105c83d86916e7b4a73b1bbda omc=2c82480596bbdc50cda9626866209ef662d5834f278f839ff4ce0250c495029f converter=3a416aea14770bd3fab4880337e89020c10476214d7a3c139d8cd1a8ba1423e6 frontend=23ff7a2356bb5d980060a4685a27ac46587013069b7d24b37c2873d7ee659b37 path=om/encoder/amp_runtime.om diff --git a/src/cudapp_om_units.ident b/src/cudapp_om_units.ident index fba74303..41da0b4e 100644 --- a/src/cudapp_om_units.ident +++ b/src/cudapp_om_units.ident @@ -1,7 +1,7 @@ omega-ident 2 unit=bw_enc_sm121 abi=bridge-seeded src=901e9dc7921663fa5e91d2df2267910c85d7933acc9421e55828f55fbba6c946 c=f0270d0a3742b310301233954fdd5d28f216e8c4ade64eb7770b5db9b99c1c5a omc=2c82480596bbdc50cda9626866209ef662d5834f278f839ff4ce0250c495029f converter=3a416aea14770bd3fab4880337e89020c10476214d7a3c139d8cd1a8ba1423e6 frontend=23ff7a2356bb5d980060a4685a27ac46587013069b7d24b37c2873d7ee659b37 path=om/encoder/bw_enc_sm121.om omega-ident 2 unit=cubin_elf abi=bridge-seeded src=7c9bacf500bc116b110bbf884041005f1fb957ca3f34495e3e01a1d94bc1683e c=da7a5521b9f7953eb165cd26d4d8a38a0ffa49e5ca25009d0a4306aa6185f42a omc=2c82480596bbdc50cda9626866209ef662d5834f278f839ff4ce0250c495029f converter=3a416aea14770bd3fab4880337e89020c10476214d7a3c139d8cd1a8ba1423e6 frontend=23ff7a2356bb5d980060a4685a27ac46587013069b7d24b37c2873d7ee659b37 path=om/cubin/cubin_elf.om omega-ident 2 unit=ampere_enc_sm86 abi=bridge-seeded src=fe3b212baa4afe9eb18b2a8c11a174fcf52a9a247a66d859bf32cabe0cd1ea64 c=a9db496848193bdcad72f9e4676a398466bc717233da8f22db50b3ab54e92166 omc=2c82480596bbdc50cda9626866209ef662d5834f278f839ff4ce0250c495029f converter=3a416aea14770bd3fab4880337e89020c10476214d7a3c139d8cd1a8ba1423e6 frontend=23ff7a2356bb5d980060a4685a27ac46587013069b7d24b37c2873d7ee659b37 path=om/encoder/ampere_enc_sm86.om -omega-ident 2 unit=amp_runtime abi=bridge-seeded src=841f6da18d2ea43d42f4d7434b4ca84e5168b5d96d3527628f79e2f77cbb265c c=6f3cf40c3f759ac380e6e2e46fbeed75cb2206c26bcf32db33ade6dc0dae7476 omc=2c82480596bbdc50cda9626866209ef662d5834f278f839ff4ce0250c495029f converter=3a416aea14770bd3fab4880337e89020c10476214d7a3c139d8cd1a8ba1423e6 frontend=23ff7a2356bb5d980060a4685a27ac46587013069b7d24b37c2873d7ee659b37 path=om/encoder/amp_runtime.om +omega-ident 2 unit=amp_runtime abi=bridge-seeded src=452f52245a62cc7f50b4a14cf8475aa3ca829cef52a71342632190d926226cc5 c=54302a88d9a9a9efbdf0c401fde6fc88d6518a0105c83d86916e7b4a73b1bbda omc=2c82480596bbdc50cda9626866209ef662d5834f278f839ff4ce0250c495029f converter=3a416aea14770bd3fab4880337e89020c10476214d7a3c139d8cd1a8ba1423e6 frontend=23ff7a2356bb5d980060a4685a27ac46587013069b7d24b37c2873d7ee659b37 path=om/encoder/amp_runtime.om omega-ident 2 unit=repack abi=bridge-seeded src=ab03c810d7517ef3c2db9083f5073851cfda77f0a7f7bd83269dcc43e6be64ae c=c4037b20a46345b86c404be57defe318da8d6a9695349eb5ff3f1bc517dd3d5f omc=2c82480596bbdc50cda9626866209ef662d5834f278f839ff4ce0250c495029f converter=3a416aea14770bd3fab4880337e89020c10476214d7a3c139d8cd1a8ba1423e6 frontend=23ff7a2356bb5d980060a4685a27ac46587013069b7d24b37c2873d7ee659b37 path=repack=om/quant/quant.om,om/quant/repack.om omega-ident 2 unit=quant_wire abi=bridge-seeded src=657e765de09ad4962165b11a18d7e8fb19b650aa2d7bb321ac11d27b8a63a3c5 c=97ca9ef6470cd8d1bbb33781c4e09c64d7a977339553f7f8d404be612a98b1c0 omc=2c82480596bbdc50cda9626866209ef662d5834f278f839ff4ce0250c495029f converter=3a416aea14770bd3fab4880337e89020c10476214d7a3c139d8cd1a8ba1423e6 frontend=23ff7a2356bb5d980060a4685a27ac46587013069b7d24b37c2873d7ee659b37 path=quant_wire=om/quant/quant.om,om/quant/runtime.om omega-ident 2 unit=compose abi=bridge-seeded src=91cfa4c561ee04a8db7cb70e43829eeb19df2e4aa1030573a762242c788195e0 c=3ff9e84cea668f9ad53693e10fe6af67654d8c9918ba27f6c05be22c059c4dbe omc=2c82480596bbdc50cda9626866209ef662d5834f278f839ff4ce0250c495029f converter=3a416aea14770bd3fab4880337e89020c10476214d7a3c139d8cd1a8ba1423e6 frontend=23ff7a2356bb5d980060a4685a27ac46587013069b7d24b37c2873d7ee659b37 path=compose=om/components/bw_alloc.om,om/components/bw_effect.om,om/components/bw_check.om,om/components/bw_project.om,om/components/rowpipe.om,om/components/staged.om,om/components/compose.om,om/components/runtime.om diff --git a/tests/test_ampere_runtime.c b/tests/test_ampere_runtime.c index bc400597..eff8d99f 100644 --- a/tests/test_ampere_runtime.c +++ b/tests/test_ampere_runtime.c @@ -103,14 +103,24 @@ static void class_and_bdf(void) { const uint32_t amp[] = {OMEGA_AMPERE_COMPUTE_B, OMEGA_AMPERE_GPFIFO_A, VOLTA_USERMODE_A}; const uint32_t bw[] = {BLACKWELL_COMPUTE_B, BLACKWELL_CHANNEL_GPFIFO_A, HOPPER_USERMODE_A}; rm.sm_version = NV2080_CTRL_GR_INFO_SM_VERSION_8_06; - assert(select_classes(&rm, amp, 3) == 0 && rm.target_sm == 86 && rm.discrete); + assert(select_classes(&rm, amp, 3, 0, 0) == 0 && rm.target_sm == 86 && rm.discrete); assert(rm.compute_class == OMEGA_AMPERE_COMPUTE_B && rm.gpfifo_class == OMEGA_AMPERE_GPFIFO_A); - assert(select_classes(&rm, amp, 2) != 0); /* missing usermode: no stale class reuse */ - assert(select_classes(&rm, bw, 3) != 0); /* a Blackwell class set cannot authorize SM86 */ + assert(select_classes(&rm, amp, 2, 0, 0) != 0); /* missing usermode: no stale class reuse */ + assert(select_classes(&rm, bw, 3, 0, 0) != 0); /* a Blackwell class set cannot authorize SM86 */ rm.sm_version = NV2080_CTRL_GR_INFO_SM_VERSION_12_01; - assert(select_classes(&rm, bw, 3) == 0 && rm.target_sm == 121 && !rm.discrete); + assert(select_classes(&rm, bw, 3, 0, 0) == 0 && rm.target_sm == 121 && !rm.discrete); + /* Physical Spark reported raw RM SM 10.04 with PCI 10de:2e12. Its + * execution target is SM121. Do not normalize an unrelated device. */ + rm.sm_version = NV2080_CTRL_GR_INFO_SM_VERSION_10_04; + assert(select_classes(&rm, bw, 3, 0x10de, 0x2e12) == 0 && rm.target_sm == 121 && !rm.discrete); + assert(select_classes(&rm, bw, 3, 0x10de, 0x2e13) != 0); + assert(select_classes(&rm, bw, 3, 0x10df, 0x2e12) != 0); + assert(select_classes(&rm, bw, 2, 0x10de, 0x2e12) != 0); + assert(select_classes(&rm, amp, 3, 0x10de, 0x2e12) != 0); + rm.sm_version = NV2080_CTRL_GR_INFO_SM_VERSION_10_03; + assert(select_classes(&rm, bw, 3, 0x10de, 0x2e12) != 0); rm.sm_version = NV2080_CTRL_GR_INFO_SM_VERSION_8_09; - assert(select_classes(&rm, amp, 3) != 0); /* Ada is not silently treated as GA10x */ + assert(select_classes(&rm, amp, 3, 0, 0) != 0); /* Ada is not silently treated as GA10x */ nv_ioctl_card_info_t cards[64] = {0}; cards[0].valid = cards[1].valid = 1; cards[1].pci_info.bus = 1; cards[1].pci_info.slot = 2;