[tolgacangoz]

@S1ro1 @msaroufim @owenhilyard

[owenhilyard]

I'm relatively unfamiliar with GPU Mode's submission process, but is there some way to allow using another file for Mojo source code? Losing LSP support is already not great, but LLMs already have some issues confusing Mojo and Python, and I expect that to be much worse if the Mojo source is embedded in a python string.

Additionally, how is timing handled here? Mojo JIT compiles relevant standard library code and built-in kernels as part of building a python module from Mojo source code. Python modules can also be build ahead of time via a more normal compilation flow, which is what I would expect to do when benching a kernel.

[tolgacangoz]

Thanks a bunch for your reviews @owenhilyard! AFAIU, a competitor usually make their experiments with usual Cuda and Mojo files, then submit within a Python string like that -that .py file is just for the submission part.

[tolgacangoz]

AFAIU, compilation happens at import, before the timer. Runner.py_init() compiles the function while the module loads. The benchmark only times custom_kernel(), which enqueues that compiled kernel.
GPU MODE/reference-kernels/problems/nvidia/eval.py:

        torch.cuda.synchronize()
        start_event = torch.cuda.Event(enable_timing=True)
        end_event = torch.cuda.Event(enable_timing=True)
        clear_l2_cache()

        start_event.record()
        output = custom_kernel(data)
        end_event.record()
        torch.cuda.synchronize()
        duration = start_event.elapsed_time(end_event) * 1e6  # Convert ms to ns

Am I understanding it correctly?

[owenhilyard]

That fragment means Mojo kernels would need to convert from pytorch tensors via dlpack during the timed portion, which isn't exactly cheap. I someone willing to weigh in on whether there can be a "convert to native tensor format" hook that isn't timed? No production kernel would ever end up taking that interop hit, so I don't think it's worth timing.

[tolgacangoz]

Is there currently the usage of DLPack here since we are using .data_ptr()s?