Fix NCU CSV NaN parser crash on empty unit cells#134
Merged
Conversation
The unit-row detection in ``load_ncu_metrics`` reads the first CSV row,
lowercases each cell, and checks whether any cell contains a
unit-marker substring. ``Series.str.lower()`` propagates ``pd.NA`` /
``NaN`` cells through as float NaN, and the subsequent
``any(tok in x for tok in unit_tokens)`` then raises::
TypeError: argument of type 'float' is not iterable
This surfaces on real NCU outputs (e.g. profiles from the
``gemma3_swiglu`` benchmark in SOL-ExecBench) and aborts the entire
profiling step, which then propagates up to the worker as
``Profiling failed`` and the round produces 0 successful kernels.
Fix: chain ``.fillna("")`` after ``.str.lower()`` so NaN cells become
empty strings before the substring check. The matching logic itself is
unchanged — empty strings legitimately don't contain any of the unit
tokens.
Test plan:
- ``pytest tests/`` (existing suite passes)
- Reproduces no longer: running the optimizer against
``ka-review-gate-runs/gemma3_swiglu`` previously hit the TypeError on
every worker's first NCU profile; with the patch the round produces
successful kernels.
meta-cla
Bot
added
the
CLA Signed
Jack-Khuu
approved these changes
May 14, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
4 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
The NCU cvs report contains a header row including
"ID","Process ID","Process Name",…,"Kernel Name",…,"Block Size","Grid Size",…,"sm__cycles_active.avg",…and a unit row
"","","","","","","","","","",…,"cycle","%","block","block",…The units row is the one we're parsing. It contains a unit string for every metric column (cycle, %, block, register, …) but empty strings for the identifier columns (ID, Kernel Name, Block Size, Grid Size) — because IDs and names don't have units.
When pandas parses those "" cells, it loads them as numpy.float64(nan). Later
tok in nanraisesTypeError: argument of type 'float' is not iterable.This PR fix the typeerror by making the detection NaN-safe.