Repository navigation
kokkos: W3SIN4's stress scan with one sea point per vector lane… - #199
Merged
Merged
Conversation
… 005 L2) Unit L2 of plan 005, goal c3e7eecf-6142, Claude Code session ec4548a3-46a8-46e2-be0a-2d74c3e7eecf. w3sin4_point is split into sin4_prep, sin4_scan and sin4_tail with its lines moved verbatim. sin4_scan_lanes<8> runs the scan (w3src4md.F90:696-797) for up to eight points, the ITH loop on GCC vector types of eight REAL: each vector operation is the scalar statement's, lane by lane, operands in the same order; a branch is a select of the value its taken side computes; expf and powf are called per lane where the branch calls them; each lane keeps its own stress sums in ITH order. w3srce_lanes calls it at both W3SIN4 sites of the integration step. Explicit vectors because g++ leaves the plain lane loop scalar under its default -ftrapping-math, and that flag is not changed for the whole library. Tested: just kokkos-test serial-debug and openmp-release, 11/11 each, no warning; bash scripts/ww3-gpu.sh --ww3-gpu smoke and full --cpu --kokkos resident with WW_KOKKOS_SRCE_LANES=8 WW_KOKKOS_STRICT=1 WW_KOKKOS_VA_POISON=1 on the final code: every model file byte-identical to the Fortran of the same executable, every file but the NetCDF to --ww3; bash scripts/ww3-gpu.sh --ww3-gpu smoke --gpu --kokkos resident --build-only, compiles, no warning, nothing run on a device; srce.batch.kernel one thread pinned (taskset -c 2), 3 alternated runs each: 105.17 s with 1 point per team, 99.77 s with 8 (the log entry's script); the three moved ranges found verbatim; python3 scripts/port_score.py --write then --check; scripts/vale.sh and lychee --offline on the changed Markdown, 0 errors Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Brings in #198 (ADR-0019, plan 006, the abbb0456 handover). One conflict, in the dated table of docs/log/README.md: both sides appended a 2026-10-18 row; both are kept, main's first. Tested: no conflict marker left in docs/log/README.md (grep) Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
What this session finished (#197: the NOAA record and its ten-run median, plan 005, S0, L1; the merge that unblocked #198), what is open (#199's regtest run and review, plan 005 L3 to L7, the uncommitted opportunity and GFS-Wave analysis), what waits on the owner, and the traps met: unpinned one-thread timings on a hybrid CPU, the load average counting the job itself, g++'s -ftrapping-math and lane loops, -Wpsabi without -march, pkill -f killing its own shell. Tested: scripts/vale.sh on the entry and docs/log/README.md, exit 0, 0 errors; lychee --offline on both, 0 errors Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The owner asked for 8 as the default of WW_KOKKOS_SRCE_LANES, so the lane W3SIN4 runs without a variable. A launch with fewer points than 8 x threads takes one point per team, so a small grid keeps every thread busy (ww3_ts1, one sea point, does); like the schedule, this decides only which points share a team. A device keeps one point per team. At 32 threads on full the default takes 2.6 % less ww3_shel wall time than one point per team. Tested: just kokkos-test serial-debug and openmp-release, 11/11 each, no warning; bash scripts/ww3-gpu.sh --ww3-gpu smoke and full --cpu --kokkos resident with no WW_KOKKOS_SRCE_LANES, WW_KOKKOS_STRICT=1 WW_KOKKOS_VA_POISON=1: every model file byte-identical to the Fortran of the same executable, every file but the NetCDF to --ww3; full, 32 threads, 3 alternated runs each: 15.88 s with 1 point per team, 15.47 s with the default, every out_grd.ww3 identical; python3 scripts/port_score.py --write then --check; scripts/vale.sh and lychee --offline on the changed Markdown, 0 errors Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…ault just regtest-selected at 919c5cf: ww3_tp2.15 x4 and ww3_ts2 ST4_PR3_UQ give test.comp 0 files different and nccmp-tol PASS, smoke byte-identical, the verdicts of main's run. The four ww3_tp2.15 entries ran the lane path (276 and 5 480 points per launch against the 48 of 8 lanes x 6 threads); ww3_ts2, 24 points per launch, ran one point per team. Tested: just regtest-selected (5 entries PASS, 8/8 ctest, smoke byte-identical); scripts/vale.sh on the entry, exit 0 Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Before. After L1 (#197), the batched
W3SRCEcould take up to eight sea points per team, but only whenWW_KOKKOS_SRCE_LANESwas set; the default of 1 was one point per team. Each point still ranW3SIN4alone on one thread. The stress scan ofW3SIN4(w3src4md.F90:696-797) has ordered sums and per-bin branches that g++ leaves scalar inside one point. By the S0 profile,W3SIN4's own code was 12.9 % of one thread's batch time.After.
w3sin4_pointis nowsin4_prep,sin4_scanandsin4_tail, with its lines moved without a change.sin4_scan_lanes<8>runs the scan for eight points at once, one point per lane, on GCC vector types of eightREAL.expfandpowfare called per lane, only where the branch calls them.ITHorder.w3srce_lanescalls it at bothW3SIN4sites of the integration step.ww3_ts1, with one sea point, takes one per team). A device keeps one point per team.full, the one-threadW3SRCEkernel takes 5.1 % less time, and the 32-threadww3_shelrun 2.6 % less. Every output stays byte-identical to the Fortran.docs/log/2026-10-18-goal-c3e7eecf-handover.md, merged here withmainafter docs: ADR-0019 (Proposed) and plan 006 #198.Goal
c3e7eecf-6142, Claude Code sessionec4548a3-46a8-46e2-be0a-2d74c3e7eecf. Plan:docs/compound/plans/2026-10-18-005-srce-points-in-lanes-plan.md, unit L2. Record:docs/log/2026-10-18-srce-lanes-sin4.md.Evidence
Intel Core i9-14900,
switch_lab_ompg, OpenMP backend,WW3@50a10730.just regtest-selectedat919c5cf(the default of 8, strict, 6 threads per run)ww3_tp2.15x4 andww3_ts2ST4_PR3_UQ:test.comp0/23, 0/23, 0/23, 0/25, 0/22,nccmp-tolPASS;smokebyte-identical; the verdicts ofmain's run. The fourww3_tp2.15entries ran the lane path (276 and 5 480 points per launch, at least 48);ww3_ts2(24 points per launch) ran one point per teamsmoke,fullwith no variable set,WW_KOKKOS_STRICT=1 WW_KOKKOS_VA_POISON=1--ww3'ssrce.batch.kernel,full, one thread pinned to a performance core, 3 alternated runs eachww3_shel,full, 32 threads, 3 alternated runs eachout_grd.ww3identicalbash scripts/ww3-gpu.sh --ww3-gpu smoke --gpu --kokkos resident --build-onlyjust kokkos-test serial-debug,openmp-releaseport_score.py --writethen--check;scripts/vale.sh,lychee --offline⚠ Some figures are not as clean as the table suggests:
vloadtook a reference, a change made only to remove a-Wpsabiwarning, and was not timed again.main's time. That's machine load:ww3_ts2, which did not take the lane path, slowed the same.Needs your review
-ftrapping-math. Changing that flag would affect every kernel, so this PR uses explicit vectors in this one loop.WW_KOKKOS_SRCE_LANES=1restores one point per team.🤖 Generated with Claude Code