Skip to content

kokkos: W3SIN4's stress scan with one sea point per vector lane… - #199

Merged
h0ffmann merged 5 commits into
mainfrom
goal/c3e7eecf-6142-srce-lanes-sin4
Oct 18, 2026
Merged

h0ffmann merged 5 commits into
mainfrom
goal/c3e7eecf-6142-srce-lanes-sin4

Conversation

@h0ffmann

@h0ffmann h0ffmann commented Oct 18, 2026 •

Copy link
Copy Markdown
Owner

Before. After L1 (#197), the batched W3SRCE could take up to eight sea points per team, but only when WW_KOKKOS_SRCE_LANES was set; the default of 1 was one point per team. Each point still ran W3SIN4 alone on one thread. The stress scan of W3SIN4 (w3src4md.F90:696-797) has ordered sums and per-bin branches that g++ leaves scalar inside one point. By the S0 profile, W3SIN4's own code was 12.9 % of one thread's batch time.

After.

  • The scalar routine is split, verbatim. w3sin4_point is now sin4_prep, sin4_scan and sin4_tail, with its lines moved without a change.
  • sin4_scan_lanes<8> runs the scan for eight points at once, one point per lane, on GCC vector types of eight REAL.
    • Each vector operation is the scalar statement's, lane by lane, with its operands in the same order.
    • A branch becomes a select of the value its taken side computes.
    • expf and powf are called per lane, only where the branch calls them.
    • Each lane keeps its own stress sums, added in ITH order.
  • Where it runs. w3srce_lanes calls it at both W3SIN4 sites of the integration step.
  • Eight points per team is now the default on host backends, as you asked. Small launches fall back to one point per team: below 8 lanes × threads points, every thread still gets a team (ww3_ts1, with one sea point, takes one per team). A device keeps one point per team.
  • Gain. On full, the one-thread W3SRCE kernel takes 5.1 % less time, and the 32-thread ww3_shel run 2.6 % less. Every output stays byte-identical to the Fortran.
  • The session's handover is docs/log/2026-10-18-goal-c3e7eecf-handover.md, merged here with main after docs: ADR-0019 (Proposed) and plan 006 #198.

Goal c3e7eecf-6142, Claude Code session ec4548a3-46a8-46e2-be0a-2d74c3e7eecf. Plan: docs/compound/plans/2026-10-18-005-srce-points-in-lanes-plan.md, unit L2. Record: docs/log/2026-10-18-srce-lanes-sin4.md.

Evidence

Intel Core i9-14900, switch_lab_ompg, OpenMP backend, WW3@50a10730.

Check Result
just regtest-selected at 919c5cf (the default of 8, strict, 6 threads per run) ww3_tp2.15 x4 and ww3_ts2 ST4_PR3_UQ: test.comp 0/23, 0/23, 0/23, 0/25, 0/22, nccmp-tol PASS; smoke byte-identical; the verdicts of main's run. The four ww3_tp2.15 entries ran the lane path (276 and 5 480 points per launch, at least 48); ww3_ts2 (24 points per launch) ran one point per team
smoke, full with no variable set, WW_KOKKOS_STRICT=1 WW_KOKKOS_VA_POISON=1 every model file byte-identical to the Fortran of the same executable; every file but the NetCDF to --ww3's
srce.batch.kernel, full, one thread pinned to a performance core, 3 alternated runs each 105.17 s with 1 point per team, 99.77 s with 8: 5.1 % less
ww3_shel, full, 32 threads, 3 alternated runs each 15.88 s with 1, 15.47 s with the default: 2.6 % less; every out_grd.ww3 identical
bash scripts/ww3-gpu.sh --ww3-gpu smoke --gpu --kokkos resident --build-only compiles and links, no warning; nothing run on a device
just kokkos-test serial-debug, openmp-release 11/11 each, no warning
port_score.py --write then --check; scripts/vale.sh, lychee --offline pass

⚠ Some figures are not as clean as the table suggests:

  • The one-thread timing ran before vload took a reference, a change made only to remove a -Wpsabi warning, and was not timed again.
  • The 32-thread loads (22 to 38) are those back-to-back runs themselves, inferred from their spread.
  • The regtest entries took about twice main's time. That's machine load: ww3_ts2, which did not take the lane path, slowed the same.

Needs your review

  • Loop nesting (AGENTS.md, "Porting"). The scan's lane loop is innermost, and every lane's arithmetic is the scalar statement's.
  • Explicit vector types in a ported routine. Written as plain loops, g++ left the scan scalar under its default -ftrapping-math. Changing that flag would affect every kernel, so this PR uses explicit vectors in this one loop.
  • The new default. Resident runs on host backends now take the lane path with no variable set. WW_KOKKOS_SRCE_LANES=1 restores one point per team.

🤖 Generated with Claude Code

… 005 L2)

Unit L2 of plan 005, goal c3e7eecf-6142, Claude Code session
ec4548a3-46a8-46e2-be0a-2d74c3e7eecf. w3sin4_point is split into sin4_prep, sin4_scan and
sin4_tail with its lines moved verbatim. sin4_scan_lanes<8> runs the scan (w3src4md.F90:696-797)
for up to eight points, the ITH loop on GCC vector types of eight REAL: each vector operation
is the scalar statement's, lane by lane, operands in the same order; a branch is a select of the
value its taken side computes; expf and powf are called per lane where the branch calls them;
each lane keeps its own stress sums in ITH order. w3srce_lanes calls it at both W3SIN4 sites
of the integration step. Explicit vectors because g++ leaves the plain lane loop scalar under
its default -ftrapping-math, and that flag is not changed for the whole library.

Tested: just kokkos-test serial-debug and openmp-release, 11/11 each, no warning; bash scripts/ww3-gpu.sh --ww3-gpu smoke and full --cpu --kokkos resident with WW_KOKKOS_SRCE_LANES=8 WW_KOKKOS_STRICT=1 WW_KOKKOS_VA_POISON=1 on the final code: every model file byte-identical to the Fortran of the same executable, every file but the NetCDF to --ww3; bash scripts/ww3-gpu.sh --ww3-gpu smoke --gpu --kokkos resident --build-only, compiles, no warning, nothing run on a device; srce.batch.kernel one thread pinned (taskset -c 2), 3 alternated runs each: 105.17 s with 1 point per team, 99.77 s with 8 (the log entry's script); the three moved ranges found verbatim; python3 scripts/port_score.py --write then --check; scripts/vale.sh and lychee --offline on the changed Markdown, 0 errors
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@h0ffmann h0ffmann changed the title kokkos: W3SIN4's stress scan with one sea point per vector lane… kokkos: W3SIN4's stress scan with one sea point per vector lane (plan 005 L2) Oct 18, 2026
h0ffmann and others added 2 commits October 18, 2026 18:56
Brings in #198 (ADR-0019, plan 006, the abbb0456 handover). One conflict, in the dated table of
docs/log/README.md: both sides appended a 2026-10-18 row; both are kept, main's first.

Tested: no conflict marker left in docs/log/README.md (grep)
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
What this session finished (#197: the NOAA record and its ten-run median, plan 005, S0, L1;
the merge that unblocked #198), what is open (#199's regtest run and review, plan 005 L3 to
L7, the uncommitted opportunity and GFS-Wave analysis), what waits on the owner, and the traps
met: unpinned one-thread timings on a hybrid CPU, the load average counting the job itself,
g++'s -ftrapping-math and lane loops, -Wpsabi without -march, pkill -f killing its own shell.

Tested: scripts/vale.sh on the entry and docs/log/README.md, exit 0, 0 errors; lychee --offline on both, 0 errors
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@github-actions github-actions Bot changed the title kokkos: W3SIN4's stress scan with one sea point per vector lane (plan 005 L2) kokkos: W3SIN4's stress scan with one sea point per vector lane… Oct 18, 2026
h0ffmann and others added 2 commits October 18, 2026 19:05
The owner asked for 8 as the default of WW_KOKKOS_SRCE_LANES, so the lane W3SIN4 runs without
a variable. A launch with fewer points than 8 x threads takes one point per team, so a small
grid keeps every thread busy (ww3_ts1, one sea point, does); like the schedule, this decides
only which points share a team. A device keeps one point per team. At 32 threads on full the
default takes 2.6 % less ww3_shel wall time than one point per team.

Tested: just kokkos-test serial-debug and openmp-release, 11/11 each, no warning; bash scripts/ww3-gpu.sh --ww3-gpu smoke and full --cpu --kokkos resident with no WW_KOKKOS_SRCE_LANES, WW_KOKKOS_STRICT=1 WW_KOKKOS_VA_POISON=1: every model file byte-identical to the Fortran of the same executable, every file but the NetCDF to --ww3; full, 32 threads, 3 alternated runs each: 15.88 s with 1 point per team, 15.47 s with the default, every out_grd.ww3 identical; python3 scripts/port_score.py --write then --check; scripts/vale.sh and lychee --offline on the changed Markdown, 0 errors
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…ault

just regtest-selected at 919c5cf: ww3_tp2.15 x4 and ww3_ts2 ST4_PR3_UQ give test.comp 0 files
different and nccmp-tol PASS, smoke byte-identical, the verdicts of main's run. The four
ww3_tp2.15 entries ran the lane path (276 and 5 480 points per launch against the 48 of 8 lanes
x 6 threads); ww3_ts2, 24 points per launch, ran one point per team.

Tested: just regtest-selected (5 entries PASS, 8/8 ctest, smoke byte-identical); scripts/vale.sh on the entry, exit 0
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@h0ffmann
h0ffmann merged commit b828041 into main Oct 18, 2026
10 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant