Add memory-light online streamfunction mode (lbas-indexed)#18
Open
hdrake wants to merge 3 commits into
Open
Conversation
The streamfunction flux arrays and their per-zone loops were hard-coded to a fixed 21 kill zones, and the kill-zone (10) and tracer (10) capacities were scattered as bare literals across many files (including per-project kill_zones.F90). This generalizes those limits and sizes the streamfunction allocation to the zones a run actually uses. - Add named capacity constants in mod_precdef: MAXGEOZONES, MAXTRACERS, and MAXZONES = 1 + MAXTRACERS + MAXGEOZONES, used to dimension the kill-zone and tracer configuration arrays. - Compute maxlbas (the actual streamfunction zone count) from the run: namelist geographic zones + subdomain wall zones + tracer kill zones, finalised in init_subdomain once exitType and all zones are known. The flux arrays (offline) and the compute_stream loop now size to maxlbas instead of a fixed 21. - Derive runtime loop bounds and the subdomain "last 4" wall slots from SIZE() of the arrays, removing the hard-coded slot numbers (7-10) in mod_subdomain and the 1,10 loops in kill_zones.F90 (x5), mod_diffusion, and mod_init. - Initialise ienw/iene/jens/jenn to 0 so undefined slots are well-defined. Fixes a latent out-of-bounds when the configured zone count differs from 21, and an under-allocation when an exitType=2 run uses a subdomain (which promotes exitType to 3 and adds geographic walls). Behaviour is otherwise unchanged: streamfunction output now contains the actual number of zone blocks, verified byte-identical to the previous output for the real zones. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
`./runtracmass rerun` could not re-read the trajectory labels it needs, due to two issues in the rerun file path (both independent of streamfunctions): - The output-file prefix default (outDataFile = 'TRACMASS' when empty) was applied only inside open_outfiles, which runs AFTER read_rerun. So read_rerun looked for '_rerun.csv' instead of 'TRACMASS_rerun.csv' and reported "No rerun file", leaving every trajectory label unset. Apply the default in init_namelist, before it is first used. - _rerun.csv is written comma-separated but read_rerun parsed it with a fixed "(I8,I3,I10)" format, mis-reading lbas. Use list-directed input. With both fixes a rerun pass correctly re-seeds and re-integrates the labelled trajectories (verified: exit positions identical to a normal run). Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Online streamfunction accumulation (l_psi=.TRUE., l_offline=.FALSE.) allocated the flux arrays with a per-trajectory third dimension (0:ntracmax) -- the source of out-of-memory failures for runs with millions of particles. This adds an additive online mode that indexes fluxes by killing zone (lbas) instead of by trajectory, sizing the arrays 0:maxlbas (tens) so memory no longer scales with particle count. Because lbas is only known once a trajectory terminates, the new mode runs as a second pass: pass 1 produces _rerun.csv with the labels; pass 2 (./runtracmass rerun) loads them via read_rerun and accumulates fluxes live, binned by lbas. - New namelist flag l_psi_rerun (INIT_POSTPROCESS), default .FALSE. - init_stream sizes the flux arrays 0:maxlbas in the new mode; legacy per-trajectory online still uses 0:ntracmax. - update_fluxes indexes the flux slot by the trajectory's lbas (skipping trajectories whose zone is not yet known). - compute_stream skips the per-trajectory summation in the new mode; the new path shares the offline per-zone integration. Verified byte-identical to legacy per-trajectory online streamfunctions on single- and multi-killzone configurations. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Adds a memory-light online streamfunction mode that indexes fluxes by killing zone (
lbas) instead of by trajectory, sizing the flux arrays0:maxlbas(~tens) instead of0:ntracmax(millions). This removes the per-trajectory memory that causes out-of-memory failures when computing streamfunctions online for runs with millions of particles.How it works
lbasis only known once a trajectory terminates, so the new mode runs as a second pass:_rerun.csvwith the(ntrac, lbas)labels../runtracmass rerunwithl_psi=.TRUE.,l_offline=.FALSE.,l_psi_rerun=.TRUE.loads the labels (viaread_rerun) and accumulates fluxes live, binned bylbas.Changes
l_psi_rerun(INIT_POSTPROCESS), default.FALSE.— additive; legacy per-trajectory online accumulation is unchanged.init_streamsizes the flux arrays0:maxlbasin the new mode (legacy online still uses0:ntracmax).update_fluxesindexes the flux slot by the trajectory'slbas.compute_streamskips the per-trajectory summation in the new mode; the new path reuses the offline per-zone integration.Correctness
The new mode and legacy online both write
fluxes(:,:,lbas)and integrate per zone, so they should produce identical streamfunctions. Verified byte-identical to legacy per-trajectory online on single- and multi-killzone configurations (and differs from offline only by the pre-existing online-vs-offline one-cell shift).Depends on
maxlbas(the per-zone array size).🤖 Generated with Claude Code
Addresses #19.