SCETlib NP continuous parameter model#701
Open
lucalavezzo wants to merge 31 commits into
Open
Conversation
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
_DISCRETE_NP_SUBSTRING lives at module scope; reading it via self raised AttributeError on any datacard with systs. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
btgrid_numpy was only consumed at runtime through load_btgrid_shards; move that into btgrid_cache and keep the numpy reference + parity tests in the development tree only. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
kdlong
reviewed
Jun 16, 2026
kdlong
reviewed
Jun 16, 2026
kdlong
reviewed
Jun 16, 2026
kdlong
reviewed
Jun 16, 2026
kdlong
reviewed
Jun 16, 2026
…d to histmaker meta, de-verbose docstring - Move Z mass/width to common.MZ_S_DEP_WIDTH / GAMMAZ_S_DEP_WIDTH (s-dependent-width scheme 91.1535/2.4932, not the PDG pole values); btgrid_integrate uses them for the arctan-Q^2 change of variable. The choice only sets the transform centre/scale: <0.001% effect on the integral on the real btgrid Q grid. - Propagate the SCETlib NP runcard from the theoryCorr pkl into the histmaker output meta_info (scetlib_np_lambda_central); the fit reads lambda_central from metadata instead of re-opening the pkl by filename. Non-backward- compatible: fit inputs produced before this must be remade. - De-verbose the param_model module docstring (drop "single source of truth" framing, convert Unicode box dividers to ASCII). (The "drop the numpy reference implementation" item was already on this branch.) Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
# Conflicts: # wremnants/production/datasets/dataset_tools.py
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Carry the reco x gen response matrix R (and the gen-total N_gen) through the setupRabbit datacard instead of a separate file, so the fit is self-contained and R is always consistent with the run that built the card (review comment WMass#6). - setupRabbit: after the input loop, extract R + N_gen once from the unfolding histmaker output and embed them in the fit input via rabbit add_auxiliary (the "scetlib_np" group). Presence-based lenient guard (response_matrix.has_response): embed only when an input carries both the response hist and the prefsr gen-total; raise if more than one qualifies. - response_matrix: add has_response() (cheap both-present check, never raises). - param_model: read R only from indata.auxiliary["scetlib_np"]; drop the unfolding_hdf5_path argument and the sigma_gen(lambda_c) N_gen fallback (raise if N_gen is absent -- the proxy made the central closure circular). - scetlib_np package __init__: import SCETlibNPParamModel lazily (PEP 562) so response_matrix can be imported by setupRabbit without pulling in TensorFlow. Requires a rabbit with TensorWriter.add_auxiliary (WMass/rabbit#145); the submodule bump is a follow-up. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
The PR previously pinned an off-main rabbit feature commit (6bfa4ff). All required rabbit-side changes (ParamModel priors/blinding, auxiliary array group, composite POU [POIs|POUs] layout fix, asym impacts) are now merged into rabbit main, so track it directly. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Revert the inadvertent wremnants-data bump so the PR matches upstream main. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
- input_tools.read_dyturbo_vars_hist: remove two leftover debug print()s. - btgrid_integrate / param_model: the "numpy reference" comments named functions that live in the external scetlib_run.factorize library, not in this package; qualify them so a maintainer grepping the package isn't lost. - param_model module docstring: condense the GN-vs-full-K implementation history (kept the Hessian formula, the Asimov/GN rationale, and the _frozen_eq_zero footgun; dropped the blow-by-blow narration). Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
lucalavezzo
changed the title
gen_level=1 makes the fit channel the gen (ptVGen, absY) binning and skips Step 3 (the gen→reco fold): compute() returns the per-gen-bin ratio σ_gen(λ)/σ_gen(λ_central) from Steps 1–2. No scetlib_np auxiliary / R / N_gen is required — the gen binning is read from the single fit channel. For the direct-theory σUL closure. Recovered from an uncommitted stash where it had been lost during the PR701 sync. param_model.py: gen_level constructor flag + docstring; branch the R / gen-axes setup, the central-denominator caching, and _ratio_from_param so the normal reco path is byte-for-byte unchanged when gen_level is off. Also folds in the pending non-finite btgrid-cell sanitization (NaN/inf -> 0 for kinematically-forbidden grid points) that was sitting in the working tree. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
lucalavezzo
force-pushed
the
scetlib-np-param-model
branch
from
e8d0fda to
21a71a0
Compare
Two companion modules for visualizing the SCETlib NP form factors from a fit: - np_function_plots.py: a PURE plotter for the CS γ_ν^NP(b_T) and TMD F_eff(b_T, y) form factors. Calls the same btgrid_tf.F_eff_tf / gamma_nu_NP_tf the fit integrates (driven by the np_model strings), so a drawn curve is exactly the fitted model. Takes physical λ sets + optional toys for a percentile band; knows nothing about fitresults. Raw-λ and --from-fitresult CLI modes. - fitresult_lambdas.py: the OUTPUT-side reader that turns a rabbit fitresults HDF5 into the λ sets / toy ensembles the plotter consumes. Reads λ from EITHER fit flavour behind one NPLambdas interface: the new continuous-λ param model (physical λ in parms + covariance) and the old template-based fit (discrete-nuisance pulls mapped to physical λ). Also prints the prefit/postfit/constraint/frozen λ table (CLI). Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
lucalavezzo
force-pushed
the
scetlib-np-param-model
branch
from
21a71a0 to
ccab63c
Compare
kdlong
reviewed
Jun 25, 2026
| fn = fn.replace("-mur0p5-", "-murH-") | ||
| if "-muf0p5-" in fn.split("/")[-1]: | ||
| fn = fn.replace("-muf0p5-", "-mufH-") | ||
| # TODO the naming convention is unclear and inconsistent. These may be needed in the future. |
Collaborator
There was a problem hiding this comment.
The old names, with things like "murH" were just what Simone called the files. If we (Valentina) are making the new files with different/more logical names we can just support that.
One nested registry {model: {param: {value, sigma}}}, split by family, storing
each model's λ set AND its fit defaults inline (value = neutral start fallback,
sigma = default prior width, None = free). Replaces the hardcoded active_params
branches, the flat DEFAULT_PRIOR_SIGMAS, and btgrid_tf's EFF_MODELS/GNU_MODELS
name-sets. active_params()/param_defaults()/valid-name sets derive from it.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
The form factors take a values dict + np_model selector; each branch reads ONLY the λ its formula uses (missing -> KeyError = fail out). Adding a model with new λ is a new branch + a registry row, no signature churn. EFF_MODELS/GNU_MODELS + alias maps imported from params. Updated the 7 internal call sites and the np_function_plots / param_model_diagnostics callers (**dict -> dict). Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
param_model: _param_order = active_params(fit forms) — inert λ (e.g. lambda6/ lambda6_nu under tanh_2) are no longer registered, so they can't add a zero-derivative Hessian row (fixes the postfit 'Cholesky failed, Hessian not positive-definite'). Start precedence xparam_default > card λ_central > registry neutral; prior sigmas from the registry; impact groups intersected; poi_params validated; prior_sigmas/xparam_default naming a non-fitted λ warn-and-ignore. lambda_central: _fill_missing_params validate-only (no 0.0 fill). Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
SCETlib-NP: fit only the active λ (fixes singular postfit Hessian) + model→λ registry
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Split out of #689 (which now carries only the alpha_s rapidity-decorrelation changes).
ParamModelimplementing the SCETlib nonperturbative model with continuous λ parameters (wremnants/postprocessing/scetlib_np/): the model computes the per-reco-bin signal response on the fly from the SCETlib bT-grid (Q-integration + Simpson rebinning, TF-differentiable), replacing the discrete scetlibNP κ-template variations.--storeResponseMatrixtosetupRabbit.pyto store the gen x reco response matrix in the datacarrot ingested by rabbit, needed for the parameter model.wremnants/utilities/data_paths.py: extracts the per-host data-path / file-list helpers out ofdataset_tools.pyinto a ROOT/narf-free module, so the param model can resolve the default bT-grid location without importing ROOT mid-fit.wremnants/utilities/io_tools/input_tools.py: it's unclear what the correct nomenclature for these is, but all the ones I have seen disagree with the current code, but agreed with the old one, so I've reverted it, but left it as a comment.Depends on rabbit's 138 & related PRs.
🤖 Generated with Claude Code