Repository navigation
FEAT: evaluate mass models, the likelihood and the VT in log space - #144
Merged
Merged
Conversation
set_backend gains enable_x64. True or False sets jax_enable_x64
explicitly. The default, None, enables 64-bit floats the first time the
jax backend is set, as before, and leaves the precision alone
afterwards, so a later set_backend("jax"), e.g., from a downstream
package, does not silently undo an explicit choice of single precision.
jax is now configured after the backend is imported, so a missing jax
still raises the usual ModuleNotFoundError.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…dels Add ln_powerlaw, ln_truncnorm, ln_two_component_single and ln_two_component_primary_mass_ratio. They return the log density, -inf outside the support, and never leave log space, so they stay finite where the linear densities underflow (below ~1e-38 in single precision, ~1e-308 in double). ln_powerlaw computes the normalisation from log(low) and log(high), so it does not overflow for large |alpha|; xlogy keeps x^0 = 1 at x = 0. The two-component mixture uses logaddexp. truncnorm is now exp(ln_truncnorm). logsubexp uses log1p(-exp(.)) in place of log(1 - exp(.)), which is slightly more accurate, so values can differ from before in the last few bits. The constant log(2 pi) is a Python float, as a numpy float64 scalar promoted single precision inputs to double. On a batch of 1000 population draws over 266k injections on an A100, ln_two_component_primary_mass_ratio takes 14 ms in float64 and 5.6 ms in float32, against 28 ms for log(two_component_primary_mass_ratio) in float64. Float32 matches float64 to 2e-3 in log density with the same support. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
gwpopulation.hyperpe.Model subclasses bilby.hyper.model.Model and adds ln_prob, the sum of the component log densities. prob is unchanged. For each component the log density is, in order: its ln_prob attribute (classes, or a function with one attached); ln_<name> from the component's module, if it takes the same arguments; else the log of the component, so existing models keep working. Log values are cached separately from linear ones under the same rules as bilby. CosmoModel now subclasses this Model, and its ln_prob subtracts the log of the Jacobian. HyperparameterLikelihood still wraps plain functions with bilby's Model, so likelihoods are unchanged; moving them to the log path is left for later. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
ln_two_component_primary_mass_ratio is evaluated on large fixed sample sets for many hyperparameters, so its cost is set by the per-element transcendentals: 7 before (3 logs, a divide, exp, log1p, expm1), 4 now (log, exp, log1p, expm1, the minimum for logaddexp and the m1-dependent mass-ratio normalisation). - New ln_powerlaw_from_logs(ln_xx, alpha, ln_high, ln_low), the log power law given logs of the abscissa and bounds. The mass-ratio lower bound mmin / m1 becomes ln mmin - ln m1, with no log of the ratio. - The power-law normalisation no longer evaluates a separate exponent == 0 branch on every element: a tiny exponent (1e-12) gives the same -ln(width) limit, saving a log where the bounds depend on the data. ln_powerlaw shares the helper. - ln_truncnorm multiplies by 1 / sigma rather than dividing and sums the terms independent of the data first. - ln_two_component_primary_mass_ratio uses dataset["ln_mass_1"] and dataset["ln_mass_ratio"] when present, else takes the logs. With jax, XLA already hoists those logs out of a vmap over hyperparameters, so the columns mainly help other backends. On an A100 in single precision (500 hyperparameter draws x 266k samples) the model takes 2.17 ms instead of 2.57 ms. Float64 results agree with before to 2e-11, and float32 accuracy against float64 is unchanged. Tests compare the log-input path with the plain one in double and single precision and cover beta = -1, alpha = 0 and a zero lower bound. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
- HyperparameterLikelihood and ResamplingVT weight samples by exp(ln_prob - ln_prior), from the model's ln_prob if it has one (e.g., hyperpe.Model) and the log of prob otherwise, and form the means, variances and P_det with logsumexp. Weights that underflow, e.g., in single precision, no longer give -inf. - Posteriors and injections can give their sampling prior as ln_prior, used in preference to prior, so it cannot underflow either. - With unequal numbers of samples, each event is summed separately (utils.segment_logsumexp) instead of by differences of one cumulative sum over all events, which lost precision when events' sums differ greatly (in float32, up to 1e-2 in ln L on 46 GWTC-4.0 events; now 3e-4). With jax the events are gathered into a padded array, as fast as before on GPU; segment_max/segment_sum were 8x slower. - ResamplingVT stores total_generated as a float: its square overflowed a 32-bit integer without x64. - Plain functions passed to HyperparameterLikelihood and the VTs are wrapped in hyperpe.Model, so a matching ln_<name> is used. posterior_predictive_resample and the numpyro helpers still use the linear weights. Tests compare the log-space per-event ln BF, variances and P_det with the linear formulas, check ln_prior against prior, weights of exp(-1000), events whose weights differ by e^500, and logsumexp and segment_logsumexp against scipy on each backend. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Codecov Report❌ Patch coverage is
Additional details and impacted files@@ Coverage Diff @@
## main #144 +/- ##
==========================================
- Coverage 95.25% 95.12% -0.14%
==========================================
Files 14 14
Lines 948 1085 +137
==========================================
+ Hits 903 1032 +129
- Misses 45 53 +8
Flags with carried forward coverage won't be shown. Click here to find out more. ☔ View full report in Codecov by Harness. 🚀 New features to boost your workflow:
|
bilby.hyper.model.Model has no ln_prob, so a likelihood or VT given one, as in the examples, took the log of the product of the components and never used their log densities, e.g., the ln_* mass models. HyperparameterLikelihood and the VTs now convert a plain bilby Model to gwpopulation.hyperpe.Model with the same components, cache setting and parameters (as_ln_model). Subclasses of the bilby Model, which may change prob, are left as they are. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
- Population models: how a model provides its log density (an ln_prob attribute or an ln_<name> function), an example, the provided ln_* mass models and the optional ln_mass_1 / ln_mass_ratio columns. The class example defines __call__, not __init__. - Setting the Backend: a Precision section on enable_x64=False and when single precision is safe. - Likelihood and selection: the Monte Carlo sums are in log space and the sampling prior can be given as ln_prior. - FAQ: double precision is no longer strictly required of a backend. - Examples: use gwpopulation.hyperpe.Model; GWTC1 mentions ln_prior. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Builds on the
enable-x64PR: the first commit (2b9a91a) is that PR, and the single-precision tests here useset_backend("jax", enable_x64=False). Review from 028f525 on; this targetsmainso CI runs.In single precision the linear densities underflow below ~1e-38 (~1e-308 in double), and the Monte Carlo weights with them, so ln L becomes -inf far from the bulk. This adds a log-probability path that never leaves log space, and uses it for the likelihood and selection-function sums.
Log-space models (028f525, b485a47)
utils:ln_powerlaw,ln_powerlaw_from_logs(logs of the abscissa and bounds) andln_truncnorm.models.mass:ln_two_component_singleandln_two_component_primary_mass_ratio. They return the log density,-infoutside the support.ln_powerlawnormalises fromlog(low)andlog(high), so it doesn't overflow for large |alpha|;xlogykeeps x^0 = 1 at x = 0. The mixture useslogaddexp.ln_two_component_primary_mass_ratiousesdataset["ln_mass_1"]anddataset["ln_mass_ratio"]when present (mainly helps non-jax backends, as XLA hoists the logs out of a vmap anyway).log(two_component_primary_mass_ratio)in float64. b485a47 cuts the per-element transcendentals from 7 to 4 (2.57 → 2.17 ms for 500 draws in float32). Float32 matches float64 to 2e-3 in log density.hyperpe.Model.ln_prob(c89b461)gwpopulation.hyperpe.Modelsubclassesbilby.hyper.model.Modeland addsln_prob, the sum of the component log densities;probis unchanged.ln_probattribute;ln_<name>from the component's module, if it takes the same arguments; else the log of the component, so existing models keep working. Log values are cached separately under bilby's rules.CosmoModelsubclasses thisModel; itsln_probsubtracts the log Jacobian.Likelihood and
ResamplingVTin log space (3a6c0fa)HyperparameterLikelihoodandResamplingVTweight samples byexp(ln_prob - ln_prior), using the model'sln_probwhen it has one andlog(prob)otherwise, and form the means, variances and P_det withlogsumexp.ln_prior, used in preference toprior.utils.segment_logsumexp) instead of by differences of one cumulative sum, which lost precision when events' sums differ greatly (float32 error in ln L on 46 GWTC-4.0 events: 1e-2 → 3e-4). With jax the events are gathered into a padded array, as fast as before on GPU.ResamplingVT.total_generatedis stored as a float; its square overflowed a 32-bit integer without x64.HyperparameterLikelihoodand the VTs are wrapped inhyperpe.Model, so a matchingln_<name>is picked up.Changes for existing users
ResamplingVTnow always use the log-space sums, in double precision too. Results agree with the linear formulas to rounding (tested), but this is the default path for everyone.truncnormis nowexp(ln_truncnorm), andlogsubexpuseslog1p(-exp(.)), so values can differ in the last few bits.posterior_predictive_resampleand the numpyro helpers still use linear weights.For
CosmoModelin single precision, distances and volumes also need the wcosmo single-precision PR (ColmTalbot/wcosmo, branchfloat32).Testing
ln_priorwithprior, and cover weights of exp(-1000) and events whose weights differ by e^500;logsumexpandsegment_logsumexpare checked against scipy on each backend.🤖 Generated with Claude Code