Skip to content

Use distmarg for distance marginalization - #1163

Open
ColmTalbot wants to merge 4 commits into
bilby-dev:mainfrom
ColmTalbot:distmarg-distance-marginalization
Open

ColmTalbot wants to merge 4 commits into
bilby-dev:mainfrom
ColmTalbot:distmarg-distance-marginalization

Conversation

@ColmTalbot

@ColmTalbot ColmTalbot commented Oct 21, 2026 •

Copy link
Copy Markdown
Collaborator

This replaces the interpolated lookup table used for distance marginalization in GravitationalWaveTransient with distmarg. distmarg evaluates the distance-marginalized likelihood directly from the matched-filter and optimal SNRs at the reference distance. The ROQ, relative-binning and multiband likelihoods use the same distance_marginalized_likelihood, so they pick up the change without modification.

What changes

  • Likelihood: distance_marginalized_likelihood calls distmarg instead of the 400×800 lookup table and its spline.
    • The result follows the array backend of the inputs, so the JAX path still works with SCIPY_ARRAY_API=1.
    • Time-bin and calibration axes are handled.
  • Prior: the distance prior is tabulated from the prior's own cosmology for UniformSourceFrame and UniformComovingVolume, rather than from the interpolated prob. Nothing else is precomputed; previously, building the table took ~40 s per phase setting.
  • Distance reconstruction samples the exact conditional posterior. If the template is identically zero, it samples the distance prior.
  • Deprecation: distance_marginalization_lookup_table is deprecated, with a DeprecationWarning, for removal in v3.1.0. A small record of the tabulated prior is still written to cached_lookup_table_filename, because bilby_pipe reads that file during data generation.
  • Removals: BoundedRectBivariateSpline and its test are removed. The lookup table was its only user.
  • Dependency: distmarg[rust] is added to gw_requirements.txt. The rust extra installs the compiled backend: a single-point evaluation then takes ~2 µs, against ~7 µs for the table.
  • This removes the need to carry around a lookup table, so will reduce startup times for many examples and production uses by a few minutes
  • bilby_pipe can stop carrying around precomputed tables, reducing the wheel size dramatically.
  • It is also much more accurate (especially at large SNR) than the grid method (see below).

Accuracy

distmarg is accurate to ≲1e-4 in ln L across the full stress grid:

  • d* from d_min/5 to 5 d_max;
  • rho_mf from 1e-3 to 1e4, including negative values;
  • with and without phase marginalization.

This is checked against a dense reference quadrature and an independent mpmath oracle.

In a version of the fast-tutorial example:

  • distmarg matched a brute-force distance integral to ≤1.2e-6;
  • the lookup table was off by up to 1e-2;
  • sampling reproduced the evidence and distance posterior of the lookup-table likelihood within sampling noise.

Testing

  • The test suite runs at least a few minutes faster because we don't have to construct any of the lookup tables.
  • Nothing has been tested on a GPU yet.

🤖 Generated with assistance from Claude Code

ColmTalbot and others added 2 commits October 20, 2026 22:38
Replace the 400x800 interpolated lookup table with distmarg, which
evaluates the distance-marginalized likelihood directly from the
matched-filter and optimal SNRs at the reference distance. It is accurate
to ~1e-4 in ln L everywhere, needs no precomputation, and follows the
array backend of its inputs. Distance reconstruction samples the exact
conditional posterior.

The lookup-table argument is still accepted for backwards compatibility,
and a small record of the tabulated prior is still written to
cached_lookup_table_filename, which bilby_pipe reads.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
- Deprecate distance_marginalization_lookup_table (and the cache file it
  controls) for removal in bilby v3.1.0, with a DeprecationWarning.
- Remove BoundedRectBivariateSpline and its test; the distance lookup
  table was its only user.
- Reconstruct distance for a zero template by sampling the distance prior.
- Drop accuracy details from the docstring and the clip of <h|h>.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
ColmTalbot and others added 2 commits October 20, 2026 23:00
CI builds its environments from pixi.toml, which lists the gw
dependencies itself, so distmarg[rust] has to be added there as well as
to gw_requirements.txt.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
prior.rescale returns a numpy scalar; mixed with torch or JAX inputs it
made distmarg see two array namespaces in distance_marginalized_likelihood.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@ColmTalbot
ColmTalbot requested a review from a team October 21, 2026 15:32
@GregoryAshton

Copy link
Copy Markdown
Collaborator

I'm a fan of this approach. Is it possible to add a second maintainer (no need for code credit) to the repo in case you choose to leave us and do something else, and we need to bug-fix distmarg?

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants