Repository navigation
feat(indicators): document and expose the approximate hypervolume, with its seed and method - #363
Merged
Merged
Conversation
…th its seed and method
HVApproxIndicator (hv_approx), MooCore's hv_approx, was internal and
documented only in docs/dev/add_metric.md, and it passed neither a seed
nor a method to MooCore (VAMOS-Studio request U2). The exact
hypervolume can take minutes from about six objectives, so a tool that
computes indicators on request needs the approximation as a public,
documented indicator.
- The experimental vamos.ux.api exports HVIndicator, HVApproxIndicator,
HV_APPROX_METHODS and IndicatorResult, the existing classes of
vamos.foundation.quality_indicators (no parallel registry).
- HVApproxIndicator.compute takes seed (an integer, a Generator or
None, used by "DZ2019-MC") and method, one of HV_APPROX_METHODS
("Rphi-FWE+", "DZ2019-HW", "DZ2019-MC"); an unknown method raises
ValueError. None keeps MooCore's default, read from hv_approx's
signature ("Rphi-FWE+" since MooCore 0.3.0, "DZ2019-HW" in 0.2.x), so
the value without arguments does not change. details records samples,
seed and the method used.
- docs/topics/analysis.md documents both indicators, the exact
hypervolume's measured cost from six objectives, the options and the
accuracy on DTLZ2, with an example that the docs smoke runs.
Tests: on 286 points of DTLZ2's front with four objectives the estimate
is within 1 % of the exact value with every method; with "DZ2019-MC" a
seed gives the same value and another seed another one; the details
record MooCore's default; an unknown method raises; the facade exports
the classes that get_indicator builds.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Codex Review Summary: ✅ CompletedThis comment shows the latest Codex review activity on this pull request.
ℹ️ About Codex in GitHubYour team has set up Codex to review pull requests in this repo. Reviews are triggered when you
Codex reacts with 👀 while any review is running, comments if it has suggestions, and reacts with 👍 once all reviews finish with no findings. |
Contributor
VAMOS documentation previewhttps://pr-363-vamos-docs-preview.vamos-optimization.workers.dev Built from |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Makes the approximate hypervolume a documented, public indicator next to the exact one, and passes
seedandmethodthrough to MooCore (VAMOS-Studio change request U2). VAMOS-Studio computes indicators on request, and the exact hypervolume can take minutes from about six objectives; it can offer the approximation once it is reachable through a documented facade instead of an internal module.vamos.ux.apifacade (the one for analysis helpers) now exportsHVIndicator(exact,moocore.hypervolume),HVApproxIndicator(moocore.hv_approx),HV_APPROX_METHODSand theirIndicatorResult. They are the existing classes ofvamos.foundation.quality_indicators, re-exported: no parallel registry,get_indicator("hv_approx", ...)returns the same class, and the internal package exportsHV_APPROX_METHODStoo. The API snapshot was regenerated withtools/update_public_api_snapshot.py.HVApproxIndicator.compute(front, samples=10_000, seed=None, method=None):seed(an integer, a NumPyGeneratororNone) andmethodare passed tomoocore.hv_approx;sampleskeeps its default of 10 000.methodis one ofHV_APPROX_METHODS = ("Rphi-FWE+", "DZ2019-HW", "DZ2019-MC"), MooCore's methods. An unknown method raisesValueErrorlisting them (MooCore's own message is"Unknown method = {method}", without the value).method=Nonekeeps MooCore's default, read frominspect.signature(moocore.hv_approx):"Rphi-FWE+"since MooCore 0.3.0 and"DZ2019-HW"in 0.2.x (VAMOS allowsmoocore>=0.2.0, and 0.2.0 has no"Rphi-FWE+"). So the value computed without arguments does not change.detailsnow recordssamples,seedand themethodused, next toreference_point.docs/topics/analysis.md(the experimental Analysis and Visualization page) gains a section "Hypervolume, exact and approximate": what both indicators take and return, the exact hypervolume's cost from about six objectives with measured times, the three options, the measured accuracy on DTLZ2, and a runnable example (part of the analysis docs smoke, which now also needs MooCore and counts five code blocks).docs/dev/add_metric.mdrecords the exported names and the pass-through in its indicator contract.Sources: MooCore's
hv_approxsignature and docstring in the installed 0.3.2 (hv_approx(points, /, ref, *, maximise=False, nsamples=262144, seed=None, method='Rphi-FWE+');"Rphi-FWE+"and"DZ2019-HW"deterministic,"DZ2019-MC"Monte Carlo fromseed) and in the 0.2.0 and 0.3.0 wheels (0.2.0: methods"DZ2019-HW"(default) and"DZ2019-MC"); MooCore's documentation ofhv_approx(the integral over the hypersphere of Deng and Zhang, 2019, which all three methods approximate).Measured on this laptop, MooCore 0.3.2, points near DTLZ2's sphere with reference point 1.1: the exact hypervolume of 2000 points with six objectives took 17 s, 400 with seven 16 s and 200 with eight 32 s;
hv_approxwith 10 000 samples took 0.01–0.03 s. On 286 points of DTLZ2's front with four objectives, the deterministic methods with 10 000 samples are within 0.04 % of the exact value, and"DZ2019-MC"with 100 000 samples within 0.26 % over seeds 1–10.Tests
tests/foundation/test_moocore_indicators.py:test_hv_approx_agrees_with_the_exact_hypervolume_on_dtlz2_with_four_objectives: 286 points of DTLZ2's front with four objectives (a Das-Dennis lattice projected onto the sphere), reference point 1.1; the estimate agrees withHVIndicator's exact value within 1 % for the default method and each of the three methods (10 000 samples, or 100 000 for"DZ2019-MC").test_hv_approx_passes_its_seed_to_moocore: with"DZ2019-MC", the same seed gives the same value, another seed another value, and the details record seed and method.test_hv_approx_records_moocores_default_method: without arguments, the details record 10 000 samples, seedNoneand MooCore's default method, and the value equalsmoocore.hv_approxwith that method.test_hv_approx_rejects_an_unknown_method.test_the_analysis_facade_exports_the_exact_and_the_approximate_hypervolume:vamos.ux.apiexports the same classes thatget_indicatorbuilds.tests/docs/test_analysis_docs.pychecks the new section;tests/docs/smoke_manifest.pyruns its example.Compatibility
Additive. Four new names in the experimental
vamos.ux.api;hv_approxtakes two new optional keyword arguments; its value without them is unchanged (MooCore's default method, 10 000 samples);detailsgains three keys.HV_APPROX_METHODSis new invamos.foundation.quality_indicators.Validation
Run from the worktree with
PYTHONPATHset to itssrc(vamos.__file__checked):python -m pytest -q -n 4 tests/foundation/test_moocore_indicators.py tests/foundation/test_hypervolume_fallback.py tests/architecture/test_public_api_snapshot.py tests/test_public_api_guard.py tests/docs/test_analysis_docs.py tests/docs/test_docs_smoke.py tests/test_import_time_smoke.py tests/integration/test_public_api.py tests/architecture/test_no_facade_imports.py tests/architecture/test_layer_boundaries.py: 84 passed, 4 xfailed (the existing strict xfails of the import-time smoke where MooCore is installed).ruff checkandruff format --checkon the changed Python: clean.python -m compileall: clean.git diff --check: clean.python tools/typecheck.py --scope strictand--scope full: 0 diagnostics; ratchet new=0, increased=0.python tools/check_agent_docs.py: passed.python tools/check_repository_hygiene.py: passed.python -m pytest -m "not slow" -n 4 -p no:cacheprovider -rfE: 10847 passed, 253 skipped, 9 xfailed, 2 failed.tests/experiment/test_cli_ablation.py::test_cli_ablation_runs_and_writes_variantsis the known failure under xdist on Windows that also fails on untouchedmainand passes alone.tests/docs/test_cookbook_recipes.py::test_cookbook_python_recipe[15. Verify and reproduce a stored run]failed because a comment insrc/vamos/ux/api.pywas edited while the tier ran, which changed the source between recording and verifying the run (compatibleinstead ofexact); the whole cookbook module then passed (22 passed), and the targeted tests above were rerun on the final files (119 passed withtests/experiment/test_study_analysis.pyandtests/experiment/test_benchmark_reporting.pyadded).python tools/health.py: all health checks passed (on the final files).mkdocs build --strict: built.Changelog entry
Added (under [Unreleased])
vamos.ux.apiexportsHVIndicator,HVApproxIndicator,HV_APPROX_METHODSandIndicatorResult, documented in Analysis and Visualization with a note on the exact hypervolume's cost from about six objectives.HVApproxIndicator.computepassesseedandmethodtomoocore.hv_approx(an unknown method raisesValueError;Nonekeeps MooCore's default) and recordssamples,seedand the method used indetails.Checklist
Architecture health
docs/dev/architecture_health.mdrelevant to this change.python tools/health.py.docs/dev/architecture_health.mdif I changed guardrails. (No guardrail changed.)Public API & dependencies
python tools/update_public_api_snapshot.pyif needed. (Four names added to the experimentalvamos.ux.api.)[project].dependencies.Behavior & tests
Changelog
CHANGELOG.mdis unchanged, and the changelog entry is under a "Changelog entry" heading in this description.Reports/retention
docs/dev/repository_hygiene.mdandpython tools/check_repository_hygiene.pypasses.🤖 Generated with Claude Code