Skip to content

feat: one source of reference vectors for MOEA/D, NSGA-III and RVEA, two-layer lattices and moead_populations (A13, A7, A12) - #365

Merged
NicolasRodriguezUribe merged 5 commits into
mainfrom
feat/reference-vector-files
Nov 19, 2026
Merged

NicolasRodriguezUribe merged 5 commits into
mainfrom
feat/reference-vector-files

Conversation

@NicolasRodriguezUribe

@NicolasRodriguezUribe NicolasRodriguezUribe commented Nov 19, 2026 •

Copy link
Copy Markdown
Contributor

Builds on #361 (fix/weight-vector-files, A14 and A8): this branch starts from its two commits, which this PR also shows until #361 is merged. Review the last two commits: the vector sources, then moead_populations.

Problem

Items A13, A7 and A12 of the VAMOS-Studio change requests (2026-11-18). jMetal and Evolver configure MOEA/D, NSGA-III and RVEA from the same weight-vector files (jMetal's MOEADBuilder and NSGAIIIBuilder take a directory or a list of vectors, RVEABuilder a list), so the three start from the same vectors. In VAMOS 1.4.0 each algorithm differed:

  • MOEA/D: weight_vectors={"path": directory or file, "divisions": H}.
  • NSGA-III: reference_directions={"path": file, "divisions": H}, a single file (comma-separated only until fix: every packaged weight file loads (W4D_100.dat, W4D_165.dat) and NSGA-III reads them (A14, A8) #361).
  • RVEA: only n_partitions, so its population had to be a lattice: 100 individuals with three objectives failed (RVEA requires pop_size == #ref_dirs), and it could not read W3D_100.dat (A7).
  • With many objectives MOEA/D had only single-layer lattices or the few packaged files, and no listing of the populations it accepts (A12): with six objectives and 100 individuals the studio had to fall back to 56.

Change

One kind of option for the three algorithms, through the shared load_or_generate_weight_vectors, with the existing names kept:

Algorithm Option (field and builder method)
MOEA/D weight_vectors (existing)
NSGA-III reference_directions (existing)
RVEA reference_vectors (new, the last field, default None)

Each takes None, a path, or a mapping with:

  • path: a file with one vector per line, components separated by whitespace or commas (fix: every packaged weight file loads (W4D_100.dat, W4D_165.dat) and NSGA-III reads them (A14, A8) #361's reader), or a directory with jMetal's W{n_obj}D_{pop_size}.dat names. NSGA-III gains directories; RVEA gains files and directories.
  • divisions: the complete Das-Dennis lattice. For RVEA, reference_vectors(divisions=12) runs exactly the lattice of n_partitions(12) (same generator, same results).
  • inner_divisions (new, with divisions): Deb and Jain's two-layer lattice, the boundary lattice of divisions plus the lattice of inner_divisions shrunk by half towards the centre (w / 2 + 1 / (2 m), as pymoo's multi-layer directions with scaling 0.5, PlatEMO's UniformPoint and VAMOS's own C-TAEA weights). reference_directions.two_layer_lattice(8, 3, 2) has 120 + 36 = 156 vectors, the size of jMetal's W8D_156.dat. (jMetal's file stacks the two lattices without shrinking the inside one, so it repeats the eight axes; the shrunk layer is the published construction.)

Why these names: path and divisions are the keys MOEA/D and NSGA-III already had, so existing configurations keep their meaning and their serialized form; inner_divisions names Deb and Jain's "inside layer" next to divisions; RVEA's field is reference_vectors, the algorithm's own term (and the name A7 proposes), with the same keys. n_partitions stays RVEA's stable required field and its default lattice; with reference_vectors it is not used, and the builder no longer requires it (it records divisions, or 12 for a file).

Rules (vector_source, applied when a configuration is built):

  • The vectors are the population: MOEA/D and RVEA need exactly one individual per vector, NSGA-III at least one (N >= H, unchanged).
  • A path that does not exist raises ConfigurationError before any evaluation, for all three; a directory without the file raises for NSGA-III and RVEA. MOEA/D keeps its documented 1.3 behaviour for an existing directory that lacks the file (it saves the exact lattice there with a RuntimeWarning), but no longer creates a missing directory.
  • A mismatching file or lattice raises (ValueError, as before for MOEA/D).
  • inner_divisions needs divisions and excludes path; an unknown key, a non-positive count or a non-string path raises ConfigurationError; a plain string is a path.

A12 (1): vamos.algorithms.moead_populations(n_obj, *, max_pop_size=1000) lists the populations MOEA/D runs without weight options (every size from 2 with one or two objectives; the complete lattices and the packaged jMetal files with more), as MOEADConfig.default(pop_size=...) and optimize build them. It is exported from vamos.algorithms and documented in the MOEA/D configuration reference; the public API snapshot is regenerated with tools/update_public_api_snapshot.py.

Run metadata, serialization, CLI and replay:

  • MOEA/D's algorithm.resolution.weight_vectors gains {"source": "two_layer", "divisions": 3, "inner_divisions": 2, "vectors": 156}.
  • NSGA-III and RVEA record a path in algorithm.resolution.reference_directions ({"source": "file", "file": "W3D_100.dat"}) and algorithm.resolution.reference_vectors (also "vectors", the population). Lattices are not recorded: the configuration already states them, and recording them would change the resolved spec, and the task id, of every NSGA-III and RVEA run, so runs and studies stored by 1.4.0 would no longer verify, replay or resume exactly (study execution reconstructs each task from its stored spec). The record is built from the file's name, never its contents, since verification rebuilds it from an untrusted stored path.
  • to_dict() leaves out reference_vectors while it is None and inner_divisions while it is unset, so configurations without the new options serialize, and resolve, exactly as in 1.4.0; from_dict round-trips the new ones. The 1.0 compatibility snapshots pass unchanged.
  • Command line and experiment specs: an rvea: block takes reference_vectors (a mapping or a path string); the command-line RVEA builder no longer derives n_partitions from the population when reference_vectors is given. weight_vectors and reference_directions blocks take inner_divisions.
  • Study planning: RVEA's reference_vectors.path is rejected as UNSAFE_REFERENCE_DIRECTIONS (as NSGA-III's path already is), and pop_size must equal the vectors of divisions/inner_divisions; NSGA-III's check counts a two-layer lattice. Planning no longer reports a field that to_dict() leaves out at its default as unknown.
  • Exact replay of runs with a file (RVEA), a directory (NSGA-III) and two-layer lattices (MOEA/D, RVEA) is tested with save_result, verify_run and reproduce.

Docs, each with Development after 1.4.0:: a new "Reference and weight vectors" section in the algorithms reference, the configuration references of MOEA/D (two layers, moead_populations), NSGA-III and RVEA (with an example), the RVEA tutorial, the CLI guide (an rvea: block with a weight file) and the run-artifact contract.

Sources

  • jMetal (C:/tmp/jmetal-pr, local/integration 9b6734c12): resources/weightVectorFiles/moead/W3D_100.dat and W8D_156.dat (VAMOS ships both), WeightVectorNeighborhood (whitespace reader), RVEAEnvironmentalSelection (takes any list of vectors).
  • Deb and Jain (2014), NSGA-III part I, two-layer reference points; pymoo's MultiLayerReferenceDirectionFactory with scaling=0.5; PlatEMO's UniformPoint (also in VAMOS's ctaea/weights.py).

Tests

tests/engine/test_reference_vector_sources.py (new):

  • MOEA/D, NSGA-III and RVEA with three objectives and W3D_100.dat (population 100), and with eight objectives and W8D_156.dat (156), given the file and given the packaged directory, start from the same vectors (compared after initialize); so do the three with the two-layer lattice of 8 objectives, divisions=3, inner_divisions=2.
  • The two-layer lattice of 8 objectives (3, 2) has 156 vectors, as W8D_156.dat, with the boundary layer first and the inside layer shrunk.
  • A missing path raises ConfigurationError for all three and creates nothing; a directory without the file raises for NSGA-III and RVEA; MOEA/D still saves the lattice in an existing directory; a file with another number of vectors raises for MOEA/D and RVEA.
  • RVEA runs 100 individuals with three objectives from W3D_100.dat with the exact budget; reference_vectors(divisions=12) gives the same results as n_partitions(12).
  • Invalid options are rejected for all three; a path string is a path; configurations round-trip through to_dict/from_dict; configurations without the new options serialize as in 1.4.0.
  • Resolved-spec records (NSGA-III and RVEA file, MOEA/D two layers, none for NSGA-III and RVEA lattices); exact replay of four new cases; the command-line builder reads RVEA's reference_vectors; study planning of RVEA and NSGA-III two-layer lattices and of an RVEA path.
  • moead_populations for 2, 3, 4, 8 and 20 objectives; for 4 to 20 objectives every listed population up to 300 loads its weights and builds MOEADConfig.default, and the smallest runs; MOEA/D runs a 10-objective two-layer lattice (65 vectors) that no single lattice has.
  • Checked by hand: vamos --config spec.yaml with rvea: {reference_vectors: .../W3D_100.dat} and population_size: 100 runs and records {"source": "file", "file": "W3D_100.dat", "vectors": 100}.

Compatibility

Additive for configurations that do not use the new options: same fields, same to_dict(), same resolved specs and task ids, same results (the 1.0 compatibility snapshots pass unchanged; RVEA's new field is last, with a default). Changes:

  • A weight_vectors path that does not exist now raises ConfigurationError where VAMOS 1.4.0 created it as a directory and saved the lattice there (with a RuntimeWarning).
  • An unknown key in weight_vectors or reference_directions now raises ConfigurationError; VAMOS 1.4.0 ignored it.
  • RVEA's builder no longer requires n_partitions when reference_vectors is given.
  • vamos.algorithms exports moead_populations (new, not yet in the 1.0 baseline).

Validation

All on Windows 11, Python 3.12, with PYTHONPATH set to the worktree's src (checked with vamos.__file__).

Command Result
pytest -q -n 4 on the new and nearest tests (test_reference_vector_sources, MOEA/D weight and provenance tests, NSGA-III tests, RVEA tests, config round-trip and field tests, tests/compatibility, tests/architecture, tests/experiment/study_manifest, CLI/API parity, scientific defaults, tuning population, registry, smoke and warm-start tests) 1029 passed, 1 skipped, 2 xfailed; after the last fixes, tests/architecture, tests/compatibility, tests/experiment/study_manifest, the run-artifact replay and ablation-study tests and the new tests: 904 passed; on the final commits: 688 passed, 1 skipped
ruff check and ruff format --check on the changed Python passed
python tools/typecheck.py --scope strict / --scope full 0 diagnostics / 0 diagnostics, ratchet new=0, increased=0
python -m compileall on the changed Python; git diff --check passed
python tools/check_agent_docs.py passed
pytest -m "not slow" -n 4 -p no:cacheprovider -rfE 10958 passed, 253 skipped, 9 xfailed, 3 failed: test_constraint_handling_follows_the_fields_of_1_0[RVEAConfig] (the list of fields added after 1.0 now names reference_vectors, updated) and tests/test_monolith_guard.py (planning.py over its 450-line budget, fixed by moving the reference-vector checks to study/reference_planning.py), both passing after the fix, and the known tests/experiment/test_cli_ablation.py::test_cli_ablation_runs_and_writes_variants, which fails under xdist on Windows on untouched main too and passes alone
python tools/health.py all health checks passed (after the fixes)
mkdocs build --strict built, no warnings

Update after review

Two commits answer the automated review:

  • 51a8804, overlapping layers. A two-layer lattice whose inside layer repeats directions of the boundary layer is rejected. For example, 2 objectives with divisions=4 and inner_divisions=2 gives 8 rows but only 5 distinct directions.
    • two_layer_lattice raises ValueError; MOEA/D, NSGA-III and RVEA raise ConfigurationError.
    • Study planning rejects such settings as INVALID_REFERENCE_DIRECTIONS.
    • two_layer_shared counts the repeated directions exactly. With inner_divisions < n_obj and divisions < 2 n_obj there are none.
  • 660d085, vector files bound by content. When a run starts, a path is resolved to the file the run reads, and sha256, the digest of its vectors, is added to the resolved configuration.
    • Binding happens in optimize, the command-line builders, build_algorithm with a configuration, and study planning for MOEA/D.
    • The run checks the digest before any evaluation, so replay refuses a changed file instead of running another experiment. MOEA/D's and NSGA-III's files had this gap on main too.
    • The file records are rebuilt from the stored configuration alone, so a run verifies when its file has moved, or on another machine.
    • The helpers move to components/vector_files.py, because weight_vectors.py reached its size budget.

Validation of the updated branch:

  • pytest -m "not slow" -n 4: 10971 passed, 253 skipped, 9 xfailed, 1 failed (the known test_cli_ablation_runs_and_writes_variants).
  • tools/health.py: passed. mkdocs build --strict: no warnings.
  • Strict and full typecheck: 0 diagnostics, no ratchet increase.
  • ruff, compileall, git diff --check and check_agent_docs: passed.

Changelog entry

Added

  • MOEA/D, NSGA-III and RVEA take their vectors from one kind of option, as jMetal and Evolver configure them: MOEA/D's weight_vectors, NSGA-III's reference_directions and RVEA's new reference_vectors accept a file (whitespace- or comma-separated) or a directory with jMetal's W<m>D_<n>.dat names, the lattice of divisions, or, with the new inner_divisions, Deb and Jain's two-layer lattice (156 vectors with eight objectives, 3 and 2 divisions). The vectors are the population, so RVEA can run 100 individuals with three objectives from W3D_100.dat, and the three algorithms can start from the same vectors. A path that does not exist raises ConfigurationError. The resolved spec records a file for NSGA-III and RVEA and the two-layer lattice for MOEA/D. vamos.algorithms.moead_populations(n_obj) lists the populations MOEA/D runs without weight options.

Changed

  • Two-layer lattices whose layers repeat a direction raise ConfigurationError. A MOEA/D weight_vectors path that does not exist raises ConfigurationError instead of being created as a directory for the generated lattice, and an unknown key in weight_vectors or reference_directions raises instead of being ignored.

Fixed

  • A run whose MOEA/D weight_vectors, NSGA-III reference_directions or RVEA reference_vectors names a file records the file it read and the sha256 of its vectors in its resolved configuration. Replay refuses the file if its vectors have changed, where VAMOS 1.4.0 replayed whatever the file then held. The run's record of the file is rebuilt from the stored configuration, so the run verifies without the file.

🤖 Generated with Claude Code

…, with two-layer lattices

MOEA/D's weight_vectors, NSGA-III's reference_directions and RVEA's new
reference_vectors (the last field, default None) take the same input, as jMetal
and Evolver configure the three from the same weight files: a file with
whitespace or commas, or a directory with jMetal's W<m>D_<n>.dat names (new
for NSGA-III and RVEA); divisions, the complete Das-Dennis lattice (for RVEA the
lattice of n_partitions); and, new, inner_divisions with divisions, Deb and
Jain's two-layer lattice, the inside layer shrunk by half towards the centre
as in pymoo and PlatEMO (156 vectors with eight objectives, 3 and 2
divisions). All go through load_or_generate_weight_vectors, and vector_source
checks the options when a configuration is built.

The vectors are the population: RVEA runs 100 individuals with three
objectives from W3D_100.dat. A path that does not exist raises
ConfigurationError for all three (MOEA/D created it as a directory); a
directory without the file raises for NSGA-III and RVEA, while MOEA/D keeps
saving the lattice in an existing directory.

to_dict() leaves out reference_vectors while it is None and inner_divisions
while unset, so configurations without them keep their 1.4.0 resolved specs
and task ids. The resolved spec records MOEA/D's two-layer lattice and the
file of NSGA-III and RVEA (lattices are already in their configuration).
RVEA's command-line block takes reference_vectors; study planning rejects
its path, as NSGA-III's, checks two-layer counts, and moves these checks to
study/reference_planning.py.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
… vectors

vamos.algorithms.moead_populations(n_obj, *, max_pop_size=1000) returns the
populations MOEA/D runs without weight options, as MOEADConfig.default and
optimize build them: every size from 2 with one or two objectives, and with
more the complete Das-Dennis lattices and the packaged jMetal files (100 with
three or four objectives, 156 with eight). Other sizes need a file or a
two-layer lattice, which the docstring and the MOEA/D reference explain.

The tests run MOEA/D for 4 to 20 objectives with the populations it lists, and
with a 10-objective two-layer lattice that no single lattice has. The public
API snapshot is regenerated with tools/update_public_api_snapshot.py.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@chatgpt-codex-connector

chatgpt-codex-connector Bot commented Nov 19, 2026 •

Copy link
Copy Markdown
Codex Review Summary: ✅ Completed

This comment shows the latest Codex review activity on this pull request.

Review Status Commit Review trigger
📝 Code Review ✅ Completed 2026-11-19T00:11:25.008514Z d96b6cc PR opened
ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review" or "@codex security review".

Codex reacts with 👀 while any review is running, comments if it has suggestions, and reacts with 👍 once all reviews finish with no findings.

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: d96b6cc2e8

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

@github-actions

github-actions Bot commented Nov 19, 2026 •

Copy link
Copy Markdown
Contributor

VAMOS documentation preview

https://pr-365-vamos-docs-preview.vamos-optimization.workers.dev

Built from b01be51. Canonical metadata points to the production .org domain; the preview itself remains isolated.

…ions

Settings such as two objectives with divisions=4 and inner_divisions=2 gave
eight vectors but five directions: the three inside directions lie on the
boundary lattice. RVEA accepted them as an eight-vector population, although a
repeated niche never gets its own survivor and its zero angle to the nearest
vector is clamped to 1e-64 in the penalty. two_layer_shared counts the shared
directions exactly on the grid of both layers; two_layer_lattice raises
ValueError, and MOEA/D, NSGA-III and RVEA raise ConfigurationError, for any
overlap; study planning rejects it as INVALID_REFERENCE_DIRECTIONS. With
inner_divisions < n_obj and divisions < 2 n_obj the layers never overlap.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…ther vectors

A path in MOEA/D's weight_vectors, NSGA-III's reference_directions or RVEA's
reference_vectors was replayed with whatever the file held when replay ran,
while the resolved spec recorded only its name: a file replaced by other valid
vectors of the same shape still verified as exact and reproduce() ran another
experiment. MOEA/D's and NSGA-III's files had this gap on main too. The record
of NSGA-III's and RVEA's files was also rebuilt with os.path.isfile, so a run
whose file was moved, or verified on another machine, no longer matched it.

When a run starts (optimize, the command-line builders, build_algorithm with a
configuration, and study planning for MOEA/D), bind_config_vector_files
resolves the path to the file the run reads (a directory becomes its
W<m>D_<n>.dat) and records sha256, the digest of its vectors' shape and float64
values, in the resolved configuration. The run checks the digest before any
evaluation, so replay refuses a changed file. The file records are rebuilt
from the stored configuration alone; a MOEA/D record with sha256 is das_dennis
when the digest is that of the lattice. A MOEA/D directory without the file,
whose run generates the lattice, stays unbound, and runs stored without a
digest keep their 1.4.0 description. The helpers move to
components/vector_files.py, as weight_vectors.py reached its size budget.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
NSGAIIIConfig.__post_init__ keeps both checks: the reference-direction source
of this branch and the selection of #364.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
NicolasRodriguezUribe added a commit that referenced this pull request Nov 19, 2026
Brings #365's merge of main (NSGA-III selection of #364 and the other pull
requests since), so this branch no longer conflicts with main.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@NicolasRodriguezUribe
NicolasRodriguezUribe added this pull request to the merge queue Nov 19, 2026
Merged via the queue into main with commit 962fd39 Nov 19, 2026
15 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant