Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 2 additions & 0 deletions docs/reference/algorithms.md
Original file line number Diff line number Diff line change
Expand Up @@ -299,6 +299,8 @@ config = (

`final_selection` is `"none"` (the default: the whole archive, as before), `"farthest"`, `"crowding"`, `"hv_greedy"` or `"kmeans"`, the selections of `top_k(method=...)`; `kmeans` is seeded with the archive's `rng_seed`, which without a value derives from the run seed (see [Reproducibility](../experiment/stopping_and_archive.md#reproducibility)). `final_size` defaults to the population size. `result.X` and `result.F` hold the subset, the archive stays whole in `result.data["archive"]` (`archive_X`/`archive_F` for NSGA-III and IBEA, `external_archive` for SPEA2), and `result.data["archive_subset"]` records the selection, the size and the indices of the selected rows in the archive. A saved run stores the subset as its result and the whole archive under `archive/` (SPEA2: `external_archive/`, since its `archive/` is the environmental archive), and its resolved specification records `final_selection` and `final_size`, so a replay selects the same subset. The subset also applies to a bounded archive larger than `final_size`, not with `result_mode="population"`, and not to hypervolume termination or live callbacks, which see the whole archive during the run.

**Development after 1.4.0:** an unbounded external archive no longer filters itself once per generation. Once it holds feasible, finite rows, it keeps the rows each generation offers and merges them, with moocore's non-dominated filter when moocore is installed, only when something reads the archive (the result, a live callback in `population_archive` mode, hypervolume termination, SMPSO's leaders) or when the kept rows outnumber twice the archive. One merge gives the same rows, in the same order, as one update per generation, so results do not change; a run that reads its archive only at the end filters it once. Over the 200 000 evaluations of an NSGA-II run, the archive took 0.1 s instead of 5.0 s on CRE21 (7 000 rows), 0.16 s instead of 11.5 s on RCM08 and 0.28 s instead of 85 s on RCM25 (112 000 rows). Before the first feasible row, and after rows with non-finite feasible objectives, the archive is updated at once, as before.

NSGA-III and RVEA require a population compatible with the configured reference directions. MOEA/D requires one weight vector per population member. For a simplex lattice with `p` divisions and `m` objectives, use `comb(p + m - 1, m - 1)`; for three objectives and 12 divisions, that is **91**. Explicit incompatible cardinalities fail; they are not warning-only cases. Follow the tutorials before changing objective count or population size.

**VAMOS 1.2.0:** NSGA-III needs at least one individual per reference direction (N >= H) instead of exactly one, as Deb and Jain, jMetal and pymoo do; 100 individuals with 12 divisions and three objectives run with 91 directions, and fewer than 91 fail. Without `divisions`, NSGA-III uses the complete Das-Dennis lattice with the most directions that fit in `pop_size`; VAMOS 1.1.0 and earlier cut the next larger lattice to `pop_size` directions and lost the extreme direction of the first objective. With two objectives, NSGA-III and RVEA default to `pop_size - 1` divisions, or 99 without `pop_size`, instead of 6. RVEA and MOEA/D still need exactly one individual per vector; RVEA then keeps one survivor per occupied vector, so its population can end smaller than `pop_size` (see [RVEA](../algorithms/rvea.md)).
Expand Down
3 changes: 2 additions & 1 deletion src/vamos/engine/algorithm/agemoea/agemoea.py
Original file line number Diff line number Diff line change
Expand Up @@ -18,6 +18,7 @@
from vamos.engine.algorithm.components.hooks import get_live_viz, live_should_stop, send_live_generation
from vamos.engine.algorithm.components.metrics import setup_hv_tracker
from vamos.engine.algorithm.components.population import initialize_evaluated_population, resolve_bounds
from vamos.engine.algorithm.components.state import record_archive_rows
from vamos.engine.algorithm.components.termination import (
capped_offspring_size,
hv_target_reached,
Expand Down Expand Up @@ -405,7 +406,7 @@ def tell(self, eval_result: Any, problem: ProblemProtocol | None = None) -> bool
else:
st.selection_ranks, st.selection_crowding = selection_metrics(self.kernel, st.F, st.G, st.constraint_mode, survival_scores)
if st.archive_manager is not None:
st.archive_X, st.archive_F = st.archive_manager.update(st.X, st.F, st.G)
record_archive_rows(st, st.X, st.F, st.G)

st.pending_offspring = None
st.generation += 1
Expand Down
3 changes: 2 additions & 1 deletion src/vamos/engine/algorithm/ccmo/ccmo.py
Original file line number Diff line number Diff line change
Expand Up @@ -17,6 +17,7 @@
from vamos.engine.algorithm.components.hooks import get_live_viz, live_should_stop, send_live_generation
from vamos.engine.algorithm.components.metrics import setup_hv_tracker
from vamos.engine.algorithm.components.population import initialize_evaluated_population, resolve_bounds
from vamos.engine.algorithm.components.state import record_archive_rows
from vamos.engine.algorithm.components.termination import capped_offspring_size, hv_target_reached, parse_termination
from vamos.engine.algorithm.config.base import normalize_constraint_mode
from vamos.engine.archive.factory import resolve_external_archive, setup_archive
Expand Down Expand Up @@ -370,7 +371,7 @@ def tell(self, eval_result: Any, problem: ProblemProtocol | None = None) -> bool

st.n_eval += X_off.shape[0]
if st.archive_manager is not None:
st.archive_X, st.archive_F = st.archive_manager.update(st.X, st.F, st.G)
record_archive_rows(st, st.X, st.F, st.G)
st.pending_offspring = None
st.pending_from_main = 0
st.generation += 1
Expand Down
3 changes: 2 additions & 1 deletion src/vamos/engine/algorithm/cmoeams/cmoeams.py
Original file line number Diff line number Diff line change
Expand Up @@ -18,6 +18,7 @@
from vamos.engine.algorithm.components.hooks import get_live_viz, live_should_stop, send_live_generation
from vamos.engine.algorithm.components.metrics import setup_hv_tracker
from vamos.engine.algorithm.components.population import initialize_evaluated_population, resolve_bounds
from vamos.engine.algorithm.components.state import record_archive_rows
from vamos.engine.algorithm.components.termination import capped_offspring_size, hv_target_reached, parse_termination
from vamos.engine.algorithm.config.base import normalize_constraint_mode
from vamos.engine.archive.factory import resolve_external_archive, setup_archive
Expand Down Expand Up @@ -418,7 +419,7 @@ def tell(self, eval_result: Any, problem: ProblemProtocol | None = None) -> bool
st.stage_b_generations += int(st.stage_b)

if st.archive_manager is not None:
st.archive_X, st.archive_F = st.archive_manager.update(st.X, st.F, st.G)
record_archive_rows(st, st.X, st.F, st.G)
st.pending_offspring = None
st.generation += 1
st.hv_reached = st.hv_reached or hv_target_reached(st)
Expand Down
10 changes: 9 additions & 1 deletion src/vamos/engine/algorithm/components/archive.py
Original file line number Diff line number Diff line change
Expand Up @@ -260,7 +260,13 @@ def _select_subset(


class UnboundedArchive(_BaseArchive):
"""Archive that keeps all non-dominated solutions without size limit."""
"""Archive that keeps all non-dominated solutions without size limit.

It is ``buffered`` unless ``buffered=False``: ``add`` keeps the rows offered while the contents are
clean and merges them, with moocore's non-dominated filter when moocore is installed, when the
contents are read or the kept rows outnumber twice the archive. The contents, their order and the
rows kept are those of one ``update`` per call.
"""

def __init__(
self,
Expand All @@ -273,6 +279,7 @@ def __init__(
decision_tolerance: float = 1e-32,
n_con: int | None = None,
initial_capacity: int = 256,
buffered: bool = True,
) -> None:
super().__init__(
None,
Expand All @@ -285,6 +292,7 @@ def __init__(
n_con=n_con,
initial_capacity=initial_capacity,
)
self.buffered = bool(buffered)


__all__ = [
Expand Down
143 changes: 143 additions & 0 deletions src/vamos/engine/algorithm/components/archive_buffer.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,143 @@
"""The row buffer of the unbounded archive: rows offered with ``add`` are merged when they are needed.

``_BaseArchive`` (``archive_core``) inherits it. A buffered archive whose contents are clean keeps the
rows it is offered and merges them in one update when its contents are read, when ``update`` is
called, or when the kept rows outnumber ``max(PENDING_ROWS, 2 * size)``, with the contents of one
update per call.
"""

from __future__ import annotations

from typing import Any, TypeAlias

import numpy as np
from numpy.typing import NDArray

from vamos.foundation.constraints.utils import is_feasible

FloatArray: TypeAlias = NDArray[np.float64]
DecisionArray: TypeAlias = NDArray[np.generic]

# A buffered archive merges the rows offered with ``add`` once they outnumber this, or twice its size.
PENDING_ROWS = 4096


class RowBuffer:
"""The rows that ``add`` keeps for a later merge, and their checks; mixed into ``_BaseArchive``."""

buffered: bool
capacity: int | None
covers_last_input: bool
unchanged_by_last_input: bool
last_capacity_pruned: int
_pending: list[tuple[DecisionArray, FloatArray, FloatArray | None]]
_pending_rows: int
_clean: bool
_size: int
_G: FloatArray | None
_dtype: np.dtype[Any]
_n_var: int
_n_obj: int

def update(
self,
population_X: DecisionArray,
population_F: NDArray[np.generic],
population_G: NDArray[np.generic] | None = None,
) -> tuple[DecisionArray, FloatArray]:
raise NotImplementedError

def _update_now(
self,
population_X: DecisionArray,
population_F: NDArray[np.generic],
population_G: NDArray[np.generic] | None = None,
) -> tuple[DecisionArray, FloatArray]:
raise NotImplementedError

def add(
self,
population_X: DecisionArray,
population_F: NDArray[np.generic],
population_G: NDArray[np.generic] | None = None,
) -> None:
"""Offer rows as ``update`` does, without computing the contents.

A ``buffered`` archive with clean contents (mutually non-dominated, finite, feasible when it holds
constraint values, deduplicated with a zero objective tolerance) keeps the rows whose feasible
objective vectors are finite, and merges them when ``contents``, ``constraint_values`` or
``update`` needs them, or when they outnumber ``max(PENDING_ROWS, 2 * size)``. One merge of many
rows gives the contents, in the order, of one ``update`` per call: a clean archive keeps the
feasible non-dominated rows in the order they were offered, and the first of equal ones. Other
rows, or rows offered to an archive that is not buffered or not clean, are merged at once.
"""
pop_X, pop_F, pop_G = self._checked_rows(population_X, population_F, population_G)
if self.buffered and pop_X.shape[0] > 0 and self._bufferable(pop_F, pop_G):
# Copies: the caller may write into its population after offering it.
self._pending.append((pop_X.copy(), pop_F.copy(), None if pop_G is None else pop_G.copy()))
self._pending_rows += pop_X.shape[0]
# As after an update of clean contents: every offered row will be archived or covered.
self.last_capacity_pruned = 0
self.covers_last_input = True
self.unchanged_by_last_input = False
if self._pending_rows > max(PENDING_ROWS, 2 * self._size):
self._flush()
return
self.update(pop_X, pop_F, pop_G)

def _bufferable(self, pop_F: FloatArray, pop_G: FloatArray | None) -> bool:
"""Whether ``add`` may keep these rows for a later merge (see ``add``)."""
if not self._clean or self._size == 0 or self.capacity is not None or (self._G is None) != (pop_G is None):
return False
if pop_G is None:
return bool(np.isfinite(pop_F).all())
assert self._G is not None
if pop_G.shape[1] != self._G.shape[1]:
return False
return bool(np.isfinite(pop_F[is_feasible(pop_G, n=pop_G.shape[0])]).all())

def _flush(self) -> None:
"""Merge the rows that ``add`` kept, as one update."""
if not self._pending:
return
pending = self._pending
self._pending = []
self._pending_rows = 0
X = np.concatenate([rows[0] for rows in pending])
F = np.concatenate([rows[1] for rows in pending])
Gs = [rows[2] for rows in pending if rows[2] is not None]
G = np.concatenate(Gs) if Gs else None
self._update_now(X, F, G)
# As after the calls that offered them: the merge covered every row.
self.covers_last_input = self._clean
self.unchanged_by_last_input = False

def _checked_rows(
self,
population_X: DecisionArray,
population_F: NDArray[np.generic],
population_G: NDArray[np.generic] | None,
) -> tuple[DecisionArray, FloatArray, FloatArray | None]:
pop_X = np.asarray(population_X, dtype=self._dtype, order="C")
pop_F = np.asarray(population_F, dtype=float, order="C")
if pop_X.ndim != 2:
raise ValueError("population_X must be a 2D array.")
if pop_F.ndim != 2:
raise ValueError("population_F must be a 2D array.")
if pop_X.shape[0] != pop_F.shape[0]:
raise ValueError("population_X and population_F must have the same number of rows.")
if pop_X.shape[1] != self._n_var:
raise ValueError(f"population_X has {pop_X.shape[1]} columns, expected {self._n_var}.")
if pop_F.shape[1] != self._n_obj:
raise ValueError(f"population_F has {pop_F.shape[1]} columns, expected {self._n_obj}.")
pop_G: FloatArray | None = None
if population_G is not None and pop_X.shape[0] > 0:
pop_G = np.asarray(population_G, dtype=float, order="C")
if pop_G.ndim != 2:
raise ValueError("population_G must be a 2D array when provided.")
if pop_G.shape[0] != pop_X.shape[0]:
raise ValueError("population_G must have the same number of rows as population_X.")
return pop_X, pop_F, pop_G


__all__ = ["PENDING_ROWS", "RowBuffer"]
40 changes: 19 additions & 21 deletions src/vamos/engine/algorithm/components/archive_core.py
Original file line number Diff line number Diff line change
Expand Up @@ -11,6 +11,7 @@
except ImportError: # pragma: no cover - optional dependency
_moocore = None

from vamos.engine.algorithm.components.archive_buffer import RowBuffer
from vamos.engine.algorithm.components.archive_insertion import INSERTION_ROWS, drop_rows, insertion
from vamos.foundation.constraints.utils import compute_violation, is_feasible
from vamos.foundation.quality_indicators.pareto import pareto_filter
Expand Down Expand Up @@ -129,7 +130,7 @@ def _subset_arrays(
return X[idx], F[idx], G[idx] if G is not None else None


class _BaseArchive:
class _BaseArchive(RowBuffer):
"""
Base class for bounded external archives.

Expand Down Expand Up @@ -218,39 +219,34 @@ def __init__(
self.unchanged_by_last_input = False
# The configuration of the external archive this archive holds (``setup_archive``), whose final subset the result reports.
self.external_config: ExternalArchiveConfig | None = None
#: Whether ``add`` keeps the rows it is offered and merges them only when needed (see ``add``).
self.buffered = False
self._pending: list[tuple[DecisionArray, FloatArray, FloatArray | None]] = []
self._pending_rows = 0

def update(
self,
population_X: DecisionArray,
population_F: NDArray[np.generic],
population_G: NDArray[np.generic] | None = None,
) -> tuple[DecisionArray, FloatArray]:
pop_X = np.asarray(population_X, dtype=self._dtype, order="C")
pop_F = np.asarray(population_F, dtype=float, order="C")
if pop_X.ndim != 2:
raise ValueError("population_X must be a 2D array.")
if pop_F.ndim != 2:
raise ValueError("population_F must be a 2D array.")
if pop_X.shape[0] != pop_F.shape[0]:
raise ValueError("population_X and population_F must have the same number of rows.")
if pop_X.shape[1] != self._n_var:
raise ValueError(f"population_X has {pop_X.shape[1]} columns, expected {self._n_var}.")
if pop_F.shape[1] != self._n_obj:
raise ValueError(f"population_F has {pop_F.shape[1]} columns, expected {self._n_obj}.")
"""Merge the rows into the archive and return its contents; rows kept by ``add`` are merged first."""
self._flush()
return self._update_now(population_X, population_F, population_G)

def _update_now(
self,
population_X: DecisionArray,
population_F: NDArray[np.generic],
population_G: NDArray[np.generic] | None = None,
) -> tuple[DecisionArray, FloatArray]:
pop_X, pop_F, pop_G = self._checked_rows(population_X, population_F, population_G)
self.last_capacity_pruned = 0
if pop_X.shape[0] == 0:
self.covers_last_input = self._clean
self.unchanged_by_last_input = True
return self._snapshot()

pop_G: FloatArray | None = None
if population_G is not None:
pop_G = np.asarray(population_G, dtype=float, order="C")
if pop_G.ndim != 2:
raise ValueError("population_G must be a 2D array when provided.")
if pop_G.shape[0] != pop_X.shape[0]:
raise ValueError("population_G must have the same number of rows as population_X.")

if self._insertable(pop_X.shape[0], pop_G) and self._insert(pop_X, pop_F, pop_G):
return self._snapshot()

Expand Down Expand Up @@ -361,10 +357,12 @@ def _insert(self, pop_X: DecisionArray, pop_F: FloatArray, pop_G: FloatArray | N
return True

def contents(self) -> tuple[DecisionArray, FloatArray]:
self._flush()
return self._snapshot()

def constraint_values(self) -> FloatArray | None:
"""Constraint values of the archived rows, aligned with ``contents()``, or ``None`` without constraints."""
self._flush()
if self._G is None:
return None
return self._G[: self._size].copy()
Expand Down
Loading
Loading