Make containment-based selection first-class - #1
Open
DanielSprockett wants to merge 10 commits into
Open
DanielSprockett wants to merge 10 commits into
DanielSprockett wants to merge 10 commits into
Conversation
- pyproject.toml: installable package with entry point refrover.cli:main - src/refrover/selectors/: BaseSelector ABC, RandomSelector, MaxMinSelector, KMedoidsSelector (pure-numpy PAM), ArchetypeSelector (archetypes package) - src/refrover/io.py: manifest read/validate - tests/: 23 passing unit tests covering correctness, diversity, and edge cases - .gitignore: excludes data/, *.sig, *.zip, build artifacts
- cli.py: Click entry point with sketch/select/align/coverage/format/run/benchmark subcommands - sketch.py: sourmash sketch + compare wrappers - similarity.py: load/filter Jaccard similarity matrices; direct Python API computation - align.py: BWA-MEM2, BWA, minimap2 wrappers; handles paired/single/long/hybrid reads - coverage.py: CoverM wrapper for per-contig depth - pipeline.py: RefRoverPipeline orchestrator - benchmark.py: stub (requires CheckM2) - selectors/greedy_var.py: greedy coverage-variance maximization selector - selectors/containment.py: read-containment-based selector with optional species weighting - selectors/feedback.py: stub (requires alignment loop) - formatters/: MetaBAT2, SemiBin2, MaxBin2, CONCOCT, generic TSV with registry dispatch - 61 tests passing; added coverage_df and tiny_manifest fixtures
…rve) and --k auto CLI flag
…y weight computation
compare_selectors.py evaluates four prototype selection strategies on the 183-sample containment matrix using two metrics: - Naive: sum of selected column variances (biased toward correlated selectors) - Unique: orthogonal projection of the full 182-column matrix onto the selected set (removes double-counting of correlated assemblies) explore_selectors.R generates five diagnostic plots: variance-explained curves (both metrics side-by-side), per-sample distribution at k=5, selector agreement heatmap, adaptive-k distribution, and per-sample saturation curves. Key algorithm: the saturation curve uses greedy forward selection by residual variance (Gram-Schmidt incremental QR) rather than MaxMin order, which ensures a monotonically decreasing gains sequence necessary for threshold-based elbow detection.
The containment-based selection validated on the rewilded-mouse dataset was implemented but unreachable: ContainmentSelector used a bespoke interface, was absent from SELECTOR_REGISTRY, and had no CLI or pipeline path. This makes it a first-class selector. Interface unification - ContainmentSelector now subclasses BaseSelector and returns prototype sample_ids, the same contract every selector follows (a prototype is the sample whose assembly it is; align resolves sample_id -> FASTA). This removes the implicit sample==assembly coupling in favour of the explicit manifest mapping that align already uses. - Accepts min_jaccard= as an alias for min_containment so the registry can construct it uniformly. Gains BaseSelector's k/threshold validation. Performance - Precompute the candidate correlation-distance matrix once instead of calling np.corrcoef pairwise inside the greedy loop. ~4 ms/sample on the 183-sample matrix (was O(k*n) corrcoef calls per query). Wiring - Registered as "containment" in SELECTOR_REGISTRY; added CONTAINMENT_SELECTORS so the CLI/pipeline know to feed it a containment matrix. - refrover select / refrover run accept --containment-matrix; error clearly when the required matrix (or --sketches for Jaccard selectors) is missing. - RefRoverPipeline accepts containment_matrix= and uses it for selection, skipping the sketch+compare step. - load_containment_matrix() added to similarity.py. Tests: 9 new (registry membership, BaseSelector conformance, registry construction, CLI select happy path + both missing-input errors). 78 pass. Descoped from this branch: the MetaBAT2 variance fix (it is a coverage-schema change touching load_coverage and all five formatters, not the few lines first estimated) — to follow as its own change.
The MetaBAT2 formatter wrote every per-sample variance column as 0 because
CoverM was only ever asked for mean depth, and the internal coverage schema had
no place to carry variance. MetaBAT2's jgi format uses that column, so binning
silently lost differential signal.
Coverage schema now carries variance:
length, {sample}_depth (mean), {sample}_var (variance)
- run_coverm requests `--methods length mean variance` (configurable).
- load_coverage normalizes CoverM's `{sample} {Method}` columns into the schema
via normalize_coverage: ` Mean` -> _depth, ` Variance` -> _var, ` Length` ->
a single collapsed `length`. Already-normalized frames pass through unchanged.
- MetaBAT2 formatter emits the real paired variance, falling back to 0.0 only
when a sample has no variance column (mean-only CoverM runs).
- generic / semibin2 / maxbin2 / concoct select mean columns by suffix via a
shared formatters/_columns helper, so variance columns never leak into the
mean-only formats. MaxBin2 .abund files are now named by clean sample id.
- The standalone `refrover format` command routes through load_coverage instead
of a raw pd.read_csv, so it normalizes raw CoverM tables too.
Tests: new test_coverage.py locks the CoverM-parsing behaviour (which can't be
exercised without CoverM installed); test_metabat2 now asserts real variance
passes through and that absent variance falls back to zero. 85 pass.
The cross-sample containment matrix that ContainmentSelector depends on was only produced by an out-of-tree script (data/mouse_rewilding/compute_containment.py). This makes it a first-class package stage so containment selection is self-sufficient. - refrover/containment.py: compute_containment_matrix(read_sigs, assembly_sigs) computes containment(reads_i, assembly_j) via sourmash MinHash.contained_by, returning a sample x sample DataFrame (query_id index) ready for ContainmentSelector and load_containment_matrix. Plus sigs_by_sample() and write_containment_matrix() helpers. - refrover containment CLI subcommand: takes --read-sketches / --assembly-sketches directories, writes containment_matrix.tsv. Idempotent (skips existing unless --force). Uses the sourmash API rather than the analysis script's raw-JSON hash parsing — slower but correct across sketch parameters and maintainable in-tree. Tests: new test_containment.py builds real sourmash sketches in a tmp dir and checks exact (asymmetric) containment values, labelling, TSV round-trip through load_containment_matrix, and that the output feeds ContainmentSelector. 90 pass.
…electors CLI Turns benchmark.py from a stub into a working alignment-free selector evaluator, operationalising the data/mouse_rewilding selector-comparison analysis with the real package selectors. - unique_variance_explained(matrix, selected_ids): fraction of total cross-sample column variance the selected columns explain, via orthogonal (QR) projection of the full matrix onto the selected subspace. Correlated/redundant selections count once, so it rewards prototypes spanning independent axes of variation. - rank_selectors(matrix, selectors, k_values, ...): feeds the same matrix (Jaccard or containment) to every selector for an apples-to-apples ranking. Skips unavailable selectors (feedback; archetype without its package) with a warning instead of aborting; seeds the random baseline for reproducibility. - refrover rank-selectors CLI: takes --sketches or --containment-matrix, writes selector_ranking.tsv and prints the table. - run_benchmark (Tier 2, CheckM2 MAG yield) documented as the gold standard the Tier-1 proxy is meant to predict; still NotImplementedError. On the real 183-sample containment matrix the ranking reproduces the analysis: maxmin and containment lead, greedy_var trails, variance rises with k. Tests: test_benchmark.py covers the projection metric (all-columns=1.0, empty=0, redundancy adds little, nested monotonicity) and ranking (columns, k-monotonicity for nested maxmin, feedback skipped, reproducibility, containment on shared matrix). 100 pass.
The three existing estimators (scree_elbow, similarity_gap, saturation_curve) all run on a Jaccard matrix as a proxy. This adds the method validated on the rewilded-mouse data: greedy forward selection by residual cross-sample variance (incremental Gram-Schmidt), which gives a monotonically decreasing marginal-gain curve, with the elbow at the first k whose gain drops below 10% of the first. - _containment_saturation in adaptive_k.py; wired into estimate_k and the AdaptiveMethod literal. - refrover select --adaptive-k-method gains the containment_saturation choice (use with a containment matrix for the intended behaviour). On the real 183-sample containment matrix it reproduces the analysis exactly: median k=4 (182 samples at 4, one at 5), matching compare_selectors.py. Tests: containment_saturation added to the all-methods validity check, plus a low-dimensional-data test and a dominant-axis saturation test. 102 pass.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Makes the containment-based prototype selection validated on the rewilded-mouse dataset reachable end-to-end through shipped commands. Previously
ContainmentSelectorwas implemented but unreachable (bespoke interface, absent from the registry, no CLI/pipeline path), and the containment matrix it needs was only produced by an out-of-tree analysis script.Changes
ContainmentSelectoris now aBaseSelectorsubclass returning prototypesample_ids (the same contract every selector follows;alignresolvessample_id → FASTA). Registered ascontainment; constructible uniformly fromSELECTOR_REGISTRY. Correlation-distance matrix is precomputed once instead of pairwisenp.corrcoefin the greedy loop (~4 ms/sample on 183 samples).refrover.containment+refrover containmentCLI. Computes the reads-vs-assemblies containment matrix in-package viasourmash MinHash.contained_by, so containment selection is self-sufficient.{sample}_var;run_covermrequestslength mean variance;load_coveragenormalizes CoverM's{sample} {Method}columns. MetaBAT2 emits real per-sample variance (was hardcoded0, which silently degraded binning). Other formatters select mean columns by suffix; theformatCLI routes throughload_coverage.benchmark.rank_selectors+refrover rank-selectorsrank selectors in seconds with no alignment, by variance explained via orthogonal projection (correlated selections don't double-count). Reproduces the analysis on real data (MaxMin/containment lead,greedy_vartrails).containment_saturationadaptive-k. Greedy residual-variance saturation (the validated method) ported intoadaptive_k.pyand exposed via--adaptive-k-method. Reproduces the analysis exactly: median k=4 across 183 samples.End-to-end
Tests
33 → 102 passing; ruff clean. New:
test_cli.py,test_coverage.py,test_containment.py,test_benchmark.py, plus additions to selector and adaptive-k tests. CoverM parsing is locked by unit tests since CoverM isn't available in the test env.Notes
opus_suggestions.md(a code-review document) is included for reference.