Read the control scale off the controls that did not fit the transform - #98
Merged
Merged
Conversation
hit_calling fits the centroid and covariance on half the controls, then writes a distance for every row. The fitting half sits closer to the centroid it placed, so dose_response's 3 * MAD cutoff came out low and every hit call was permissive. The held-out half is now recorded in obs so anything reading a scale off the controls can read it off the honest rows. Closes #83
distance_key defaulted to hits_distance, the group statistic broadcast over its rows, so the median cytotoxicity takes returned the value it was handed and no genuinely per-row response could be given. It now defaults to hits_row_distance and compares against the held-out controls. The DMSO distance in tutorial 8's prose was already stale against main. Closes #84
|
Check out this pull request on See visual diffs & provide feedback on Jupyter Notebooks. Powered by ReviewNB |
Documentation build overview
10 files changed ·
|
Codecov Report❌ Patch coverage is
Additional details and impacted files@@ Coverage Diff @@
## main #98 +/- ##
==========================================
- Coverage 89.15% 89.05% -0.11%
==========================================
Files 80 80
Lines 5865 5884 +19
==========================================
+ Hits 5229 5240 +11
- Misses 636 644 +8
🚀 New features to boost your workflow:
|
The response is a distance from the controls, with a long right tail, so one stray well stretched the linear axis until every other point sat flat on the baseline. Staurosporine's wells climb from 5 to 18 over the dose range in tutorial 11 and none of it was visible. A response that reaches zero or below is still drawn on a linear scale.
Two of the twelve plate maps record the dose to three decimals where the rest use four, so one concentration arrives as both 3.704 and 3.7037. Metadata_ Perturbation was built from the raw column, so the wells of a single treatment split across the two spellings and most groups held too few wells to be replicates of anything. Metadata_Concentration still holds what each plate map recorded. The new Metadata_ConcentrationRounded puts levels agreeing to within 1% onto the value the most wells carry, so every dose it holds is one that was written down, and it names the perturbation groups. No rounding precision does this: none separates the pairs that differ from those that do not.
The column is produced by clustering levels within a tolerance, not by rounding, so Metadata_ConcentrationNominal says what it holds. Two places still read the raw column and should not have: the loader's summary log counted the split levels it exists to correct, and the tutorial drew wells against raw doses beside a curve fitted on aligned ones. _aligned_doses splits the sorted levels at the gaps rather than walking them with an index, and maps through Series.replace as the cell-line fixup beside it does.
timtreis
added a commit
that referenced
this pull request
Sep 20, 2026
#98 added Metadata_ConcentrationNominal, which merges levels within 1% across the whole plate map. That catches 3.704 against 3.7037 but not 0.015 against 0.0152, and not 0.000762 against 0.001, where one batch rounds to three decimals and a small dose loses a third of its value. Widening the tolerance is not available: berberine's 25 uM and the main ladder's 33.3 uM are a third apart and are different doses. The plate maps disagree on precision, not on value, so a level written to fewer decimals is the level that rounds to it. Matching runs within a compound, whose levels are one dilution series and cannot collide. Every dosed compound then lands on the ten three-fold steps it was plated on, or the eight two-fold ones the assay-development plates used. That belongs to the loader, so dose_tolerance and the binning behind it are gone from the four dose functions, along with the binned dose dose_direction used to broadcast. Every dose call in the tutorial reads the nominal column. The amplitude floor now holds each drawn control group out of the scale it is measured in, as hit_calling's own cutoff does after #98. Scored against a baseline they helped define, control groups sit closer to it than a treated group can: 64% low at sixteen control wells, 30% at thirty-two, 4% at the OASIS pilot's 256. Tutorial 11 re-executed. Cycloheximide is now called in both cell lines, so the cross-line paragraph is #98's corrected one.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Both issues are one confusion: two
obscolumns claim to be "distance from the controls", and the one everything read was not honest.hit_callingfits the centroid and covariance on half the controls, then writeshits_row_distancefor every row. The fitting half sits closer to the centroid it placed, sodose_response's3 * MADcutoff came out low and everyhitcallwas permissive (#83). Separately,cytotoxicitydefaulteddistance_keytohits_distance, the group statistic broadcast over its rows, so the median it documents returned the value it was handed (#84).hit_callingnow writesobs[key_added + "_reference_held_out"], the mask it already computed and discarded.held_out_referencein_core/masks.pynarrows a control mask with it, reading the flag through_flag_masklike the other two flag columns, and falls back to every control when the column is absent.dose_responseandcytotoxicityboth use it;cytotoxicitynow defaults tohits_row_distance.Two things found while re-running the tutorials, both kept as separate commits:
pl.dose_responsedrew a heavy-tailed distance on a linear axis, so one stray well flattened every other point onto the baseline. It is drawn on a log scale when the response is positive.3.704and3.7037, because two of its twelve plate maps use three decimals and the rest four.Metadata_Perturbationwas built from that column, so a treatment's replicates split across the two spellings.Metadata_Concentrationstill holds what was recorded; the newMetadata_ConcentrationNominalputs levels agreeing to within 1% onto the value the most wells carry, and names the groups.This changes published
hitcallnumbers,cytotoxicitydistances and the OASIS replicate groups, so tutorials 5, 8 and 11 and theoasis_pilotpage are re-run. Tutorial 11's prose is corrected where it argued from numbers that moved.Verified: full suite green, ruff, mypy, pre-commit and a
-Wdocs build.Closes #83
Closes #84