Read the dose response per feature, and find the window it happens in - #97
Merged
Merged
Conversation
tl.dose_features gives every feature a benchmark dose, the lowest concentration at which its median response reaches three times the controls' MAD. It is defined where an EC50 is not: most compounds never plateau inside the tested range, so a four-parameter fit has no EC50 to report for them. tl.dose_direction reads the direction as well as the distance. A distance grows both when a phenotype gets louder and when a different one takes over, and it cannot tell them apart. In OASIS HepaRG, staurosporine's amplitude saturates at 0.41 uM while its direction keeps turning to 300 uM. All three now merge doses within dose_tolerance of each other first. The OASIS batches write the same nominal concentration to different precision, so a ten-point ladder read as eighteen: n_doses was wrong and the hit call's per-concentration product ran over twice as many, half as deep groups.
Most of a ten-point ladder is not worth reading: a screen covers three decades because it does not know where the compound's window is, so the low end sits below the effective range and the high end can be past the cells. Nothing so far said which stretch was which. dose_direction now labels each concentration: silent, responding while the step from the concentration below beats the noise, saturated once it stops, and cytotoxic once half the cells are gone. The label lands on obs, so the window is an ordinary subset and dose_features and the rest work on it unchanged. The window is a run rather than a scatter, because thresholding each concentration on its own let one lucky replicate open amperozide's window at 3.7 uM instead of 33. pl.dose_direction draws that reading with the ladder banded by phase. tl.dose_trajectory reads each compound's window onto a shared relative axis, so compounds are compared on what they do in their own effective range rather than at a concentration one of them has not reached yet. Actinomycin D's nearest neighbour there is cycloheximide.
|
Check out this pull request on See visual diffs & provide feedback on Jupyter Notebooks. Powered by ReviewNB |
Documentation build overview
18 files changed ·
|
Codecov Report❌ Patch coverage is
Additional details and impacted files@@ Coverage Diff @@
## main #97 +/- ##
==========================================
+ Coverage 89.05% 89.32% +0.27%
==========================================
Files 80 80
Lines 5884 6176 +292
==========================================
+ Hits 5240 5517 +277
- Misses 644 659 +15
🚀 New features to boost your workflow:
|
Viability was read against the whole screen's controls. The OASIS pilot's eight plates carry 484 to 708 control cells per field, so a sparse plate read as one whose treated wells were dying: actinomycin D was called cytotoxic above 60 uM when only its 300 uM well is. It is now read against each plate's own controls. The two HepaRG batches still disagree about cell loss after that, and the tutorial now shows the check rather than the conclusion. dose_direction also broadcasts the dose it binned each well to, so wells can be grouped the way its table reports them. Grouping on the raw column alternates between batches at every step of the ladder. The tutorial gains the biology: what the cited CellProfiler features measure, what each compound does, and a figure of staurosporine's early and late features against dose. Its distance saturates at 0.4 uM where the cells round up and chromatin moves outward, then at 300 uM every stain begins to correlate with every other inside the nucleus, which is a cell losing its compartments. From the cleanup review: a shared control-scale helper, a nanmedian that skips numpy's masked-array path on short blocks, one row-index pass per group, and a memo that no longer recomputes what it memoizes. dose_direction goes from 55s to 4s on a 500-compound screen.
#98 added Metadata_ConcentrationNominal, which merges levels within 1% across the whole plate map. That catches 3.704 against 3.7037 but not 0.015 against 0.0152, and not 0.000762 against 0.001, where one batch rounds to three decimals and a small dose loses a third of its value. Widening the tolerance is not available: berberine's 25 uM and the main ladder's 33.3 uM are a third apart and are different doses. The plate maps disagree on precision, not on value, so a level written to fewer decimals is the level that rounds to it. Matching runs within a compound, whose levels are one dilution series and cannot collide. Every dosed compound then lands on the ten three-fold steps it was plated on, or the eight two-fold ones the assay-development plates used. That belongs to the loader, so dose_tolerance and the binning behind it are gone from the four dose functions, along with the binned dose dose_direction used to broadcast. Every dose call in the tutorial reads the nominal column. The amplitude floor now holds each drawn control group out of the scale it is measured in, as hit_calling's own cutoff does after #98. Scored against a baseline they helped define, control groups sit closer to it than a treated group can: 64% low at sixteen control wells, 30% at thirty-two, 4% at the OASIS pilot's 256. Tutorial 11 re-executed. Cycloheximide is now called in both cell lines, so the cross-line paragraph is #98's corrected one.
Metadata_Concentration is now the dose a well was meant to get and Metadata_ConcentrationRecorded keeps what the plate map wrote. The corrected value was reaching nobody: every dose function defaulted to the recorded column, so the tutorial had to name the other one eight times to get a ten-point ladder. The amplitude floor refitted its scale around each draw, which is right but repeats for every distinct plate layout, and the number of layouts is combinatorial in the plates rather than bounded by the compounds. One split now serves the whole run, halved within each plate so the fitted half cannot be one plate's, and the floor still holds its draw out of the scale it is measured in. From the cleanup review: one viability in tl/_design, taking the level to normalize within, so cytotoxicity and dose_direction stop meaning two things by one column name; _Scale carries the control scale instead of four loose arrays; group_rows joins group_offsets in _core; the phase vocabulary and its colours are one mapping and unexported, as the cell-cycle pair already was; and the plot tests share the fixture they had copied.
timtreis
force-pushed
the
feat/dose-features
branch
from
September 20, 2026 21:46
6a2409a to
c523704
Compare
Docstrings and comments in _effect, _aggregate and _batch quoted measured times and speedups. The reason a path was chosen belongs there; a number from one machine on one dataset does not.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
tl.dose_responsegrades one number per well against the dose. That says a compound did something, not what it did, and not where on the ladder it did it.tl.dose_featuresgives every feature a benchmark dose, the lowest concentration reaching three times the controls' MAD. It is defined where an EC50 is not, since most compounds never plateau in range. Mupirocin crosses on none of 99 features, which is the calibration.tl.dose_directionreads the direction as well as the distance, and labels each concentrationsilent,responding,saturatedorcytotoxic. The label lands onobs, so the window is an ordinary subset thatdose_featuresand the rest work on. Staurosporine's distance saturates at 0.41 uM while its direction keeps turning to 300 uM: shape and chromatin first, then every stain correlating with every other, which is a cell losing its compartments.pl.dose_directiondraws one compound's ladder banded by phase.tl.dose_trajectoryreads each window onto a shared relative axis, so compounds are compared in their own effective range rather than at a concentration one of them has not reached. Actinomycin D's nearest neighbour is cycloheximide.Two fixes to existing behaviour:
dose_directionis read against each plate's own controls. Pooled, the pilot's 484-to-708 control cells per field made sparse plates look like dying ones, and called actinomycin D cytotoxic four concentrations below where it is.oasis_pilotaligns a dose written to fewer decimals with the level that rounds to it, within each compound. Read the control scale off the controls that did not fit the transform #98 merged levels within 1% across the whole plate map, which leaves0.015and0.0152apart; a wider tolerance cannot work, since berberine's 25 uM and the main ladder's 33.3 uM are a third apart and different doses. Every dosed compound now lands on the ten three-fold steps it was plated on.Tutorial 11 gains three sections and three figures, and is re-executed. The two points of departure it computes, per-feature crossings and window onset, rank together at 0.79 with 17 of 20 compounds within one three-fold step.
Verified with a regression test for each behaviour, the full suite on 3.12 and 3.14 including the network tests, mypy, prek and the docs build with
-W.