Repository navigation
perf(bootstrap): draw all replicate samples in one vectorized call - #29
Merged
Merged
Conversation
generate_bootstrap_tasks() built one tibble per replicate (about 0.7 ms each), which took 7.8 s for 10,000 replicates and blocked the calling R process; in a Shiny app that freezes every session while a run starts. Draw every replicate in a single rbinom() call and build each sample with tibble::new_tibble(). rbinom() consumes the random stream in order, so the output is identical to the old per-replicate draws for any seed, and existing seeds keep producing the same replicates. Add tests that compare against the old algorithm exactly, for observed and fitted probabilities, all three models, and a single replicate.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Problem
generate_bootstrap_tasks()built onetibble::tibble()per replicate (about 0.7 ms each). For 10,000 replicates that is 7.8 s, and it runs on the calling R process before any worker starts. A Shiny app that starts a bootstrap freezes every session while this happens.Change
Draw all replicates in a single
stats::rbinom()call (a groups x replicates matrix) and build each sample withtibble::new_tibble().rbinom()consumes the random stream in order, so the draws are identical to taking them one replicate at a time. Existing seeds keep producing exactly the same replicates; the output of the new function isidentical()to the old one.Measurements
Ward data,
beta_poissonfit, this machine.Task generation alone:
End to end,
exponential+beta_poisson, 8 local workers (current dispatch):The "before" columns already include the separate dispatch fix in #28; this change is independent of it and touches a different function in the same file.
Tests
tests/testthat/test-bootstrap-tasks.Rkeeps the old algorithm as a reference and checks that the new output is identical to it:Full suite: 463 passed, 0 failed, 3 skipped.
Not in this PR
The mirai dispatch fix (#28), any app change, and version or NEWS edits.