Repository navigation
perf(bootstrap): stop serializing the task list with every mirai batch - #27
Closed
seanthimons wants to merge 2 commits into
Closed
seanthimons wants to merge 2 commits into
seanthimons wants to merge 2 commits into
Conversation
The mirai worker was an anonymous function created inside start_mirai_bootstrap(), so its environment was that frame. mirai serializes a function together with its environment, so every batch job carried the full task list and dispatch cost grew with the total replicate count. Build the worker in an internal bootstrap_worker() helper whose environment is globalenv(). Its body uses only namespaced calls, so it needs no free variables. Source references are stripped so that a pkgload::load_all() worker does not ship the source file with each batch. The dev_path load behavior is unchanged. Replicate output for a given seed is unchanged.
Check that the worker's environment is globalenv() and that its serialized payload stays small, that it refits a batch exactly like the sequential path, and that start_mirai_bootstrap() passes the detached worker to mirai_map().
Owner
Author
|
Superseded by #28: the branch is renamed |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Problem
start_mirai_bootstrap()passed an anonymous worker tomirai::mirai_map(). That closure was created insidestart_mirai_bootstrap(), so its environment was that function's frame, which holds every task (inputs) andbatches. mirai serializes a function together with its environment, so each batch job carried the full task list. Dispatch cost therefore grew with the total replicate count (quadratic overall).This blocks the calling R process. A Shiny app that calls
bootstrap_dose_response_async()freezes during launch, and the defaultanalyze_dose_response(bootstrap_times = 10000L)with the mirai backend is affected too.Fix
bootstrap_worker()whose environment isglobalenv(). Its body uses only namespaced calls (pkgload::,dplyr::,singlehit:::), so it needs no free variables from the creating frame.utils::removeSource(). Underpkgload::load_all()the worker otherwise serialized the whole package source file (about 213 KB) with every batch. It now serializes to under 1 KB.dev_pathbranch is unchanged. The worker is not given the namespace as its environment, so dev-loaded workers still load the same checkout.Task generation (
generate_bootstrap_tasks(), about 1 ms per replicate) is not changed here.Measurements
bootstrap_dose_response_async(fit, times, seed = 42, workers = 2)on the Ward rotavirus data with abeta_poissonfit, using 2 local daemons. "Returns" is the wall time of the async call. Task generation is included in that time and dominates the after case. Collection time is refit compute and did not change beyond run-to-run noise.Task generation at 10,000 replicates takes 7.2 to 7.5 s, so after the fix the call returns in roughly task-generation time plus about 0.4 s of dispatch. Before the fix, dispatch took about 25 s of the 33 to 35 s. Before the fix, the call took 33 to 35 s on this machine, not the 47.8 s seen earlier.
Replicate output was compared with
identical()between the pre-fix and post-fix code for 1,000, 4,000 and 10,000 replicates at seed 42. All three matched.Regression test
tests/testthat/test-bootstrap-dispatch.R, which needs no daemons and no timing:globalenv(), and its serialized size is under 10 KB.run_sequential_bootstrap().start_mirai_bootstrap()passes the detached worker tomirai_map(). This uses a mock ofmirai_mapanddaemons_set. The test fails on the pre-fix code (2 failures) and passes with the fix.Verification
testthat::test_local(): 297 passed, 0 failed, 3 skipped. Two of the skips are mirai tests that need an installed namespace.R CMD buildandR CMD check --no-manual: 305 passed, 0 failed, 1 skipped (the slow exact-model oracle), including the two mirai-gated tests. Status is 1 NOTE:singlehit:::bootstrap_refitis flagged as a:::call to the package's own namespace. This is the same call that existed before this change.pkgload::load_all().Not in this PR
generate_bootstrap_tasks()could be made lazy as a follow-up. Task generation is now the dominant cost of the async call.local_mocked_bindings()needs testthat 3.1.7 or later, but DESCRIPTION declarestestthat (>= 3.0.0). I did not change DESCRIPTION.