[data] Move non-data related stuff oustide release test timing - #65268
[data] Move non-data related stuff oustide release test timing#65268iamjustinhsu wants to merge 2 commits into
Conversation
Signed-off-by: iamjustinhsu <jhsu@anyscale.com>
![gemini-code-assist[bot]](https://avatars.githubusercontent.com/in/956858?s=60&v=4){kind=small}
There was a problem hiding this comment.
Code Review
This pull request refactors several nightly dataset benchmark scripts to ensure that only the core workload execution is timed within the benchmark runner (benchmark.run_fn). Setup steps (such as model loading, token fetching, configuration building, and datasource creation) and teardown/analysis steps (such as metrics collection, profiling delays, memory logging, and result writing) are moved outside of the timed benchmark functions. This improves the accuracy of the benchmark runtimes. I have no feedback to provide as there are no review comments.
rayhhome
left a comment
rayhhome
left a comment
There was a problem hiding this comment.
LGTM! Let's run the release tests before merging to ensure correctness
| # `default_map_logical_memory_enabled` is a best practice that's required for | ||
| # Ray Data to prevent OOMs. It's not enabled by default in Ray 2.56, but we | ||
| # intend to enable it by default in a future release. |
There was a problem hiding this comment.
It seems this comment was dropped accidentally; default_map_logical_memory_enabled still defaults to False for now.
Signed-off-by: iamjustinhsu <jhsu@anyscale.com>
2 participants
Description
As titled, i noticed that sometimes we have extra non-data related stuff in the
benchmark_fnwhich can lead to variance in runtime outside of the ray data. I want to eliminate thatRelated issues
Additional information