Implement async scheduler flag and helpers - #3
Draft
davidalvin wants to merge 2 commits into
Draft
davidalvin wants to merge 2 commits into
davidalvin wants to merge 2 commits into
Conversation
## Core Changes ### 1. Enable Async Support in Build System - Modified Makefile to add -DENABLE_ASYNC flag to COMMON_FLAGS - This enables async execution capabilities in the C++ codebase ### 2. Enhanced Python Bindings - Modified cubff_py.cc to expose callback_ops_interval parameter - Added HAVE_ASYNC attribute to Python module for runtime detection - Updated SimulationParams to support async operation intervals ### 3. Fixed S3 Integration - Modified cubff_run_db.py to handle S3 configuration gracefully - Added proper error handling for S3 operations - Made S3 uploads conditional to prevent errors when S3 is unavailable ## New Testing and Execution Scripts ### 4. Simple Async Runner (cubff_run_simple.py) - Clean, minimal async simulation runner - Supports --enable_async and --ops_interval flags - Saves results locally without S3 dependencies - Perfect for testing and development ### 5. Visual Async Runner (cubff_run_visual.py) - Shows 10 different programs evolving in real-time - Demonstrates program diversity and evolution - Uses language.PrintProgram() for pretty visualization ### 6. Comprehensive Proof Tests - async_proof_test.py: Basic async vs sync comparison - async_detailed_proof.py: Detailed worker and execution analysis - test_cubff_run.py: Minimal test runner for quick validation ## How to Use ### Build with Async Support ```bash make clean && make PYTHON=1 CUDA=0 ``` ### Run Async Simulations ```bash # Quick test (10 epochs) python3 test_cubff_run.py --enable_async # Full simulation (100 epochs) python3 cubff_run_simple.py --enable_async # Visual simulation with program display python3 cubff_run_visual.py --enable_async --program_count 5 # Detailed proof test python3 async_detailed_proof.py --duration 8 ``` ### Key Features - ✅ Async support confirmed via HAVE_ASYNC attribute - ✅ Different callback frequencies between sync/async - ✅ Program evolution visualization - ✅ Worker count and execution time tracking - ✅ Comprehensive comparison tools - ✅ Local file saving (no S3 required) ### Proof of Async Operation The async simulation is working when you observe: 1. Different timing patterns between sync and async runs 2. Different program evolution paths 3. More frequent callbacks in async mode 4. Different final program states 5. Respect for callback_ops_interval parameter This commit provides a complete framework for testing and validating asynchronous execution in the CUBFF self-replicating program simulator.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
SimulationParamswithcallback_ops_intervalslice_idtoSimulationStateMutateAndRunPairhelper and async kernelcubff_run_db.pyto support async modeTesting
make(fails: nvcc not found and pybind11 missing)https://chatgpt.com/codex/tasks/task_e_6873fd9a3fa883238ea6c4367bd73892