Skip to content

Commit 0159b10

Browse files
StormKoperStormKoper
authored
Merge pull request #15 from codegithubka/kimon
Phase 1 complete
2 parents e40bf12 + 3e60fc5 commit 0159b10

24 files changed

Lines changed: 2373 additions & 132 deletions

‎.rsync-exclude‎

Lines changed: 15 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,15 @@
1+
# .rsync-exclude
2+
.venv/
3+
__pycache__/
4+
*.pyc
5+
.git/
6+
*.nbc
7+
*.nbi
8+
.pytest_cache/
9+
results/
10+
*.png
11+
*.pdf
12+
.DS_Store
13+
.ipynb_checkpoints/
14+
15+

‎docs/HPC_GUIDE.md‎

Lines changed: 4 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -6,7 +6,8 @@ ssh kanagnostopoul@snellius.surf.nl
66
# On a separate terminal run the following
77
88
# Upload the entire project directory (including your models/ folder)
9-
scp -r ~/CSS_Project kanagnostopoul@snellius.surf.nl:~/
9+
rsync -avz --progress --exclude-from='.rsync-exclude' \
10+
~/CSS_Project/ kanagnostopoul@snellius.surf.nl:~/CSS_Project/
1011
1112
# On the Snellius terminal
1213
@@ -41,11 +42,12 @@ tail -f logs_<JOBID>.err
4142
4243
# Watch task completetion
4344
45+
watch -n 5 squeue -u $USER
4446
watch -n 10 "ls -1 results_JOBID | wc -l"
4547
4648
# Fetching the results once the job is done
4749
48-
scp -r kanagnostopoul@snellius.surf.nl:~/results_18532145 ~/Downloads/
50+
scp -r kanagnostopoul@snellius.surf.nl:~/CSS_Project/results/phase1_ 18682575/ ./results/
4951
```
5052

5153
The jobscript template can be found in ```run_analysis.sh``` (default rome paritition).

‎docs/kimon_updates.md‎

Lines changed: 1 addition & 33 deletions
Original file line numberDiff line numberDiff line change
@@ -275,28 +275,6 @@ We have 48 tests cases validating the folloiwing:
275275
- Edge cases with extreme parameter values
276276

277277

278-
## Issues to be resolved
279-
280-
1. Grid size for HPC run
281-
2. Number of replicates for statistal power
282-
3. Directed and/or undirected runs
283-
4. Evolving and non-evolving runs?
284-
5. Mean field baseline or non evolving basiline
285-
6. Warmup period and measurement steps (i.e how many steps do we need to avoid init bias?)
286-
7. Measurement frequency for statistical accuracy
287-
8. Default parameters (Need Storm's input on this one).
288-
289-
Options:
290-
291-
1. Asymmetric repliates for non-evolving runs
292-
2. Coarse initial parameter sweep grid
293-
3. Discard non-evo runs and use mean field baseline instead or the opposite
294-
295-
NOTE:
296-
297-
Warmup and Measurement time must be scaled according to grid size. We need to write a method to estimate the approximate simulaiton time required. (Sary is working on that.)
298-
299-
300278
## Updates (25/1)
301279

302280

@@ -306,14 +284,4 @@ Refactored ```experiments.py``` according to ```experiments.md```. Phase 4 and P
306284

307285
- Phase 6: Will implement if all the other phases pass successfully.
308286

309-
TODO: We need to start drafting a proper version of the analyis (```analysis.py```) for postprocessing of all the data we are currenlty collecting and plotting utilities to visualize them.
310-
311-
312-
313-
314-
315-
316-
317-
318-
319-
287+
TODO: We need to start drafting a proper version of the analyis (```analysis.py```) for postprocessing of all the data we are currenlty collecting and plotting utilities to visualize them.

hpc/run_analysis.sh renamed to hpc/run_phase1.sh

Lines changed: 37 additions & 28 deletions
Original file line numberDiff line numberDiff line change
@@ -1,37 +1,35 @@
11
#!/bin/bash
2-
#SBATCH --job-name=pp_evo
3-
#SBATCH --partition=genoa
2+
#SBATCH --job-name=pp_phase1
3+
#SBATCH --partition=rome
44
#SBATCH --nodes=1
55
#SBATCH --ntasks=1
66
#SBATCH --cpus-per-task=32
7-
#SBATCH --time=08:00:00
8-
#SBATCH --mem=32G
9-
#SBATCH --output=pp_analysis_%j.out
10-
#SBATCH --error=pp_analysis_%j.err
7+
#SBATCH --time=04:00:00
8+
#SBATCH --mem=16G
9+
#SBATCH --output=pp_phase1_%j.out
10+
#SBATCH --error=pp_phase1_%j.err
1111

1212
# =============================================================================
13-
# PP Evolutionary Analysis - Snellius Job Script
13+
# PP Hydra Effect - Phase 1: Parameter Sweep
1414
# =============================================================================
1515
#
16-
# ESTIMATED RESOURCES:
17-
# - Runtime: ~1.5-2 hours on 32 cores (15×15 grid, 25 reps)
16+
# PHASE 1: Find critical point via 2D sweep of prey_birth × prey_death
17+
# - 15 × 15 × 15 reps × 2 (evo/non-evo) = 6,750 simulations
18+
# - Estimated runtime: ~2 min on 32 cores
1819
# - Memory: ~8 GB peak
19-
# - CPU hours: ~50-60 core-hours
2020
#
21-
# SUBMIT: sbatch run_analysis.sh
21+
# SUBMIT: sbatch run_phase1.sh
2222
# MONITOR: squeue -u $USER
2323
# CANCEL: scancel <job_id>
24-
# OUTPUT: results/ directory
2524
#
2625
# =============================================================================
2726

2827
echo "========================================"
29-
echo "PP Evolutionary Analysis"
28+
echo "PP Hydra Effect - Phase 1"
3029
echo "========================================"
3130
echo "Job ID: $SLURM_JOB_ID"
3231
echo "Node: $(hostname)"
3332
echo "CPUs: $SLURM_CPUS_PER_TASK"
34-
echo "Memory: $SLURM_MEM_PER_NODE"
3533
echo "Start: $(date)"
3634
echo "Working dir: $(pwd)"
3735
echo "========================================"
@@ -40,49 +38,60 @@ echo "========================================"
4038
# Environment Setup
4139
# -----------------------------------------------------------------------------
4240

43-
# Load modules (adjust to your Snellius setup)
4441
source ~/snellius_venv/bin/activate
4542

46-
# If you have a virtual environment with your models package:
47-
# source ~/venvs/ca_analysis/bin/activate
48-
49-
# Prevent numpy/scipy from spawning extra threads (we use joblib instead)
43+
# Prevent numpy/scipy from spawning extra threads (joblib handles parallelism)
5044
export OMP_NUM_THREADS=1
5145
export OPENBLAS_NUM_THREADS=1
5246
export MKL_NUM_THREADS=1
5347
export NUMEXPR_NUM_THREADS=1
5448

5549
# -----------------------------------------------------------------------------
56-
# Run Analysis
50+
# Run Phase 1
5751
# -----------------------------------------------------------------------------
5852

59-
OUTPUT_DIR="results_${SLURM_JOB_ID}"
53+
OUTPUT_DIR="results/phase1_${SLURM_JOB_ID}"
6054
mkdir -p $OUTPUT_DIR
6155

6256
echo ""
6357
echo "Output directory: $OUTPUT_DIR"
6458
echo ""
6559

66-
# Run the full analysis
67-
python3 -u scripts/pp_analysis.py \
68-
--mode full \
60+
# Dry run first to verify setup
61+
echo "Dry run check:"
62+
python3 -u scripts/experiments.py \
63+
--phase 1 \
6964
--output $OUTPUT_DIR \
70-
--cores $SLURM_CPUS_PER_TASK
65+
--cores $SLURM_CPUS_PER_TASK \
66+
--dry-run
67+
68+
echo ""
69+
echo "Starting Phase 1..."
70+
echo ""
7171

72-
# For asynchronous execution (uncomment if needed)
73-
#python3 pp_analysis.py --mode full --output $OUTPUT_DIR --cores $SLURM_CPUS_PER_TASK --async
72+
# Run phase 1
73+
python3 -u scripts/experiments.py \
74+
--phase 1 \
75+
--output $OUTPUT_DIR \
76+
--cores $SLURM_CPUS_PER_TASK
7477

7578
# -----------------------------------------------------------------------------
7679
# Completion
7780
# -----------------------------------------------------------------------------
7881

7982
echo ""
8083
echo "========================================"
81-
echo "Analysis Complete"
84+
echo "Phase 1 Complete"
8285
echo "========================================"
8386
echo "End time: $(date)"
8487
echo "Results in: $OUTPUT_DIR/"
8588
echo ""
8689
echo "Output files:"
8790
ls -lh $OUTPUT_DIR/
91+
echo ""
92+
echo "Next steps:"
93+
echo " 1. Download phase1_results.jsonl"
94+
echo " 2. Run analysis.py to find critical point"
95+
echo " 3. Update config.py with critical_prey_birth/death"
96+
echo " 4. Run phase 2-5"
8897
echo "========================================"

‎hpc/run_phase2.sh‎

Lines changed: 97 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,97 @@
1+
#!/bin/bash
2+
#SBATCH --job-name=pp_phase2
3+
#SBATCH --partition=rome
4+
#SBATCH --nodes=1
5+
#SBATCH --ntasks=1
6+
#SBATCH --cpus-per-task=32
7+
#SBATCH --time=04:00:00
8+
#SBATCH --mem=16G
9+
#SBATCH --output=pp_phase2_%j.out
10+
#SBATCH --error=pp_phase2_%j.err
11+
12+
# =============================================================================
13+
# PP Hydra Effect - Phase 2: Self-Organization (SOC Test)
14+
# =============================================================================
15+
#
16+
# PHASE 2: Test if prey_death evolves toward critical point
17+
# - 6 initial prey_death values × 30 reps = 180 simulations
18+
# - Longer runs (5000 steps) for evolution to equilibrate
19+
# - Tracks evolved_prey_death_timeseries
20+
#
21+
# SUBMIT: sbatch run_phase2.sh
22+
# MONITOR: squeue -u $USER
23+
# CANCEL: scancel <job_id>
24+
#
25+
# =============================================================================
26+
27+
echo "========================================"
28+
echo "PP Hydra Effect - Phase 2"
29+
echo "========================================"
30+
echo "Job ID: $SLURM_JOB_ID"
31+
echo "Node: $(hostname)"
32+
echo "CPUs: $SLURM_CPUS_PER_TASK"
33+
echo "Start: $(date)"
34+
echo "Working dir: $(pwd)"
35+
echo "========================================"
36+
37+
# -----------------------------------------------------------------------------
38+
# Environment Setup
39+
# -----------------------------------------------------------------------------
40+
41+
source ~/snellius_venv/bin/activate
42+
43+
# Prevent numpy/scipy from spawning extra threads (joblib handles parallelism)
44+
export OMP_NUM_THREADS=1
45+
export OPENBLAS_NUM_THREADS=1
46+
export MKL_NUM_THREADS=1
47+
export NUMEXPR_NUM_THREADS=1
48+
49+
# -----------------------------------------------------------------------------
50+
# Run Phase 2
51+
# -----------------------------------------------------------------------------
52+
53+
OUTPUT_DIR="results/phase2_${SLURM_JOB_ID}"
54+
mkdir -p $OUTPUT_DIR
55+
56+
echo ""
57+
echo "Output directory: $OUTPUT_DIR"
58+
echo ""
59+
60+
# Dry run first to verify setup
61+
echo "Dry run check:"
62+
python3 -u scripts/experiments.py \
63+
--phase 2 \
64+
--output $OUTPUT_DIR \
65+
--cores $SLURM_CPUS_PER_TASK \
66+
--dry-run
67+
68+
echo ""
69+
echo "Starting Phase 2..."
70+
echo ""
71+
72+
# Run phase 2
73+
python3 -u scripts/experiments.py \
74+
--phase 2 \
75+
--output $OUTPUT_DIR \
76+
--cores $SLURM_CPUS_PER_TASK
77+
78+
# -----------------------------------------------------------------------------
79+
# Completion
80+
# -----------------------------------------------------------------------------
81+
82+
echo ""
83+
echo "========================================"
84+
echo "Phase 2 Complete"
85+
echo "========================================"
86+
echo "End time: $(date)"
87+
echo "Results in: $OUTPUT_DIR/"
88+
echo ""
89+
echo "Output files:"
90+
ls -lh $OUTPUT_DIR/
91+
echo ""
92+
echo "Next steps:"
93+
echo " 1. Download phase2_results.jsonl"
94+
echo " 2. Plot evolved_prey_death_final vs initial prey_death"
95+
echo " 3. Check if all runs converge to ~0.095-0.105 (critical point)"
96+
echo " 4. If SOC confirmed, proceed to Phase 3 (finite-size scaling)"
97+
echo "========================================"

‎hpc_data/phase1.5_18682575/experiments.log‎

Lines changed: 37 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,37 @@
1+
2026-01-26 11:37:31,253 [INFO] ============================================================
2+
2026-01-26 11:37:31,253 [INFO] PREDATOR-PREY HYDRA EFFECT EXPERIMENTS
3+
2026-01-26 11:37:31,253 [INFO] ============================================================
4+
2026-01-26 11:37:31,253 [INFO] Phases: [1]
5+
2026-01-26 11:37:31,253 [INFO] Output: results/phase1_18682575
6+
2026-01-26 11:37:31,253 [INFO] Cores: 32
7+
2026-01-26 11:37:31,254 [INFO] Numba: ENABLED
8+
2026-01-26 11:37:31,254 [INFO]
9+
2026-01-26 11:37:31,254 [INFO] ============================================================
10+
2026-01-26 11:37:31,254 [INFO] PHASE 1
11+
2026-01-26 11:37:31,254 [INFO] ============================================================
12+
2026-01-26 11:37:31,254 [INFO] Estimated: 9,000 sims, ~13.2h on 32 cores (~423 core-hours)
13+
2026-01-26 11:37:31,254 [INFO] Dry run - skipping execution
14+
2026-01-26 11:37:31,254 [INFO]
15+
2026-01-26 11:37:31,254 [INFO] ============================================================
16+
2026-01-26 11:37:31,254 [INFO] EXPERIMENTS COMPLETE
17+
2026-01-26 11:37:31,254 [INFO] ============================================================
18+
2026-01-26 11:37:32,419 [INFO] ============================================================
19+
2026-01-26 11:37:32,419 [INFO] PREDATOR-PREY HYDRA EFFECT EXPERIMENTS
20+
2026-01-26 11:37:32,419 [INFO] ============================================================
21+
2026-01-26 11:37:32,419 [INFO] Phases: [1]
22+
2026-01-26 11:37:32,419 [INFO] Output: results/phase1_18682575
23+
2026-01-26 11:37:32,419 [INFO] Cores: 32
24+
2026-01-26 11:37:32,419 [INFO] Numba: ENABLED
25+
2026-01-26 11:37:32,420 [INFO]
26+
2026-01-26 11:37:32,420 [INFO] ============================================================
27+
2026-01-26 11:37:32,420 [INFO] PHASE 1
28+
2026-01-26 11:37:32,420 [INFO] ============================================================
29+
2026-01-26 11:37:32,420 [INFO] Estimated: 9,000 sims, ~13.2h on 32 cores (~423 core-hours)
30+
2026-01-26 11:37:38,321 [INFO] Phase 1: 600 simulations
31+
2026-01-26 11:37:38,321 [INFO] Grid: 20 prey_death values × 30 reps (prey_birth=0.2)
32+
2026-01-26 12:11:49,630 [INFO] Phase 1 complete. Results: results/phase1_18682575/phase1_results.jsonl
33+
2026-01-26 12:11:49,668 [INFO] Phase 1 runtime: 34.3 minutes
34+
2026-01-26 12:11:49,669 [INFO]
35+
2026-01-26 12:11:49,669 [INFO] ============================================================
36+
2026-01-26 12:11:49,669 [INFO] EXPERIMENTS COMPLETE
37+
2026-01-26 12:11:49,669 [INFO] ============================================================

0 commit comments

Comments
 (0)