Autoresearch FTP

[sadrasabouri]

Submission Checklist

• My file is named submissions/<github_username>.py.
• My submission runs locally.
• My output matches the expected output.
• I did not hardcode the final output.

Approach Description

I started by exploring a smarter build ordering mechanism, but saw no meaningful gains. After some investigation I realised I had been benchmarking against Python's standard (GIL-enabled) build, which masks the benefit of parallelism entirely. Switching to Python 3.14t (the free-threading build, GIL disabled) changed the results and I started to actual improving: 3.61× total speedup over the single-threaded reference across five graph shapes.

From the baseline implementation, which uses the first-to-finish flag (this commit), I applied the autoresearch loop described by Andrej Karpathy, committed it, ran the benchmark, and either kept the improvement or reverted and tried again. Using Claude Code as the agent, the loop explored event-driven callbacks (replacing a polling executor), inline chain execution to avoid thread-handoff overhead on serial graphs, critical-path prioritisation for task selection, a custom persistent thread pool with a queue.

SimpleQueue to eliminate per-call thread creation cost, and various thread-count tunings. Each winning idea was layered onto the last; failed experiments (process pools, puts inside the lock, integer-indexed arrays, cross-trial caching) were automatically discarded. The log of the process is also documented below:

commit	total_speedup	status	description
713041b	3.09	keep	baseline — ThreadPoolExecutor FIRST_COMPLETED polling
272b5ff	3.39	keep	event-driven callbacks, drop FIRST_COMPLETED poll loop
d3f4801	3.49	keep	inline chain execution — execute one successor in same thread
09e9735	3.51	keep	critical-path priority for inline task selection
aa1d725	3.52	keep	write results outside lock, use max() for inline selection
9a92a25	3.53	keep	custom thread pool with SimpleQueue, no Future overhead
cadf82d	3.63	keep	3x cpu_count workers
9a92a25	3.53	discard	custom thread pool with SimpleQueue, no Future overhead (reverted, now part of later exps)
aa1d725	3.52	discard	write results outside lock (reverted)
09e9735	3.51	discard	critical-path priority for inline task selection (reverted)
d3f4801	3.49	discard	inline chain execution — superseded by later exps
49db1aa	3.54	keep	persistent pool + pre-sort dependents
dba124f	3.61	keep	persistent pool, no pre-sort, keep max() inline selection
67d9d99	3.48	discard	cache critical dict across trials — within noise, not worth complexity

The final solution has four interlocking ideas.

I) Persistent thread pool with a lock-free queue: rather than spawning and joining threads on every build_all() call, a pool of min(128, cpu_count × 3) daemon threads is created once at module load and reused indefinitely. Workers sit in a loop on a queue.SimpleQueue and dispatch to whatever run closure is currently active.

II) Event-driven, callback-style scheduling. There is no polling loop. Each worker, after completing a task, inspects a dependents map to find which successors just had their last dependency satisfied (tracked via an in_degree counter under a single shared lock).

III) Inline chain execution. When a task completes and exactly one successor becomes ready, the worker does not put that successor into the queue and return. it loops directly and executes it in the same thread.

[mpage]

Thanks for the submission @sadrasabouri! Can you please remove the files other than submissions/sadrasabouri.py?

[sadrasabouri]

@mpage requested changed applied.

[mpage]

@sadrasabouri - Thanks! It looks like there are a couple of new files that got added. Can you remove those as well?

[sadrasabouri]

@mpage ops, sorry. I rushed through committing. It should be fixed now.

[github-actions]

✅ Results

Metric	Value
Speedup	5.5153741718883955x
Passed	true
Graphs	5

Per-graph results

Graph	Speedup	Status
chain.json	0.9880832003340783x	✅
diamond.json	10.598057654484656x	✅
realistic.json	17.66487868494784x	✅
tree.json	17.408604744721426x	✅
wide.json	18.192699893246612x	✅

---

View the leaderboard