Skip to content

perf: cache call graph call nodes#1215

Merged
mldangelo-oai merged 2 commits intomainfrom
mdangelo/codex/perf-call-graph-call-node-cache
May 3, 2026
Merged

perf: cache call graph call nodes#1215
mldangelo-oai merged 2 commits intomainfrom
mdangelo/codex/perf-call-graph-call-node-cache

Conversation

@mldangelo-oai
Copy link
Copy Markdown
Contributor

Summary

  • cache repeated _iter_call_nodes() AST walks by function node
  • include the new cache in the existing source-sensitive invalidation sweep
  • add a regression that proves the second lookup reuses the cached walk

Benchmarks

  • repeated same-function call-node walk, 5,000 lookups x 5 passes:
    • before: 1.000113s median
    • after: 0.000891s median
  • matched cProfile run on tests/assets/exploits/exploit_ultimate_50pct.pkl from origin/main:
    • before: 82.470s
    • after: 66.972s

Validation

  • UV_CACHE_DIR=/tmp/modelaudit-uv-cache uv run pytest packages/modelaudit-picklescan/tests/test_call_graph_import_statements.py::test_iter_call_nodes_reuses_cached_walk -q
  • UV_CACHE_DIR=/tmp/modelaudit-uv-cache uv run pytest packages/modelaudit-picklescan/tests/test_call_graph_import_statements.py -q
  • UV_CACHE_DIR=/tmp/modelaudit-uv-cache uv run ruff format packages/modelaudit-picklescan/src packages/modelaudit-picklescan/tests
  • UV_CACHE_DIR=/tmp/modelaudit-uv-cache uv run ruff check --fix packages/modelaudit-picklescan/src packages/modelaudit-picklescan/tests
  • UV_CACHE_DIR=/tmp/modelaudit-uv-cache uv run mypy modelaudit/ packages/modelaudit-picklescan/src packages/modelaudit-picklescan/tests tests/ (hits existing mypy 1.20.0 internal error)
  • PROMPTFOO_DISABLE_TELEMETRY=1 UV_CACHE_DIR=/tmp/modelaudit-uv-cache uv run pytest -n auto -m "not slow and not integration" --maxfail=1 (reproduces the existing timing-sensitive directory-scan and validation-performance sentinels in this environment)

@github-actions
Copy link
Copy Markdown
Contributor

github-actions Bot commented May 2, 2026

Workflow run and artifacts

Performance Benchmarks

Compared 19 shared benchmarks with a regression threshold of 15%.
Status: 0 regressions, 0 improved, 19 stable, 0 new, 0 missing.
Aggregate shared-benchmark median: 359.73ms -> 361.96ms (+0.6%).

Benchmark Target Size Files Baseline Current Change Status
tests/benchmarks/test_picklescan_benchmarks.py::test_picklescan_dangerous_global_payloads[malicious_reduce] malicious_reduce 52 B 1 376.2us 399.2us +6.1% stable
tests/benchmarks/test_scan_benchmarks.py::test_validate_file_type_pytorch_zip state_dict.pt 1.5 MiB 1 52.5us 55.3us +5.4% stable
tests/benchmarks/test_scan_benchmarks.py::test_skip_filter_plain_text_files - 4.6 KiB 256 13.16ms 13.80ms +4.9% stable
tests/benchmarks/test_picklescan_benchmarks.py::test_picklescan_nested_payloads[nested_raw] nested_raw 78 B 1 122.5us 126.4us +3.2% stable
tests/benchmarks/test_picklescan_benchmarks.py::test_picklescan_multi_stream_padded_payload multi_stream_padded 4.1 KiB 1 448.5us 462.8us +3.2% stable
tests/benchmarks/test_picklescan_benchmarks.py::test_picklescan_chunked_stream chunked_stream 278.2 KiB 1 18.37ms 17.88ms -2.7% stable
tests/benchmarks/test_picklescan_benchmarks.py::test_picklescan_dangerous_global_payloads[stack_global] stack_global 21 B 1 322.5us 330.8us +2.6% stable
tests/benchmarks/test_scan_benchmarks.py::test_scan_pytorch_zip state_dict.pt 1.5 MiB 1 30.14ms 30.79ms +2.2% stable
tests/benchmarks/test_scan_benchmarks.py::test_scan_safe_pickle safe_model.pkl 49.4 KiB 1 11.35ms 11.11ms -2.1% stable
tests/benchmarks/test_picklescan_benchmarks.py::test_picklescan_nested_payloads[nested_base64] nested_base64 98 B 1 129.9us 132.2us +1.8% stable
tests/benchmarks/test_picklescan_benchmarks.py::test_picklescan_safe_payloads[safe_large] safe_large 278.2 KiB 1 16.12ms 15.86ms -1.6% stable
tests/benchmarks/test_picklescan_benchmarks.py::test_picklescan_opcode_budget_tail_payload opcode_budget_tail 14 B 1 91.1us 92.3us +1.3% stable
tests/benchmarks/test_picklescan_benchmarks.py::test_picklescan_safe_payloads[safe_small] safe_small 68 B 1 77.4us 76.5us -1.2% stable
tests/benchmarks/test_scan_benchmarks.py::test_detect_file_format_safe_pickle safe_model.pkl 49.4 KiB 1 31.7us 31.3us -1.1% stable
tests/benchmarks/test_scan_benchmarks.py::test_scan_duplicate_directory duplicate-corpus 840.0 KiB 81 193.64ms 195.45ms +0.9% stable
tests/benchmarks/test_picklescan_benchmarks.py::test_picklescan_hidden_suspicious_string_budget hidden_suspicious_string 8.0 KiB 1 620.2us 625.0us +0.8% stable
tests/benchmarks/test_picklescan_benchmarks.py::test_picklescan_nested_payloads[nested_hex] nested_hex 130 B 1 139.0us 139.9us +0.6% stable
tests/benchmarks/test_picklescan_benchmarks.py::test_picklescan_safe_payloads[long_benign_string] long_benign_string 1.0 MiB 1 1.06ms 1.06ms -0.2% stable
tests/benchmarks/test_scan_benchmarks.py::test_scan_mixed_directory mixed-corpus 1.7 MiB 54 73.49ms 73.53ms +0.1% stable

…all-graph-call-node-cache

# Conflicts:
#	packages/modelaudit-picklescan/src/modelaudit_picklescan/call_graph.py
#	packages/modelaudit-picklescan/tests/test_call_graph_import_statements.py
@mldangelo-oai mldangelo-oai merged commit aa52759 into main May 3, 2026
34 of 53 checks passed
@mldangelo-oai mldangelo-oai deleted the mdangelo/codex/perf-call-graph-call-node-cache branch May 3, 2026 02:17
@github-actions github-actions Bot mentioned this pull request May 3, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant