Skip to content

fix: clear remaining security-quality findings#1219

Merged
mldangelo-oai merged 3 commits intomainfrom
mdangelo/codex/fix-codeql-metadata-url-assertions
May 3, 2026
Merged

fix: clear remaining security-quality findings#1219
mldangelo-oai merged 3 commits intomainfrom
mdangelo/codex/fix-codeql-metadata-url-assertions

Conversation

@mldangelo-oai
Copy link
Copy Markdown
Contributor

@mldangelo-oai mldangelo-oai commented May 3, 2026

Summary

  • remove redundant substring-based metadata URL assertions that CodeQL mistook for sanitization
  • normalize SARIF formatter imports and strengthen its primary-result regression coverage
  • document and harden source-sensitive call-graph cache handling, including invalid-reference filtering and cache registration
  • clean up recent test-quality findings in metadata, Keras ZIP, and Skops coverage

Why

The repo had one standard CodeQL alert plus a batch of recently surfaced security-quality findings in files changed on main. The scanner behavior itself was already sound in the metadata case, but the tests and call-graph support code had several worthwhile cleanup opportunities that are safer to resolve together now while the context is fresh.

Validation

  • PROMPTFOO_DISABLE_TELEMETRY=1 uv run pytest packages/modelaudit-picklescan/tests/test_call_graph_import_statements.py::test_shared_source_sensitive_caches_clears_once_per_scope -q
  • PROMPTFOO_DISABLE_TELEMETRY=1 uv run pytest tests/scanners/test_metadata_scanner.py -q
  • PROMPTFOO_DISABLE_TELEMETRY=1 uv run pytest tests/scanners/test_skops_scanner.py -q
  • PROMPTFOO_DISABLE_TELEMETRY=1 uv run pytest tests/scanners/test_keras_zip_scanner.py -q
  • PROMPTFOO_DISABLE_TELEMETRY=1 uv run pytest tests/integrations/test_sarif_formatter.py -q (48 skipped on this Python 3.12 lane per repo allowlist)
  • uv run ruff format modelaudit/ packages/modelaudit-picklescan/src packages/modelaudit-picklescan/tests tests/
  • uv run ruff check --fix modelaudit/ packages/modelaudit-picklescan/src packages/modelaudit-picklescan/tests tests/
  • uv run mypy modelaudit/ packages/modelaudit-picklescan/src packages/modelaudit-picklescan/tests tests/
  • PROMPTFOO_DISABLE_TELEMETRY=1 uv run pytest -n auto -m "not slow and not integration" --maxfail=1

Notes

  • I kept the Skops lowercasing regression as an optimization guard and documented why it matters instead of weakening it into a less specific behavior test.

@mldangelo-oai mldangelo-oai changed the title test: avoid substring assertions in metadata URL coverage test: clear CodeQL findings in test coverage May 3, 2026
@mldangelo-oai mldangelo-oai changed the title test: clear CodeQL findings in test coverage fix: clear remaining security-quality findings May 3, 2026
@github-actions
Copy link
Copy Markdown
Contributor

github-actions Bot commented May 3, 2026

Workflow run and artifacts

Performance Benchmarks

Compared 19 shared benchmarks with a regression threshold of 15%.
Status: 0 regressions, 0 improved, 19 stable, 0 new, 0 missing.
Aggregate shared-benchmark median: 359.83ms -> 360.35ms (+0.1%).

Benchmark Target Size Files Baseline Current Change Status
tests/benchmarks/test_picklescan_benchmarks.py::test_picklescan_chunked_stream chunked_stream 278.2 KiB 1 18.05ms 19.12ms +6.0% stable
tests/benchmarks/test_picklescan_benchmarks.py::test_picklescan_safe_payloads[safe_small] safe_small 68 B 1 77.4us 80.5us +4.1% stable
tests/benchmarks/test_picklescan_benchmarks.py::test_picklescan_nested_payloads[nested_raw] nested_raw 78 B 1 129.8us 134.6us +3.7% stable
tests/benchmarks/test_picklescan_benchmarks.py::test_picklescan_nested_payloads[nested_base64] nested_base64 98 B 1 131.9us 136.6us +3.5% stable
tests/benchmarks/test_picklescan_benchmarks.py::test_picklescan_opcode_budget_tail_payload opcode_budget_tail 14 B 1 92.2us 95.1us +3.2% stable
tests/benchmarks/test_picklescan_benchmarks.py::test_picklescan_multi_stream_padded_payload multi_stream_padded 4.1 KiB 1 449.2us 462.9us +3.1% stable
tests/benchmarks/test_picklescan_benchmarks.py::test_picklescan_nested_payloads[nested_hex] nested_hex 130 B 1 139.3us 142.9us +2.6% stable
tests/benchmarks/test_scan_benchmarks.py::test_skip_filter_plain_text_files - 4.6 KiB 256 13.45ms 13.21ms -1.8% stable
tests/benchmarks/test_picklescan_benchmarks.py::test_picklescan_dangerous_global_payloads[malicious_reduce] malicious_reduce 52 B 1 380.3us 386.5us +1.6% stable
tests/benchmarks/test_scan_benchmarks.py::test_validate_file_type_pytorch_zip state_dict.pt 1.5 MiB 1 51.4us 52.1us +1.3% stable
tests/benchmarks/test_scan_benchmarks.py::test_scan_safe_pickle safe_model.pkl 49.4 KiB 1 11.05ms 10.91ms -1.3% stable
tests/benchmarks/test_picklescan_benchmarks.py::test_picklescan_dangerous_global_payloads[stack_global] stack_global 21 B 1 331.0us 335.1us +1.2% stable
tests/benchmarks/test_scan_benchmarks.py::test_detect_file_format_safe_pickle safe_model.pkl 49.4 KiB 1 31.4us 31.1us -0.8% stable
tests/benchmarks/test_scan_benchmarks.py::test_scan_mixed_directory mixed-corpus 1.7 MiB 54 73.73ms 73.18ms -0.7% stable
tests/benchmarks/test_scan_benchmarks.py::test_scan_pytorch_zip state_dict.pt 1.5 MiB 1 30.72ms 30.55ms -0.6% stable
tests/benchmarks/test_picklescan_benchmarks.py::test_picklescan_hidden_suspicious_string_budget hidden_suspicious_string 8.0 KiB 1 628.3us 624.9us -0.5% stable
tests/benchmarks/test_picklescan_benchmarks.py::test_picklescan_safe_payloads[long_benign_string] long_benign_string 1.0 MiB 1 1.05ms 1.06ms +0.4% stable
tests/benchmarks/test_scan_benchmarks.py::test_scan_duplicate_directory duplicate-corpus 840.0 KiB 81 193.50ms 193.98ms +0.2% stable
tests/benchmarks/test_picklescan_benchmarks.py::test_picklescan_safe_payloads[safe_large] safe_large 278.2 KiB 1 15.83ms 15.86ms +0.2% stable



class _CacheClearable(Protocol):
def cache_clear(self) -> None: ...
@mldangelo-oai mldangelo-oai merged commit 259f931 into main May 3, 2026
31 checks passed
@mldangelo-oai mldangelo-oai deleted the mdangelo/codex/fix-codeql-metadata-url-assertions branch May 3, 2026 02:57
@github-actions github-actions Bot mentioned this pull request May 3, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant