Skip to content

Commit 81d0d26

Browse files
committed
docs(tools): publish execution characterization
1 parent 719322d commit 81d0d26

4 files changed

Lines changed: 6474 additions & 0 deletions

File tree

CONTRIBUTING.md

Lines changed: 16 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -48,3 +48,19 @@ If you believe a new runtime dependency is genuinely necessary:
4848
automatically so reviewers know to look for the justification.
4949

5050
Dev-only dependencies under `[dependency-groups]` are not subject to this policy.
51+
52+
## Tool-execution characterization
53+
54+
Changes to `soul/tool_execution.py`, execution scheduling, deduplication, the reader/writer gate, or
55+
MCP publication should run the deterministic local characterization harness:
56+
57+
```bash
58+
uv run python scripts/benchmark_toolset.py --scenario all --runs 5 --output before-or-after.json
59+
```
60+
61+
For before/after evidence, use the same machine, Python environment, fixture matrix, warm-up count,
62+
and five-run command at both revisions. Keep the raw JSON and report timing regressions as well as
63+
improvements; do not discard samples or use `--smoke` for final evidence. Confirm every scenario has
64+
zero leaked tasks/processes/sessions, all cancellation and recovery flags are true, registry hashes
65+
are stable, and deterministic decision states are derived by the harness. Treat results as directional
66+
local engineering evidence, not universal product telemetry.

0 commit comments

Comments
 (0)