Feat/add spec mining

[bartrosa]

Summary

Introduces a seven-stage mining workflow on pickled-spec that turns a Python repo into inventory, code context, stories, Gherkin features, rule tags, and evaluation reports. Dogfoods the pickled monorepo under dogfood/mining-output/.

Also fixes several friction items discovered during dogfood runs: line-based tag injection, call-graph resolver robustness, and defensive stripping of LLM markdown fences before structured parsing.

What's new

Mining CLI (pickled-spec mine)

Stage	Command	Primary output
Inventory	mine inventory	inventory.json
Code	mine code	code-context/*.md
Stories	mine stories	stories/*.story.md
Features	mine features	features/*.feature
Tag	mine tag	tags-proposals.json + in-file tags
Evaluate	mine evaluate	evaluation/coverage.json, ambiguity.json
Report	mine report	mining-report.md

• mine all runs the full pipeline; --quick / --interactive, --surfaces, --overwrite-* supported.
• Code stage: AST-based reader with body / callgraph depth, --max-hops default 2, cycle detection, callee caps.
• Stories stage consumes code-context/ and can surface docstring drift.
• Documentation: docs/mining.md, ADRs 0005–0007.

LLM output hardening (friction #17)

• New shared helper: pickled_core.llm.sanitize.strip_markdown_fence.
• Applied in all structured-output drafters (Gherkin, YAML, SQL, JSON, OpenAPI, HCL).
• Strips only a single outermost fence wrapping the entire response; inner fences and trailing prose are preserved.
• Replaces ad-hoc fence handling in IaCDrafter.
• Tests: packages/pickled-core/tests/test_llm_sanitize.py (unit + per-drafter canned-LLM cases).

Tag injection fix (friction #16)

• Tag stage uses line-based insertion instead of character offsets.
• Repair path for legacy Scen@tag / enario: corruption; evaluate skips unparseable features with warnings.

Resolver / inventory

• Safer handling of Literal[...] and typing annotations in import resolution.
• Constructor-then-method, annotated params, noise filters for stdlib/decorators.
• Bombardment fixtures under tests/fixtures/resolver_zoo/.

Other

• py.typed on pickled-bdd for mypy when pickled-core imports BDD types.
• feature_glob configuration for flexible feature paths.
• Rules gates_runner multi-ruleset improvements.

Dogfood impact

On existing dogfood/mining-output/ before fence-strip:

• Ambiguity parse error: 2 (features starting with ```)
• After fix (verified on corrupted files + unit tests): 0 expected on re-run

Remaining fail / warn verdicts are scenario-quality / gate calibration, not parse corruption.

Test plan

uv sync --extra mcp
uv run pytest -q
uv run mypy packages/pickled-core/src
uv run ruff check packages/pickled-core/src/pickled_core/llm/ \
  packages/pickled-core/tests/test_llm_sanitize.py \
  packages/pickled-bdd/src/pickled_bdd/drafter.py \
  packages/pickled-rules/src/pickled_rules/drafter.py \
  packages/pickled-data/src/pickled_data/drafter.py \
  packages/pickled-diff/src/pickled_diff/drafter.py \
  packages/pickled-schema/src/pickled_schema/openapi/drafter.py \
  packages/pickled-iac/src/pickled_iac/drafter.py

uv run pytest -q packages/pickled-core/tests/test_llm_sanitize.py -v

# Optional full pipeline proof
uv run pickled-spec mine all . --output /tmp/nofence/ \
  --depth callgraph --max-hops 2 \
  --ruleset-config dogfood/pickled.ruleset.yaml --verbose

python3 -c "
import json
from collections import Counter
d = json.load(open('/tmp/nofence/evaluation/ambiguity.json'))
c = Counter(f['verdict'] for f in d['features'])
assert c.get('error', 0) == 0
print('ZERO PARSE ERRORS', dict(c))
"
grep -l '^\`\`\`' /tmp/nofence/features/*.feature || echo 'no stray fences'

Notes for reviewers

• Large diff includes committed dogfood/mining-output/ artifacts; focus review on packages/pickled-core/src/pickled_core/mine/, drafters, and tests.
• strip_markdown_fence is intentionally not applied to rationale / advisor prose fields.
• Full uv run ruff check packages/ may still surface pre-existing issues outside this branch’s touched paths.