prototype: reverse template variable extractor

[randlee]

Overview

New prototype/reverse_extract/ — given a .j2 template + rendered XML output, extracts the JSON variable bindings that produced it. The reverse of sc-compose render.

Why

sc-compose currently has the forward pipeline (template-init → render → verify) but no reverse path. This fills that gap with two use cases:

1. Variable extraction — "what JSON produced this rendered payload?"
2. Template identification — "which template produced this ATM message?" (via confidence scoring)

What it does

from prototype.reverse_extract import extract_variables

result = extract_variables(
    "qa-template.xml.j2",
    "rendered-payload.xml",
    include_vars=["task_id", "sprint", "branch"],
)
# → {"task_id": "AI2-QA-2", "sprint": "AI.2", "branch": "feature/..."}

Confidence score

The _confidence field measures % of static template text that appears in-order in the rendered output. This is the template identification signal:

Template	Typical confidence
Correct	0.08–0.89
Wrong	~0.005

A threshold of 0.05 cleanly separates correct from wrong templates.

Bulk validation

Tested against 500 real ATM payloads from ~/.config/atm/share/atm-dev/:

• 194/197 repo-qa-task extractions pass (98.5%)
• 3 failures due to XML attribute variants
• 15 errors due to malformed XML in rendered files (not extractor bugs)

Confidence scoring correctly identifies template mismatches at ~0.5% baseline.

Planned CLI integration

sc-compose extract qa-template.xml.j2 rendered.xml --vars task_id,sprint,branch
sc-compose identify rendered.xml --templates-dir .claude/skills/codex-orchestration/

Files

• prototype/reverse_extract/extractor.py — core engine
• prototype/reverse_extract/bulk_test.py — bulk validation harness
• prototype/reverse_extract/README.md — usage, architecture, testing