A self-healing code reasoning engine. Three LLM agents in a closed loop — Detective → QA → Patcher — that finds a bug, writes a test to expose it, applies the smallest surgical fix, and verifies the fix. No human in the loop.
Stop writing patches. Start writing the loop that writes patches.
Most "AI coding tools" autocomplete the next line. fixpoint doesn't guess — it reasons. It reads a failing test, generates 3 distinct hypotheses about the root cause (Tree-of-Thoughts), writes a pytest that would expose each one, runs them in an isolated sandbox, applies the smallest fix that makes the test pass, and verifies it didn't break anything else. If it did, it backtracks and tries the next hypothesis.
Then it does something no other tool does: it turns every reasoning trace into DPO training data, so the Detective gets smarter with every bug it touches.
def sum_range(a, b):
total = 0
for i in range(a, b): # ← bug: range() excludes b
total += i
return total
# sum_range(1, 3) returns 3, should return 6A junior engineer sees the failing test, googles "python range off by one", and fixes it. An LLM autocomplete might suggest range(a, b+1) because it's seen this pattern a million times.
But what happens when the bug is subtle? When it's a missing space in SQL generation (DISTINCTCASE instead of DISTINCT CASE)? When it's a mutable default argument that leaks state across calls? When the failing test is in a 50k-line Django codebase you've never read?
That's where the autocomplete stops and the engineer starts. fixpoint is the engineer.
┌─────────────────────────────────────────────────────────────┐
│ THE CLOSED LOOP │
│ │
│ ┌──────────────┐ ┌──────────────┐ ┌──────────────┐ │
│ │ 🔍 Detective │───▶│ 🧪 QA │───▶│ 🔧 Patcher │ │
│ │ │ │ │ │ │ │
│ │ Reads the │ │ Writes a │ │ Applies the │ │
│ │ failing test │ │ pytest that │ │ smallest fix │ │
│ │ + code, │ │ mocks deps │ │ that makes │ │
│ │ generates 3 │ │ and EXPOSES │ │ the test │ │
│ │ ranked │ │ the bug │ │ PASS │ │
│ │ hypotheses │ │ │ │ │ │
│ └──────────────┘ └──────┬───────┘ └──────┬───────┘ │
│ ▲ │ │ │
│ │ ▼ ▼ │
│ │ ┌─────────────┐ ┌─────────────┐ │
│ │ │ Sandbox │ │ Verify │ │
│ │ │ (isolated │ │ Re-run the │ │
│ │ │ pytest) │ │ test on │ │
│ │ │ │ │ patched │ │
│ │ │ Test │ │ code │ │
│ │ │ FAILS? ✓ │ │ │ │
│ │ │ (exposes │ │ PASS? ✓ │ │
│ │ │ the bug) │ │ → DONE │ │
│ │ └──────┬──────┘ └──────┬──────┘ │
│ │ │ │ │
│ │ FAIL? ──┘ FAIL? ───┘ │
│ └──────────── backtrack to next hypothesis ───────┘
│ │
└─────────────────────────────────────────────────────────────┘
The key insight: the QA Agent writes a test that should fail on the buggy code. If it does fail, the hypothesis is confirmed — the Detective was right about where the bug lives. If it passes, the hypothesis was wrong, and the loop backtracks.
This is exactly how a senior engineer debugs: form a hypothesis, write a test to confirm it, fix, verify. We just made it automatic.
The Detective doesn't write one guess. It writes 3 distinct hypotheses — genuinely different explanations, not paraphrases — and ranks them by confidence. If #1 is wrong, #2 gets a chance.
Every hypothesis is tested in an isolated pytest sandbox with mocked external dependencies. No fix is committed until a test the engine itself wrote passes on the patched code.
If the Patcher's fix breaks verification, the loop backtracks to the next hypothesis — but the failed attempt is logged. The engine learns "this kind of fix doesn't work for this kind of bug".
Every closed-loop run produces a reasoning trace: the 3 hypotheses, which one exposed the bug, what the Patcher tried, whether the judge accepted it. This trace is the data OpenAI/Anthropic don't have — they train on flat (buggy, fixed) pairs, never on the reasoning that connects them.
fixpoint converts every trace into:
- A DPO-style preference dataset (
prompt, chosen, rejected, margin) — consumable by any DPO trainer. - A reward model (logistic regression baseline; transformer-ready).
- A retrieval-augmented prompt optimizer that feeds high-reward past hypotheses back into the Detective at inference time.
| Bug | Type | Status |
|---|---|---|
sum_range(a, b) misses last element |
off-by-one | ✅ FIXED |
append_item() reuses list across calls |
mutable default arg | ✅ FIXED |
average() returns int instead of float |
integer division | ✅ FIXED |
is_adult(18) returns False |
wrong comparison op | ✅ FIXED |
| Case | Repo | Function | Status | Score |
|---|---|---|---|---|
django-10880 |
django | as_sql() |
✅ FIXED | 1.00 |
astropy-13453 |
astropy | write() |
✅ closed-loop success | — |
astropy-12907 |
astropy | _cstack() |
🟡 QA exposed, patch failed | — |
astropy-14995 |
astropy | _arithmetic_mask() |
🟡 QA exposed, patch failed | — |
Headline: the engine fixed a real Django bug (as_sql() was missing a space after DISTINCT) with an exact match to the gold patch. The Detective nailed the diagnosis, the Patcher made a one-character fix.
| Metric | Value | Meaning |
|---|---|---|
| Reward → success correlation | 0.967 | The hand-coded reward aligns almost perfectly with actual fix success |
| Preference pairs generated | 15 | (7 high-signal, margin ≥ 0.2) |
| Reward model correlation | 0.274 | Logistic regression baseline — needs more data |
fixpoint/
├── src/
│ ├── agents/
│ │ ├── detective.ts # Tree-of-Thoughts hypothesis generator
│ │ ├── qa.ts # Original QA (for self-contained bugs)
│ │ ├── qa_enhanced.ts # Mock-deps QA (for real-codebase code)
│ │ └── patcher.ts # Surgical patch generator + verifier
│ │
│ ├── graph/
│ │ └── engine.ts # The closed-loop state machine
│ │
│ ├── sandbox/
│ │ └── index.ts # Isolated pytest runner with timeout
│ │
│ ├── rl/ # ← The moat
│ │ ├── trace_analyzer.ts # Walks logs/ → structured TraceExamples
│ │ ├── reward.ts # Hand-coded reward function [-1, +1]
│ │ ├── preference_dataset.ts # DPO-style pair builder
│ │ ├── reward_model.ts # Logistic regression baseline
│ │ ├── evaluator.ts # Correlation metrics
│ │ ├── prompt_optimizer.ts # Retrieval-augmented prompting
│ │ └── run_rl_pipeline.ts # End-to-end RL pipeline runner
│ │
│ ├── adapters/
│ │ └── swebench.ts # SWE-bench → BugCase converter
│ │
│ ├── types/engine.ts # Shared types
│ ├── llm.ts # LLM wrapper (z-ai-web-dev-sdk)
│ └── utils.ts # File I/O, logging
│
├── datasets/
│ └── sample_cases.ts # 4 toy bugs with reference fixes
│
├── fetch_swebench.py # SWE-bench downloader (snippet mode)
├── fetch_swebench_v2.py # SWE-bench downloader (full repo clone)
│
└── README.md # ← you are here
- Node.js 18+ (we use 24.x)
- Python 3.11+ (for the sandbox)
- pytest (
pip install pytest) - z-ai-web-dev-sdk (installed automatically)
git clone https://github.com/modarresi1913/fixpoint.git
cd fixpoint
npm install
npx tsx src/run_demo.tsYou'll see the closed loop run end-to-end on 4 hand-crafted Python bugs:
🧠 Self-Healing Code Reasoning Engine — PoC
📦 Dataset: 4 cases
Case: off-by-one — sum_range should sum integers from a to b inclusive...
[detective] Generated 3 hypotheses. Top: "range end parameter is exclusive" (80%)
[select_next] Selected hypothesis #1/3
[qa] Wrote test. Bug-exposing test: YES ✓ (0 passed / 2 failed)
[qa_router] QA test exposes the bug. Proceeding to patch.
[patcher] Applied patch. Verification: PASS ✓ (1 passed / 0 failed)
[verify] Patch verified. Done.
✓ FIXED. Patch summary: Changed range(a, b) to range(a, b + 1)
📊 Final score: 4 / 4 cases fixed.
📄 Full report: download/self_healing_report.md
# 1. Download a SWE-bench subset (one-time, ~3 min)
python3 fetch_swebench_v2.py --n 500 --min 30 \
--out datasets/swebench_cases_v2_large.json
# 2. Run the closed loop (resume-safe — re-run anytime to continue)
SWEBENCH_MAX_CASES=8 npx tsx src/run_swebench_scale.ts
# 3. Read the report
cat download/swebench_scale_report.md# Converts every trace you've collected into a DPO dataset + reward model
npx tsx src/rl/run_rl_pipeline.ts
# Outputs:
# download/rl_layer_report.md ← full report
# download/rl_preference_dataset.json ← DPO-consumable pairs
# download/rl_reward_model.json ← trained model weightsLet's trace through a real example: the Django as_sql() bug.
# django/db/models/aggregates.py
def as_sql(self, compiler, connection, **extra_context):
extra_context['distinct'] = 'DISTINCT' if self.distinct else ''
# ↑
# BUG: should be 'DISTINCT ' (with trailing space)
# Generates "COUNT(DISTINCTCASE WHEN...)" — invalid SQL
...{
"hypotheses": [
{
"title": "Missing space between DISTINCT and CASE",
"confidence": 0.80,
"reasoning": "The symptom shows 'COUNT(DISTINCTCASE WHEN...)' which means
the DISTINCT keyword runs directly into the CASE keyword.
The string literal 'DISTINCT' should be 'DISTINCT ' (with space).",
"proposedFix": "Change 'DISTINCT' to 'DISTINCT ' in the as_sql method."
},
{
"title": "SQL template lacks space placeholder",
"confidence": 0.15,
"reasoning": "..."
},
{
"title": "Aggregate function formatting issue",
"confidence": 0.05,
"reasoning": "..."
}
]
}# test_main.py (written by the QA Agent)
from unittest.mock import MagicMock, patch
def as_sql(self, compiler, connection, **extra_context):
# (buggy function inlined here)
extra_context['distinct'] = 'DISTINCT' if self.distinct else ''
...
def test_distinct_has_trailing_space():
agg = MagicMock()
agg.distinct = True
ctx = {}
as_sql(agg, None, None, **ctx)
assert ctx['distinct'] == 'DISTINCT ', f"Got: {ctx['distinct']!r}"
# ↑
# This FAILS on the buggy code:
# AssertionError: Got: 'DISTINCT'# Patched code (written by the Patcher Agent)
def as_sql(self, compiler, connection, **extra_context):
extra_context['distinct'] = 'DISTINCT ' if self.distinct else ''
# ↑ ^^^^^^^^^^
# fixed: added trailing space
...$ python -m pytest test_main.py -v
test_main.py::test_distinct_has_trailing_space PASSED [100%]
✅ The test that exposed the bug now passes on the patched code. The loop is done.
Judge verdict: equivalent (score 1.00)
Reasoning: The proposed fix correctly adds a space after 'DISTINCT'
in the string assignment, matching the reference fix exactly.
"Every other code LLM is trained on (buggy, fixed) pairs from GitHub commits. They have no access to what hypotheses the engineer considered and rejected, which test exposed the bug, or why the first patch attempt failed. fixpoint produces exactly that trace as a side-effect of running."
+1.0 judge_verdict == 'equivalent' (perfect fix)
+0.5 judge_verdict == 'partial' (right area, imperfect)
+0.3 patcher_verified (patched code passed QA test)
+0.1 qa_exposed_bug (bug-exposing test, before patch)
-0.2 judge_verdict == 'wrong' (confidently wrong)
-0.3 patcher_verified == false (patch broke things)
-0.1 * (rank - 1) (prefer top-ranked hypotheses)
-0.05 * (attempts_before) (prefer first-try wins)
After running the closed loop on N bugs, the RL pipeline produces:
-
rl_preference_dataset.json— DPO-style pairs:{ "prompt": "<buggy code>\n<symptom>\nGenerate 3 hypotheses.", "chosen": "Hypothesis: Missing space between DISTINCT and CASE...", "rejected": "Hypothesis: SQL template lacks space placeholder...", "chosenReward": 0.7, "rejectedReward": -0.3, "margin": 1.0 }Drop this into any DPO trainer (TRL, Axolotl, Unsloth) and fine-tune your LLM.
-
rl_reward_model.json— trained logistic regression on engineered features (confidence, rank, title length, surgical-ness, hedging). Replace with a transformer model when you have ≥1000 traces. -
A retrieval-augmented prompt optimizer — at inference time, the Detective's prompt is augmented with high-reward hypotheses from similar past bugs (Jaccard similarity on code tokens). This is the cheapest way to feed the reward signal back in without a GPU.
-
The reasoning loop generalizes. The Detective correctly identified the root cause in 3/4 SWE-bench cases — including the subtle "missing space after DISTINCT" bug that requires understanding SQL generation context, not just reading the line.
-
The QA Agent can expose real bugs. With
unittest.mock.patch, it writes tests that genuinely fail on buggy code from Django and Astropy — without needing the full dependency tree installed. -
The reward signal is real. Hand-coded reward has 0.967 correlation with actual fix success. The preference dataset isn't noise.
-
LLM-as-judge is a usable proxy. Every verdict was manually inspected and correct. You can iterate on the engine without paying the Docker-per-case cost of real SWE-bench evaluation.
-
Mock-based QA is too synthetic for tightly-coupled code. 6/9 SWE-bench cases failed because the mock setup the LLM writes doesn't capture enough of the real behaviour. The test passes on the buggy code too, so the loop can't tell if the hypothesis is right.
-
Without a real Docker sandbox, the Patcher over-engineers. When the QA sandbox can't faithfully verify, the Patcher lacks the negative feedback ("your previous attempt broke X") that keeps it surgical. It rewrites whole functions instead of making one-line fixes.
-
Scale is limited by rate limits, not by the model. The 4-case run hit 429s repeatedly. Production needs either a higher tier or built-in request pacing.
- Replace subprocess sandbox with real Docker (
python:3.11-slim,--network=none,--read-only,--memory=256m). The sandbox interface is already a single swap point. - Scale to all 500 SWE-bench_Verified instances. With Docker in place, this grows the trace corpus 50×.
- Train a transformer reward model on the preference pairs (replace logistic regression).
- DPO fine-tune the Detective — LoRA on the underlying LLM, using the preference dataset. This is the "Thiel's Secret" data moat.
- Online bandit over hypothesis strategies — use the reward model to pick between Tree-of-Thoughts, plain top-3, and counterfactual prompting at inference time.
- Architecture — the RL layer design doc, inline.
- The reasoning graph — the LangGraph-style state machine that orchestrates the loop.
- The reward function — every component, explained.
- Sample reports — real outputs from real runs.
This is a PoC that proved the architecture works. The most valuable contributions right now:
- Docker sandbox — replace
src/sandbox/index.tswith a real containerized runner. Should flip several "Patcher over-engineered" failures into wins. - More SWE-bench cases — run
fetch_swebench_v2.pywith--min 100and contribute the resulting traces. - Transformer reward model — replace the logistic regression in
src/rl/reward_model.tswith a fine-tuned small BERT. - Multi-file support — switch the Patcher from full-file rewrite to tree-sitter-anchored edits.
PRs welcome. See CONTRIBUTING.md for guidelines (coming soon).
MIT — see LICENSE.
- SWE-bench (Princeton NLP) — the benchmark that made this evaluation possible.
- z-ai-web-dev-sdk — the LLM backend.
- Peter Thiel — Zero to One inspired the "build the moat, not the feature" framing.
⬆ Star this repo if you want to see where this goes next.
Built with reasoning, not autocomplete.