diff --git a/docs/README.md b/docs/README.md index 8de24b6..8f1ae5b 100644 --- a/docs/README.md +++ b/docs/README.md @@ -30,6 +30,8 @@ The decision-record docs follow the [Diátaxis](https://diataxis.fr) framework - [Configure LLM providers](how-to/configure-providers.md) — OpenAI, OpenRouter, Ollama, vLLM, LiteLLM - [Hand off to Linear](how-to/handoff-to-linear.md) - [Calibrate gates](how-to/calibrate-gates.md) — `poc` / `mvp` / `full` + overrides +- [Track outcomes](how-to/track-outcomes.md) — record post-handoff observations +- [Search decisions](how-to/search-decisions.md) — semantic + substring search and reindexing ### Reference - [CLI](reference/cli.md) — every flag, env var, exit code @@ -41,6 +43,7 @@ The decision-record docs follow the [Diátaxis](https://diataxis.fr) framework - [Why decision records?](explanation/why-decision-records.md) — Joel Parker Henderson's canonical material - [Design rationale](explanation/design-rationale.md) — why filesystem, why hard gates, why lens-rotating skeptic - [The five phases](explanation/the-five-phases.md) — what each phase does and why this shape +- [Research notes](explanation/research-notes.md) — broader DR/ADR ecosystem, prior art, and the rationale for outcomes + semantic search ## Outside the docs tree diff --git a/docs/explanation/research-notes.md b/docs/explanation/research-notes.md new file mode 100644 index 0000000..b2522ef --- /dev/null +++ b/docs/explanation/research-notes.md @@ -0,0 +1,137 @@ +# Research notes — DR/ADR discipline and how this system extends it + +This document is the "why" behind the outcome-tracking, semantic-search, and read-before-write expansion. It captures the broader DR / ADR ecosystem as it stood at the time of this writing, what's well-trodden, what's underserved, and which gaps we chose to fill. + +If you're looking for "how do I use feature X" — that's in the [how-to guides](../how-to/). This document is for understanding the design forces. + +## Lineage + +The Decision Record discipline has roughly three intellectual roots: + +- **Joel Parker Henderson** maintains the canonical [decision-record](https://github.com/joelparkerhenderson/decision-record) repo this project is forked from. The structure (title, status, context, decision, consequences) and the framing "an immutable, append-only record of significant choices" anchor most modern usage. +- **Michael Nygard's "Documenting Architecture Decisions" (2011)** popularized lightweight ADRs in the agile / DevOps mainstream. The key insight was that *the value is in writing them, not in fancy tooling* — a Markdown file in the repo beats a wiki entry that no one updates. +- **Tyree and Akerman (2005)** laid the academic foundation in "Architecture Decisions: Demystifying Architecture," covering position, argument, implications, and the explicit modeling of trade-offs that became the template variants this system supports. + +Subsequent work — **MADR v3/v4**, **e-adr (extended ADRs)**, **agile ADR templates** — refines the same skeleton. The ThoughtWorks Technology Radar moved ADRs to **Adopt** in 2018. + +## Theoretical foundations + +Three older traditions inform the shape of decision records: + +- **IBIS (Issue-Based Information System)** — Rittel and Kunz's argumentation model: Issues, Positions, Arguments. Our `Decision` schema is a direct descendant: `issue`, `positions`, `argument`. +- **QOC (Questions, Options, Criteria)** — MacLean et al.'s design-rationale model. Where IBIS focuses on debate, QOC focuses on choice. Our `assumptions`, `constraints`, and `selected_position` carry the QOC pattern. +- **DRL (Decision Representation Language)** — Lee's formal model for decision rationale. Less visible in practice but foundational. + +A specifically architectural strand: + +- **ASR (Architecturally Significant Requirements)** — Chen et al. — the framing that some requirements are load-bearing for the architecture and deserve dedicated records. Our `effort_level` calibration is a coarse stand-in for ASR-first intake. + +## Lifecycle in practice + +Most teams running ADR-style practice converge on a similar shape: + +1. **Trigger.** Either a code review reveals an unrecorded decision, a design review requires one, or a new project warrants documenting its load-bearing choices. +2. **Drafting.** Status starts at `proposed` or `rfc`. Discussion happens in the PR. +3. **Acceptance.** Status moves to `accepted` once enough stakeholders sign off. From this moment the record is immutable — *changes happen via new DRs that supersede*. +4. **Supersession or deprecation.** Older decisions can be marked `superseded` (with a forward link) or `deprecated`. + +What's **missing** in nearly every tool we surveyed: the **post-acceptance feedback loop**. The DR is written, work happens, and the record never learns whether the prediction came true. Outcomes were already implicit in the "Consequences" section of Nygard's template, but they were written *at decision time*, not *after observation*. This is the gap our `Outcome` entity fills. + +## Tooling ecosystem + +A non-exhaustive map of where the discipline lives today: + +- **`adr-tools`** (Nat Pryce) — the Bash CLI that put ADRs on most maps. Static Markdown only. +- **`log4brains`** (Thomson Reuters) — Markdown ADRs + static-site generator. Read-only browse experience. +- **Backstage TechDocs** (Spotify) — ADR plugin renders Markdown into the developer portal. Full-text search only. +- **`e-adr`** — academic extension introducing more structured fields. +- **MCP ADR Analysis Server** — recent MCP-based attempt at LLM-assisted ADR consumption. +- **Linear Projects + Docs** — many teams write decisions as Linear documents, especially when the same team executes them. Loses repo-coupling. +- **Notion, Confluence, hand-rolled wikis** — common in enterprise. Usually decays into the "DR graveyard" anti-pattern. + +Nobody we found ships **outcome tracking**, **semantic search across decisions**, or **agent-native read-before-write** as a first-class feature. That's the surface we expanded into. + +## Anti-patterns + +- **Compliance theater** — ADRs are written because a process says so, but no one reads them. Symptoms: titles are vague, arguments are absent, no DR ever cites another. +- **DR graveyard** — DRs are written, archived, and never re-examined. The new joiner can't find the relevant prior art and re-litigates instead. +- **Conflicting DRs** — over time, contradictory accepted decisions accumulate. Without supersession discipline (and ideally search to surface the conflict), the team operates on stale prior art. +- **Outcome blindness** — without an explicit outcome record, the team can't tell which DRs are predictive and which are aspirational. Selection bias takes over. +- **Over-fragmentation** — every tiny choice gets its own DR. The signal drowns in noise. +- **One-shot accept-and-archive** — the DR is treated as a write-once artifact. No revisits, no outcomes, no supersession. + +## Adjacent artifacts + +Decision records sit in a constellation of related documents. Knowing where they don't belong is as useful as knowing where they do. + +| Artifact | Purpose | When to use *instead of* a DR | +|---|---|---| +| **RFC** | Solicit feedback before a decision | While the decision is still genuinely open. Promote to DR when narrowing to a position. | +| **Design doc** | Detailed how-it-works writeup | When the *implementation* is the deliverable, not the *choice*. | +| **PRD** | Product requirements | Captures *what* and *for whom*, not *which option won*. | +| **Postmortem** | Incident analysis | After failure; the lessons may seed new DRs. | +| **Runbook** | Operational procedure | Stable how-to, not a choice. | +| **AGENTS.md / context files** | Standing guidance for agents | A persistent "always do X" instruction, often *derived* from one or more DRs. | + +## Emerging agentic applications + +Recent work explicitly connects DRs to AI-agent workflows: + +- **AgenticAKM (arXiv:2602.04445)** — proposes "Agentic Architecture Knowledge Management," with agents both consuming and producing ADRs as part of their planning loops. +- **Pollick's "ADR Comeback"** — argues that LLM-driven engineering teams rediscover ADRs because the agent context window benefits enormously from terse, structured prior art. +- **AgDR (Agent Decision Records)** — proposed schema extension where agents' own decisions about how to approach a task get the DR treatment, with explicit linkage to user prompts and tool calls. +- **AGENTS.md** — the trend toward a "living ADR" file that captures the *current* standing decisions agents must respect, derived from accepted DRs. + +This is the trajectory we're operating in: not just human-authored decisions, but decisions an agent helps author, retrieves before proposing, and circles back to validate or invalidate after handoff. + +## Open challenges + +A non-exhaustive list of things the field hasn't solved well: + +- **Cross-project decision search.** Most teams have decisions scattered across many repos. There's no canonical pattern for unified retrieval. +- **Decision aging.** When is a DR stale? When the world it was made in no longer applies. Hard to detect automatically. +- **Counter-evidence discovery.** Surfacing outcomes that *invalidate* an existing accepted DR is harder than surfacing supporting ones. +- **Quality calibration.** Distinguishing a high-quality DR (load-bearing, well-argued, observable outcome) from boilerplate. +- **Conflict detection.** Pairwise comparison of all accepted DRs is O(N²); LLM-driven detection is plausible but unshipped. + +## Opportunities — what this release ships + +Three opportunities the survey identified as both **high-value** and **schema-light** to ship in our system: + +### 1. Outcome tracking + +A new `Outcome` entity, post-handoff, links forward from an accepted decision and records what was observed. Status enum captures whether the decision held up. Evidence list captures URLs and file references. Status transitions are auditable. See [Track outcomes](../how-to/track-outcomes.md). + +### 2. Semantic search + +`dr_search_decisions` powered by OpenAI-compatible embeddings, with a deterministic substring fallback when embeddings are unavailable. Cache is hash-keyed so unchanged decisions skip re-embedding. See [Search decisions](../how-to/search-decisions.md). + +### 3. Read-before-write in the deciding agent + +The deciding-phase prompt now mandates a `dr_search_decisions` call per prospective topic *before* `dr_propose_decision`. Hits ≥ 0.85 either suppress the new DR or get cited via `related_decisions`. This is the operationalization of "agents should reuse prior art, not re-litigate" that the AgenticAKM literature names but doesn't ship. + +## Opportunities — future + +Out of scope for this release, on the medium-term roadmap: + +- **Cross-project semantic search.** A shared cache at `~/.dr-cache/` and a tool that crawls multiple project directories. +- **RFC → DR promotion.** Preserve the pre-decision deliberation thread as the DR's `summary` + `issue`. +- **AgDR schema extension.** A separate record type for agent-authored task decisions, linked back to the human-authored DR. +- **ASR-first intake.** Quality-attribute targeting (latency, availability, cost) before scoping, threaded through to gates. +- **Conflict detection.** LLM pairwise comparison over accepted DRs to flag latent contradictions. +- **Decision aging signals.** Heuristics on staleness (no outcomes, no references, no related work in N months). + +## Why this was worth shipping now + +The original Nygard / Tyree-Akerman skeleton is mature. The MADR / e-adr work refines it. The agentic literature names the next steps. But the **operationalized agent-native discipline** — write, gate, hand off, *then learn from outcomes, and retrieve before re-litigating* — wasn't represented in any tool we could find. We're not inventing the discipline; we're shipping the missing infrastructure. + +## Sources + +- Henderson, J.P. — [decision-record](https://github.com/joelparkerhenderson/decision-record) (canonical repo). +- Nygard, M. — "Documenting Architecture Decisions" (Cognitect, 2011). +- Tyree, J. & Akerman, A. — "Architecture Decisions: Demystifying Architecture," IEEE Software (2005). +- Rittel, H. & Kunz, W. — "Issues as Elements of Information Systems" (1970). +- MacLean et al. — "Questions, Options, and Criteria: Elements of Design Space Analysis," HCI (1991). +- Lee, J. — "Extending the Potts and Bruns Model for Recording Design Rationale" (1991). +- Chen, L. et al. — "Architecturally Significant Requirements" (2013). +- MADR, e-adr, AgenticAKM, Pollick's "ADR Comeback," AgDR — various blog posts and arXiv preprints (2023–2026). diff --git a/docs/how-to/search-decisions.md b/docs/how-to/search-decisions.md new file mode 100644 index 0000000..027d5f7 --- /dev/null +++ b/docs/how-to/search-decisions.md @@ -0,0 +1,84 @@ +# Search decisions semantically + +Once decisions accumulate, you'll want to ask "have we decided something like this before?" — across projects in the same repo, or across many DRs in one project. `dr_search_decisions` answers that question with vector embeddings (when available) and substring matching (when not). + +This is the same tool the deciding agent uses for **read-before-write** retrieval — before proposing a new decision, it checks whether the project already has a similar accepted one, and cites it via `related_decisions` instead of re-litigating. + +## How it works + +On every `dr_accept_decision` (and on `dr_update_decision` when the result is `accepted`), the server: + +1. Builds an embedding text from the decision's title, summary, issue, argument, selected position, position titles, implications, and tags. +2. Hashes that text. If the hash + model are already in the cache for this decision, no API call. +3. Otherwise, calls the embedding endpoint configured via `OPENAI_EMBEDDING_MODEL` (defaults to `text-embedding-3-small`). +4. Writes the resulting vector to `.dr/cache/embeddings.json`. + +`dr_search_decisions` embeds the query, computes cosine similarity against every cached vector, filters by status and `min_score`, sorts descending, and returns the top `limit`. + +## Configuration + +```bash +# Default — uses text-embedding-3-small +unset OPENAI_EMBEDDING_MODEL + +# Use a larger model +export OPENAI_EMBEDDING_MODEL=text-embedding-3-large + +# Disable embeddings entirely — search falls back to substring +export OPENAI_EMBEDDING_MODEL=none +``` + +`OPENAI_API_KEY` and `OPENAI_BASE_URL` are reused from the main LLM config. If you're running against a non-OpenAI provider that doesn't implement the embeddings endpoint, set `OPENAI_EMBEDDING_MODEL=none` to avoid noisy `embeddings_index_failed` events. + +## Search + +```jsonc +// Tool: dr_search_decisions +{ + "query": "primary data store", + "limit": 5, // top-N to return; default 5 + "min_score": 0.5, // semantic cosine threshold; default 0.5 + "status": ["accepted"] // which decision statuses to consider +} +``` + +Returns one of three modes: + +| Mode | When | Behavior | +|---|---|---| +| `semantic` | Embeddings enabled, cache populated, cache model matches `OPENAI_EMBEDDING_MODEL` | Cosine-ranked, scored, filtered by `min_score`. | +| `substring` | Embeddings disabled, cache missing, or cache model mismatched | Case-insensitive substring match across title/summary/issue/argument/selected_position/tags. Returns `score: null`. | +| `empty` | No decisions match the status filter | Empty `results[]`. | + +`warnings[]` always explains the cache state when something degrades the search quality. Surface these to humans so they know whether to trust the result or rebuild the index. + +## Reindex + +After switching models, after a manual cache wipe, or to backfill decisions that were accepted before embeddings were enabled: + +```jsonc +// Tool: dr_reindex_embeddings +{ "force": false } // when true, wipes the cache and re-embeds every accepted DR +``` + +Returns counts: `{ accepted_total, indexed, skipped, failed, failures, model }`. When `OPENAI_EMBEDDING_MODEL=none`, this fails fast — re-enable embeddings to reindex. + +## Read-before-write (the deciding agent's contract) + +When the deciding agent identifies a prospective decision topic, it calls `dr_search_decisions` *before* `dr_propose_decision`. The agent's contract: + +- If a similar **accepted** decision exists with score ≥ 0.85 and the context is transferable, do NOT propose a new DR. Cite the prior decision via `related_decisions`. +- If a similar one exists but the new context still warrants a new DR, propose anyway and cite the prior via `related_decisions`. +- Surface every score ≥ 0.85 hit in the final summary so reviewers can sanity-check the call. + +This is operationalized in `server/src/cli/agents/deciding.ts`. The 0.85 threshold is a heuristic — adjust it in the prompt for noisier embedding setups. + +## Cache hygiene + +- The cache lives at `.dr/cache/embeddings.json` and is gitignored. It's derived state, regenerable from the accepted decisions. +- On model change (`OPENAI_EMBEDDING_MODEL` differs from the cache's `default_model`), the next reindex wipes and rebuilds. +- The hash check makes accepting an already-accepted decision cheap — same content + same model = cache hit, no API call. + +## When semantic search isn't enough + +For now, `dr_search_decisions` searches inside one project's `.dr/cache/embeddings.json`. Cross-project search (a shared `~/.dr-cache/`) is on the roadmap but out of scope for the current release. If you need it today, scrape `dr/decisions/*.json` from each repo and feed them to an external vector store. diff --git a/docs/how-to/track-outcomes.md b/docs/how-to/track-outcomes.md new file mode 100644 index 0000000..8d5b9fb --- /dev/null +++ b/docs/how-to/track-outcomes.md @@ -0,0 +1,75 @@ +# Track outcomes after handoff + +Decision records are predictions. **Outcomes are the score.** + +After a project is handed off and the team has actually built (and operated) the thing, you'll learn whether each decision held up. Record that knowledge as an `Outcome` linked to the original DR. Doing this consistently closes the feedback loop most decision-record practices leave open. + +## When to record an outcome + +- A success criterion is met (or missed) and you can attribute it to a specific decision. +- A constraint or assumption from a DR turned out to be wrong. +- A position you rejected came back to bite you. +- Time has passed (30 days, a quarter, end of an experiment) and you have observable data. + +You can only record outcomes once `project.status === "handed-off"`. The pre-handoff phases are for planning; outcomes are post-handoff. + +## Record an outcome + +Use the MCP tool `dr_record_outcome`, or call the CLI in MCP mode: + +```bash +# Via the MCP server (after handoff) +# Tool: dr_record_outcome +{ + "decision_id": "0001-choose-data-store", + "observation": "After 30 days in production, p99 query latency is 290ms — within the 350ms budget set in the decision.", + "metric": "p99 latency 290ms", + "evidence": [ + "https://grafana.internal/d/db-latency", + "ops/post-launch-review.md" + ], + "status": "validated", + "tags": ["perf", "prod"] +} +``` + +Returns an outcome with id `O0001-after-30-days-in-production-p99-query-...`. The store: + +- Writes `dr/outcomes/O0001-*.json`. +- Bumps `state.next_outcome_seq`. +- Appends an `outcome_recorded` event. + +Re-run `dr_render` to refresh the Markdown and HTML — the outcome will appear in three places: the decision's `## Outcomes` section, its own `dr/outcomes/.md`, and the `Outcomes` table on `dr/index.html`. + +## Outcome statuses + +| Status | Meaning | +|---|---| +| `pending` | Recorded but not yet evaluated. Useful when you want to log an early observation and update later. | +| `validated` | The decision held up. | +| `invalidated` | The decision did not hold up. The argument was wrong, or the world changed. | +| `inconclusive` | Real but ambiguous. Note this honestly — false validation is worse than `inconclusive`. | + +Transitions emit an `outcome_status_changed` event with `{from, to}`. Use `dr_set_outcome_status` for these. + +## Update an outcome + +Use `dr_update_outcome` to patch `observation`, `metric`, `evidence`, or `tags`. Use `dr_set_outcome_status` for status changes. Both emit events so the audit trail is intact. + +## Listing and reading + +- `dr_list_outcomes` — optional `decision_id` filter and `status[]` filter. +- `dr_get_outcome` — fetch one by id. + +## Why outcomes are separate entities + +Outcomes live in `dr/outcomes/`, not inside the decision JSON. Two reasons: + +1. **Decision immutability after sign-off.** Once a DR is accepted and signed off, it shouldn't be edited. Outcomes are continuous and many-per-decision; nesting them would force constant rewrites of the canonical record. +2. **Cleaner search and embedding behavior.** The decision's text is the predictive content. Outcomes are observations *about* the prediction. Keeping them separate makes the embedding cache stable. + +## Anti-patterns + +- **Recording only validating outcomes.** Selection bias makes the DR record useless. Note invalidations and inconclusive results. +- **One outcome per project.** Most DRs deserve their own outcome. Tying them all to a single "retrospective" outcome defeats the point of per-decision tracking. +- **Outcomes with no evidence.** "It worked" without a metric or link is barely better than no outcome. Aim for at least one URL or file ref. diff --git a/docs/reference/cli.md b/docs/reference/cli.md index 13149c2..eabe18f 100644 --- a/docs/reference/cli.md +++ b/docs/reference/cli.md @@ -61,6 +61,7 @@ A positional argument can substitute for `--idea` if no other input flag is give | `OPENAI_API_KEY` | yes (unless `--api-key`) | API key for the LLM endpoint. | | `OPENAI_BASE_URL` | no | OpenAI-compatible base URL. Defaults to OpenAI's. | | `OPENAI_MODEL` | no | Default model. Defaults to `gpt-4o`. | +| `OPENAI_EMBEDDING_MODEL` | no | Embedding model for `dr_search_decisions` and the read-before-write retrieval. Defaults to `text-embedding-3-small`. Set to `"none"` to disable embeddings entirely; search will use substring fallback. | | `LINEAR_API_KEY` | no | Enables the Linear handoff branch in the handoff phase. | | `LINEAR_TEAM_ID` | no | Pre-fills the team ID prompt at Linear handoff. | | `DR_LOG_LEVEL` | no | `debug` \| `info` \| `warn` \| `error`. Default `info`. Applies to the MCP server's stderr logs. | diff --git a/docs/reference/data-model.md b/docs/reference/data-model.md index 1235b96..86aed62 100644 --- a/docs/reference/data-model.md +++ b/docs/reference/data-model.md @@ -1,6 +1,6 @@ # Data model -The pipeline stores five entity types. JSON Schemas are the source of truth in [`../../schemas/`](../../schemas/); the Zod mirrors used at runtime live in [`server/src/schemas/index.ts`](../../server/src/schemas/index.ts). +The pipeline stores six entity types. JSON Schemas are the source of truth in [`../../schemas/`](../../schemas/); the Zod mirrors used at runtime live in [`server/src/schemas/index.ts`](../../server/src/schemas/index.ts). ## Filesystem layout @@ -9,7 +9,8 @@ The pipeline stores five entity types. JSON Schemas are the source of truth in [ ├── .dr/ # internal, gitignored by default │ ├── state.json # PipelineState │ ├── events.jsonl # Event (one per line, append-only) -│ └── cache/ # derived artifacts +│ └── cache/ +│ └── embeddings.json # EmbeddingCache (vector cache for dr_search_decisions) └── dr/ # tracked ├── project.json # Project ├── project.md # rendered, derived @@ -19,6 +20,9 @@ The pipeline stores five entity types. JSON Schemas are the source of truth in [ ├── tasks/ │ ├── T0001-*.json # Task │ └── T0001-*.md # rendered, derived + ├── outcomes/ + │ ├── O0001-*.json # Outcome + │ └── O0001-*.md # rendered, derived └── index.html # rendered, derived ``` @@ -107,6 +111,39 @@ A beads-style work unit. | `external_ref` | object? | Set at handoff. `{ system: "linear" \| "github" \| "plane" \| "jira" \| "other", id, url? }`. | | `created_at`, `updated_at` | ISO datetime | | +## Outcome (`dr/outcomes/.json`) + +A post-handoff observation that an accepted decision did or didn't hold up. Outcomes close the feedback loop between a DR and reality. + +| Field | Type | Notes | +|---|---|---| +| `id` | `"O0001-slug"` | Composite identifier. | +| `number` | integer ≥1 | Monotonic per project (separate counter from decisions/tasks). | +| `slug` | string | Kebab-case. | +| `decision_id` | DecisionId | The accepted decision this observes. | +| `status` | enum | `pending \| validated \| invalidated \| inconclusive`. | +| `observation` | string | Free-form prose. | +| `metric` | string? | Optional structured metric, e.g., `"p99 latency 290ms"`. | +| `evidence` | string[] | URLs, dashboards, file refs. | +| `recorded_by` | `"agent" \| "human"` | | +| `recorded_actor` | string? | | +| `recorded_at`, `updated_at` | ISO datetime | | +| `tags` | string[] | | + +Outcomes are first-class entities, never nested in Decisions. This keeps decisions immutable after sign-off and supports many outcomes per decision. + +## EmbeddingCache (`.dr/cache/embeddings.json`) + +Vector cache for `dr_search_decisions`. Written on every `dr_accept_decision`. Hash-keyed so unchanged decisions skip re-embedding. + +| Field | Type | Notes | +|---|---|---| +| `version` | `"1"` | Schema version. | +| `default_model` | string | The embedding model used when entries were written. | +| `entries` | record | Keyed by `decision_id` → `{ decision_id, model, dim, hash, vector, embedded_at }`. | + +The cache is `.dr/cache/`-resident and gitignored — it's a derived artifact, regenerable by `dr_reindex_embeddings`. + ## PipelineState (`.dr/state.json`) Internal pipeline bookkeeping. Never edit by hand. @@ -117,7 +154,7 @@ Internal pipeline bookkeeping. Never edit by hand. | `project_id` | string | Matches `project.json.id`. | | `phase` | phase enum | Mirrors `project.status` but the pipeline writes this. | | `effective_gate_config` | object | Materialized preset + overrides. | -| `next_decision_seq`, `next_task_seq` | integer ≥1 | Monotonic counters. | +| `next_decision_seq`, `next_task_seq`, `next_outcome_seq` | integer ≥1 | Monotonic counters. | | `pending_questions` | array | Open questions the agent surfaced. | | `gate_failures` | array | History of failed advance attempts (for debugging). | | `last_event_at`, `last_render_at` | ISO datetime? | | @@ -132,14 +169,14 @@ One JSON line per pipeline action. Append-only audit log. | `actor` | `"agent" \| "human" \| "system"` | | | `actor_name` | string? | | | `kind` | enum | See below. | -| `entity_kind` | `"project" \| "decision" \| "task" \| "phase" \| "question"`? | | +| `entity_kind` | `"project" \| "decision" \| "task" \| "outcome" \| "phase" \| "question"`? | | | `entity_id` | string? | | | `payload` | object? | Event-specific. | | `correlation_id` | string? | Groups related events. | ### Event kinds -`project_initialized`, `phase_advanced`, `phase_advance_blocked`, `scope_updated`, `decision_proposed`, `decision_updated`, `decision_reviewed`, `decision_accepted`, `decision_rejected`, `task_proposed`, `task_updated`, `task_status_changed`, `graph_validated`, `gate_check_passed`, `gate_check_failed`, `question_asked`, `question_answered`, `seed_loaded`, `render_run`, `export_started`, `export_completed`, `export_failed`, `sign_off_recorded`. +`project_initialized`, `phase_advanced`, `phase_advance_blocked`, `scope_updated`, `decision_proposed`, `decision_updated`, `decision_reviewed`, `decision_accepted`, `decision_rejected`, `task_proposed`, `task_updated`, `task_status_changed`, `graph_validated`, `gate_check_passed`, `gate_check_failed`, `question_asked`, `question_answered`, `seed_loaded`, `render_run`, `export_started`, `export_completed`, `export_failed`, `sign_off_recorded`, `outcome_recorded`, `outcome_status_changed`, `outcome_updated`, `embeddings_indexed`, `embeddings_index_failed`. ## ID conventions @@ -147,6 +184,7 @@ One JSON line per pipeline action. Append-only audit log. |---|---|---| | Decision | `<4-digit>-` | `0003-define-the-agent-action-contract` | | Task | `T<4-digit>-` | `T0006-implement-the-rate-limiter` | +| Outcome | `O<4-digit>-` | `O0001-latency-held-up` | | Project | kebab-slug | `contact-list-deduper` | Slugs are 2–64 chars, lower-case alphanumerics + dashes, no leading/trailing dash. diff --git a/docs/reference/mcp-tools.md b/docs/reference/mcp-tools.md index e5a1c47..c2a0c9a 100644 --- a/docs/reference/mcp-tools.md +++ b/docs/reference/mcp-tools.md @@ -160,11 +160,82 @@ Instantiate a seed as a `proposed` DR. Pre-fills positions, assumptions, constra | `depends_on` | string[] | Decision IDs this DR depends on. | | `tags` | string[] | | +## Outcome tools + +Outcomes close the feedback loop between an accepted decision and what actually happened in practice. They live alongside decisions (never nested inside) so decisions remain immutable after sign-off. + +### `dr_record_outcome` + +Record an observed outcome for an accepted decision. Requires the project to be in `handed-off` status — pre-handoff outcomes are nonsensical. + +| Input | Type | Notes | +|---|---|---| +| `decision_id` | string | Must point at an `accepted` decision. | +| `observation` | string | Free-form prose of what was observed. | +| `status` | `"pending" \| "validated" \| "invalidated" \| "inconclusive"` | Default `pending`. | +| `metric` | string? | Optional structured metric, e.g., `"p99 latency 290ms"`. | +| `evidence` | string[] | URLs, file refs, dashboards. | +| `tags` | string[] | | +| `slug` | string? | Defaults to a slug derived from the observation. | +| `recorded_by` | `"agent" \| "human"` | Default `human`. | +| `recorded_actor` | string? | | + +Returns: `{ outcome }`. Emits `outcome_recorded` and bumps `state.next_outcome_seq`. + +### `dr_set_outcome_status` + +Transition an outcome's status (e.g., `pending` → `validated`). Emits `outcome_status_changed` unless the status is unchanged (in which case the call is a no-op and returns `{ unchanged: true }`). + +### `dr_update_outcome` + +Patch `observation`, `metric`, `evidence`, `tags`. Use `dr_set_outcome_status` for status transitions. + +### `dr_list_outcomes` + +Filter by `decision_id` and/or `status[]`. Returns summaries. + +### `dr_get_outcome` + +Fetch the full outcome by id. + +## Search tools + +Semantic search over decisions powered by an embeddings cache. Falls back to substring match when embeddings are unavailable so the tool always returns something. + +### `dr_search_decisions` + +Find prior decisions similar to a free-form topic. Used by the deciding agent for **read-before-write** retrieval — before proposing a new DR, ask whether something similar already exists. + +| Input | Type | Notes | +|---|---|---| +| `query` | string | Free-form text describing the topic. | +| `limit` | integer | Default 5, max 50. | +| `min_score` | number | Default 0.5. Only applied to semantic results. | +| `status` | array of decision statuses | Default `["accepted"]`. | + +Returns one of: + +- `{ mode: "semantic", model, results: [{ id, title, status, summary, selected_position, score }], warnings }` +- `{ mode: "substring", results: [...], warnings }` — when embeddings are disabled, the cache is empty, or the cache model doesn't match the current `OPENAI_EMBEDDING_MODEL`. +- `{ mode: "empty", results: [], warnings }` — when no decisions match the status filter. + +### `dr_reindex_embeddings` + +Re-embed every accepted decision. Useful after switching `OPENAI_EMBEDDING_MODEL`, after a manual cache wipe, or to backfill decisions that were accepted before embeddings were enabled. + +| Input | Type | Notes | +|---|---|---| +| `force` | boolean | Default `false`. When `true`, wipes the cache first to force a full re-embed. | + +Returns counts: `{ accepted_total, indexed, skipped, failed, failures, model }`. Fails fast when `OPENAI_EMBEDDING_MODEL=none`. + +> **Env var.** `OPENAI_EMBEDDING_MODEL` selects the embedding model (default `text-embedding-3-small`). Set it to `"none"` to disable embeddings entirely — search will use substring fallback only, and `dr_accept_decision` will skip indexing. + ## Render ### `dr_render` -Regenerate Markdown + `index.html` from JSON. Idempotent. +Regenerate Markdown + `index.html` from JSON. Idempotent. Includes outcomes when present. ## Handoff diff --git a/schemas/README.md b/schemas/README.md index 9f33e11..58d7de4 100644 --- a/schemas/README.md +++ b/schemas/README.md @@ -7,6 +7,7 @@ JSON Schema (draft-07) definitions for the data emitted by the decision-record p | [`project.schema.json`](project.schema.json) | `dr/project.json` (in target repo) | `dr_init`, scope updates during scoping phase | | [`decision.schema.json`](decision.schema.json) | `dr/decisions/.json` (one per DR) | `dr_propose_decision`, `dr_update_decision`, `dr_review_decision`, `dr_accept_decision` | | [`task.schema.json`](task.schema.json) | `dr/tasks/.json` (one per task) | `dr_propose_task`, `dr_update_task` | +| [`outcome.schema.json`](outcome.schema.json) | `dr/outcomes/.json` (one per post-handoff observation) | `dr_record_outcome`, `dr_update_outcome`, `dr_set_outcome_status` | | [`state.schema.json`](state.schema.json) | `.dr/state.json` (internal/derived) | All pipeline tools; never edited by hand | | [`event.schema.json`](event.schema.json) | `.dr/events.jsonl` (one event per line) | Every tool appends; never rewritten | @@ -17,7 +18,8 @@ target-repo/ ├── .dr/ │ ├── state.json # PipelineState │ ├── events.jsonl # Append-only event log (each line ⇒ Event) -│ └── cache/ # Render artifacts, seed snapshots (regeneratable) +│ └── cache/ +│ └── embeddings.json # EmbeddingCache (regeneratable via dr_reindex_embeddings) └── dr/ ├── project.json # Project (MVP manifest) ├── decisions/ @@ -26,6 +28,9 @@ target-repo/ ├── tasks/ │ ├── T0001-bootstrap.json # Task │ └── T0001-bootstrap.md # Rendered view (regeneratable) + ├── outcomes/ + │ ├── O0001-*.json # Outcome (post-handoff observation) + │ └── O0001-*.md # Rendered view (regeneratable) └── index.html # Rendered project overview (regeneratable) ``` diff --git a/schemas/event.schema.json b/schemas/event.schema.json index 3c7a745..62b3269 100644 --- a/schemas/event.schema.json +++ b/schemas/event.schema.json @@ -40,12 +40,17 @@ "export_started", "export_completed", "export_failed", - "sign_off_recorded" + "sign_off_recorded", + "outcome_recorded", + "outcome_status_changed", + "outcome_updated", + "embeddings_indexed", + "embeddings_index_failed" ] }, "entity_kind": { "type": "string", - "enum": ["project", "decision", "task", "phase", "question"] + "enum": ["project", "decision", "task", "outcome", "phase", "question"] }, "entity_id": { "type": "string" }, "payload": { diff --git a/schemas/outcome.schema.json b/schemas/outcome.schema.json new file mode 100644 index 0000000..542771d --- /dev/null +++ b/schemas/outcome.schema.json @@ -0,0 +1,68 @@ +{ + "$schema": "http://json-schema.org/draft-07/schema#", + "$id": "https://github.com/protoLabsAI/decision-record/schemas/outcome.schema.json", + "title": "Outcome", + "description": "A post-handoff observation linked to an accepted Decision. Records whether the decision held up in practice. Closes the feedback loop between a DR and reality.", + "type": "object", + "required": [ + "id", + "number", + "slug", + "decision_id", + "status", + "observation", + "recorded_at", + "updated_at" + ], + "properties": { + "id": { + "type": "string", + "pattern": "^O[0-9]{4}-[a-z0-9][a-z0-9-]{0,62}[a-z0-9]$", + "description": "Composite identifier — 'O' prefix + zero-padded number + slug, e.g., 'O0001-latency-held-up'." + }, + "number": { + "type": "integer", + "minimum": 1 + }, + "slug": { + "type": "string", + "pattern": "^[a-z0-9][a-z0-9-]{0,62}[a-z0-9]$" + }, + "decision_id": { + "type": "string", + "pattern": "^[0-9]{4}-[a-z0-9][a-z0-9-]{0,62}[a-z0-9]$", + "description": "Forward link to the Decision this outcome observes. Outcomes are not stored on the Decision itself." + }, + "status": { + "type": "string", + "enum": ["pending", "validated", "invalidated", "inconclusive"], + "description": "Whether the decision held up. 'pending' = recorded but not yet evaluated." + }, + "observation": { + "type": "string", + "minLength": 1, + "description": "Free-form prose describing what was observed." + }, + "metric": { + "type": "string", + "description": "Optional structured metric, e.g., 'p99 latency 320ms' or 'churn rate +3%'." + }, + "evidence": { + "type": "array", + "items": { "type": "string" }, + "description": "Links, file references, or other supporting artifacts." + }, + "recorded_by": { + "type": "string", + "enum": ["agent", "human"] + }, + "recorded_actor": { "type": "string" }, + "recorded_at": { "type": "string", "format": "date-time" }, + "updated_at": { "type": "string", "format": "date-time" }, + "tags": { + "type": "array", + "items": { "type": "string" } + } + }, + "additionalProperties": false +} diff --git a/schemas/state.schema.json b/schemas/state.schema.json index f49ed14..ee08ba7 100644 --- a/schemas/state.schema.json +++ b/schemas/state.schema.json @@ -44,6 +44,7 @@ }, "next_decision_seq": { "type": "integer", "minimum": 1 }, "next_task_seq": { "type": "integer", "minimum": 1 }, + "next_outcome_seq": { "type": "integer", "minimum": 1 }, "pending_questions": { "type": "array", "description": "Open questions the agent has surfaced and is waiting on the human for.", diff --git a/server/src/cli/agents/deciding.ts b/server/src/cli/agents/deciding.ts index 08a9c17..0b7687c 100644 --- a/server/src/cli/agents/deciding.ts +++ b/server/src/cli/agents/deciding.ts @@ -9,14 +9,19 @@ Your one job: identify every significant decision this project needs to make, pr Workflow: 1. Call \`dr_status\` to read the project's current state, including scope and any pre-existing decisions. 2. Call \`dr_list_decisions\` to see what's already on file. -3. For each project, identify 3-8 significant decisions (or however many the gate requires — see status.effective_gate_config.min_decisions). Significant means: would otherwise be re-litigated, has multiple defensible options, and load-bearing for the MVP. +3. **Identify prospective decision topics.** For each project, identify 3-8 significant decisions (or however many the gate requires — see status.effective_gate_config.min_decisions). Significant means: would otherwise be re-litigated, has multiple defensible options, and load-bearing for the MVP. - For each decision: - a. **Check the seed library first.** Call \`dr_seed_search\` with a query relevant to the decision topic (e.g., 'language', 'data store', 'auth'). If a seed matches, use \`dr_seed_load\` to instantiate it — this gives you well-thought-out starter content. +4. **Read before write.** For each prospective topic, call \`dr_search_decisions\` with a query describing the topic (e.g., "primary data store", "auth provider", "runtime language"). Inspect the results: + - If a similar **accepted** decision exists with score ≥ 0.85 AND its context is genuinely transferable, do NOT propose a new DR. Instead, capture the reference in the next DR's \`related_decisions\` field when relevant, and note in your final summary that you reused the prior decision. + - If a similar decision exists but a new one is still warranted (different context, different constraints), proceed but cite it via \`related_decisions\` so reviewers see the connection. + - Surface any score-≥0.85 results in your final summary so reviewers can sanity-check the call. + + For each remaining decision: + a. **Check the seed library.** Call \`dr_seed_search\` with a query relevant to the decision topic (e.g., 'language', 'data store', 'auth'). If a seed matches, use \`dr_seed_load\` to instantiate it — this gives you well-thought-out starter content. b. **If no seed matches**, call \`dr_propose_decision\` with title, issue, 2-4 positions (each with title, description, pros, cons), assumptions, and constraints. c. **Pick a position.** Call \`dr_update_decision\` with selected_position (matching one of the position titles) and a 1-2 sentence argument for why it wins. -4. After each decision is selected, the orchestrator runs antagonistic review. If a review blocks, you may be called again to revise — but for now, don't accept anything. +5. After each decision is selected, the orchestrator runs antagonistic review. If a review blocks, you may be called again to revise — but for now, don't accept anything. Constraints: - Stay inside the project's scope. Don't propose decisions about out-of-scope capabilities. diff --git a/server/src/cli/index.ts b/server/src/cli/index.ts index b3c1d90..d23506a 100644 --- a/server/src/cli/index.ts +++ b/server/src/cli/index.ts @@ -52,6 +52,8 @@ Environment: OPENAI_API_KEY Required unless --api-key is passed. OPENAI_BASE_URL Optional. Set for OpenRouter, vLLM, Ollama, LiteLLM, etc. OPENAI_MODEL Optional. Default model name. + OPENAI_EMBEDDING_MODEL Optional. Default text-embedding-3-small. Set to "none" to disable + semantic search (falls back to substring match). LINEAR_API_KEY Optional. Enables Linear handoff target. LINEAR_TEAM_ID Optional. Pre-fills the Linear team ID prompt. diff --git a/server/src/embeddings/client.ts b/server/src/embeddings/client.ts new file mode 100644 index 0000000..140b675 --- /dev/null +++ b/server/src/embeddings/client.ts @@ -0,0 +1,79 @@ +import OpenAI from "openai"; + +export interface EmbedConfig { + enabled: boolean; + model: string; + apiKey?: string; + baseURL?: string; +} + +const DEFAULT_MODEL = "text-embedding-3-small"; + +export function resolveEmbedConfig(overrides: Partial = {}): EmbedConfig { + const rawModel = overrides.model ?? process.env.OPENAI_EMBEDDING_MODEL ?? DEFAULT_MODEL; + if (rawModel === "none" || rawModel === "") { + return { enabled: false, model: rawModel }; + } + return { + enabled: true, + model: rawModel, + apiKey: overrides.apiKey ?? process.env.OPENAI_API_KEY, + baseURL: overrides.baseURL ?? process.env.OPENAI_BASE_URL, + }; +} + +let cached: OpenAI | null = null; +let override: OpenAI | null = null; + +export function getDefaultEmbedClient(cfg: EmbedConfig): OpenAI { + if (override) return override; + if (cached) return cached; + cached = new OpenAI({ + apiKey: cfg.apiKey, + baseURL: cfg.baseURL, + }); + return cached; +} + +export function resetDefaultEmbedClient(): void { + cached = null; + override = null; +} + +/** Test-only — replace the embed client with a mock. Call resetDefaultEmbedClient() to clear. */ +export function setEmbedClientForTesting(client: OpenAI | null): void { + override = client; +} + +export async function embed( + client: OpenAI, + cfg: EmbedConfig, + input: string +): Promise { + if (!cfg.enabled) { + throw new Error("embedding disabled (OPENAI_EMBEDDING_MODEL=none)"); + } + const resp = await client.embeddings.create({ model: cfg.model, input }); + const data = resp.data?.[0]; + if (!data?.embedding) { + throw new Error("embedding response missing vector"); + } + return data.embedding; +} + +export function cosineSim(a: number[], b: number[]): number { + if (a.length === 0 || b.length === 0 || a.length !== b.length) return 0; + let dot = 0; + let na = 0; + let nb = 0; + for (let i = 0; i < a.length; i++) { + const ai = a[i] ?? 0; + const bi = b[i] ?? 0; + dot += ai * bi; + na += ai * ai; + nb += bi * bi; + } + const denom = Math.sqrt(na) * Math.sqrt(nb); + if (denom === 0) return 0; + return dot / denom; +} diff --git a/server/src/embeddings/index.ts b/server/src/embeddings/index.ts new file mode 100644 index 0000000..f884d0f --- /dev/null +++ b/server/src/embeddings/index.ts @@ -0,0 +1,84 @@ +import OpenAI from "openai"; +import { Decision, EmbeddingCache, EmbeddingCacheEntry } from "../schemas/index.js"; +import { Store } from "../storage/store.js"; +import { nowIso } from "../util.js"; +import { + EmbedConfig, + embed, + getDefaultEmbedClient, + resolveEmbedConfig, +} from "./client.js"; +import { composeEmbeddingText, sha256Hash } from "./text.js"; + +export type IndexResult = + | { status: "indexed"; entry: EmbeddingCacheEntry } + | { status: "skipped"; reason: "disabled" | "unchanged" } + | { status: "failed"; error: string }; + +export async function indexDecision( + store: Store, + decision: Decision, + options: { + config?: EmbedConfig; + client?: OpenAI; + } = {} +): Promise { + const cfg = options.config ?? resolveEmbedConfig(); + if (!cfg.enabled) { + return { status: "skipped", reason: "disabled" }; + } + + const text = composeEmbeddingText(decision); + const hash = sha256Hash(text); + + let cache = await store.readEmbeddings(); + const existing = cache?.entries[decision.id]; + if (existing && existing.hash === hash && existing.model === cfg.model) { + return { status: "skipped", reason: "unchanged" }; + } + + try { + const client = options.client ?? getDefaultEmbedClient(cfg); + const vector = await embed(client, cfg, text); + const entry: EmbeddingCacheEntry = { + decision_id: decision.id, + model: cfg.model, + dim: vector.length, + hash, + vector, + embedded_at: nowIso(), + }; + const newCache: EmbeddingCache = { + version: "1", + default_model: cfg.model, + entries: { + ...(cache?.entries ?? {}), + [decision.id]: entry, + }, + }; + await store.writeEmbeddings(newCache); + await store.appendEvent({ + at: entry.embedded_at, + actor: "agent", + kind: "embeddings_indexed", + entity_kind: "decision", + entity_id: decision.id, + payload: { model: cfg.model, dim: vector.length }, + }); + return { status: "indexed", entry }; + } catch (err) { + const msg = err instanceof Error ? err.message : String(err); + await store.appendEvent({ + at: nowIso(), + actor: "agent", + kind: "embeddings_index_failed", + entity_kind: "decision", + entity_id: decision.id, + payload: { error: msg, model: cfg.model }, + }); + return { status: "failed", error: msg }; + } +} + +export { composeEmbeddingText, sha256Hash } from "./text.js"; +export { cosineSim, resolveEmbedConfig, type EmbedConfig } from "./client.js"; diff --git a/server/src/embeddings/text.ts b/server/src/embeddings/text.ts new file mode 100644 index 0000000..cb16d49 --- /dev/null +++ b/server/src/embeddings/text.ts @@ -0,0 +1,28 @@ +import { createHash } from "node:crypto"; +import { Decision } from "../schemas/index.js"; + +export function composeEmbeddingText(decision: Decision): string { + const parts: string[] = []; + parts.push(`Title: ${decision.title}`); + if (decision.summary) parts.push(`Summary: ${decision.summary}`); + if (decision.issue) parts.push(`Issue: ${decision.issue}`); + if (decision.argument) parts.push(`Argument: ${decision.argument}`); + if (decision.selected_position) { + parts.push(`Selected: ${decision.selected_position}`); + } + if (decision.positions.length > 0) { + const titles = decision.positions.map((p) => p.title).join("; "); + parts.push(`Positions: ${titles}`); + } + if (decision.implications.length > 0) { + parts.push(`Implications: ${decision.implications.join("; ")}`); + } + if (decision.tags.length > 0) { + parts.push(`Tags: ${decision.tags.join(", ")}`); + } + return parts.join("\n"); +} + +export function sha256Hash(text: string): string { + return createHash("sha256").update(text).digest("hex"); +} diff --git a/server/src/render/html.ts b/server/src/render/html.ts index 279da50..53d2239 100644 --- a/server/src/render/html.ts +++ b/server/src/render/html.ts @@ -1,4 +1,4 @@ -import { Decision, Project, Task } from "../schemas/index.js"; +import { Decision, Outcome, Project, Task } from "../schemas/index.js"; import { escapeHtml } from "../util.js"; const STYLE = `:root { @@ -19,6 +19,10 @@ const STYLE = `:root { --task-done: #34d399; --task-blocked: #f87171; --task-deferred: #c084fc; + --outcome-pending: #fbbf24; + --outcome-validated: #34d399; + --outcome-invalidated: #f87171; + --outcome-inconclusive: #9ca3af; } * { box-sizing: border-box; } body { font-family: -apple-system, BlinkMacSystemFont, "Segoe UI", Roboto, sans-serif; line-height: 1.5; margin: 0; padding: 2rem; background: var(--bg); color: var(--fg); } @@ -40,6 +44,10 @@ h1, h2, h3 { margin-top: 1.5rem; } .pill-task-done { background: var(--task-done); } .pill-task-blocked { background: var(--task-blocked); } .pill-task-deferred { background: var(--task-deferred); } +.pill-outcome-pending { background: var(--outcome-pending); } +.pill-outcome-validated { background: var(--outcome-validated); } +.pill-outcome-invalidated { background: var(--outcome-invalidated); } +.pill-outcome-inconclusive { background: var(--outcome-inconclusive); } table { width: 100%; border-collapse: collapse; margin-top: 1rem; background: white; box-shadow: 0 1px 3px rgba(0,0,0,0.05); } th, td { text-align: left; padding: 0.5rem 0.75rem; border-bottom: 1px solid var(--border); font-size: 0.9rem; vertical-align: top; } th { background: #f3f4f6; font-weight: 600; font-size: 0.8rem; text-transform: uppercase; letter-spacing: 0.04em; color: var(--muted); } @@ -62,12 +70,16 @@ a:hover { text-decoration: underline; } export function renderIndexHtml( project: Project, decisions: Decision[], - tasks: Task[] + tasks: Task[], + outcomes: Outcome[] = [] ): string { const decisionsByStatus = groupBy(decisions, (d) => d.status); const tasksByStatus = groupBy(tasks, (t) => t.status); + const outcomesByStatus = groupBy(outcomes, (o) => o.status); const sortedDecisions = [...decisions].sort((a, b) => a.number - b.number); const sortedTasks = [...tasks].sort((a, b) => a.priority.localeCompare(b.priority) || a.number - b.number); + const sortedOutcomes = [...outcomes].sort((a, b) => a.number - b.number); + const decisionsById = new Map(decisions.map((d) => [d.id, d])); return ` @@ -89,6 +101,7 @@ export function renderIndexHtml( Updated: ${escapeHtml(project.updated_at)} Decisions: ${decisions.length} (${decisionsByStatus.get("accepted")?.length ?? 0} accepted) Tasks: ${tasks.length} (${tasksByStatus.get("done")?.length ?? 0} done) + Outcomes: ${outcomes.length} (${outcomesByStatus.get("validated")?.length ?? 0} validated) @@ -101,7 +114,10 @@ export function renderIndexHtml( ${renderDecisionTable(sortedDecisions)}

Task graph

- ${renderTaskTable(sortedTasks, new Map(decisions.map((d) => [d.id, d])))} + ${renderTaskTable(sortedTasks, decisionsById)} + +

Outcomes

+ ${renderOutcomeTable(sortedOutcomes, decisionsById)}