docs: CI workflow, architecture doc, README status update

.github/workflows/tests.yml

@@ -0,0 +1,23 @@
+name: Tests
+
+on:
+  push:
+    branches: [main]
+  pull_request:
+    branches: [main]
+
+jobs:
+  test:
+    runs-on: ubuntu-latest
+    steps:
+      - uses: actions/checkout@v4
+
+      - uses: actions/setup-python@v5
+        with:
+          python-version: "3.12"
+
+      - name: Install project (with dev dependencies)
+        run: pip install -e ".[dev]"
+
+      - name: Run tests
+        run: pytest -v

README.md

@@ -1,5 +1,8 @@
 # CRM Dedup/Hygiene Toolkit
 
+[![Tests](https://github.com/codeapplied/crm-dedup-hygiene-toolkit/actions/workflows/tests.yml/badge.svg)](https://github.com/codeapplied/crm-dedup-hygiene-toolkit/actions/workflows/tests.yml)
+[![License: MIT](https://img.shields.io/badge/License-MIT-yellow.svg)](LICENSE)
+
 Finds and safely merges duplicate organizations and contacts in a CRM, with a
 full audit trail — rebuilt as a generalized, open-source system.
 
@@ -9,9 +12,11 @@ branding.
 
 ## Status
 
-🚧 Early development. DB models, config loading, and CLI skeleton are in
-place. Duplicate discovery and merge execution are not built yet — see
-[open issues](https://github.com/codeapplied/crm-dedup-hygiene-toolkit/issues).
+Core pipeline is built and tested: duplicate discovery, score-based primary
+selection, and merge execution (dry-run by default, `--execute` to actually
+merge) all work end-to-end against both the bundled sandbox demo and a real
+Pipedrive backend. See [open issues](https://github.com/codeapplied/crm-dedup-hygiene-toolkit/issues)
+for what's next.
 
 ## Architecture
 
@@ -22,9 +27,12 @@ member → score each member by a weighted combination of linked-record counts
 → the highest-scoring member becomes "primary" → dry run prints the exact
 merge plan; only an explicit `--execute` performs the merge → call the CRM's
 native merge endpoint, then post one audit note on the primary summarizing
-exactly what was absorbed. Every stage is logged to `MergeLog`, which the ops
+exactly what was absorbed. Every run is logged to `MergeLog`, which the ops
 CLI reads for history.
 
+See [docs/ARCHITECTURE.md](docs/ARCHITECTURE.md) for the full data-flow
+diagram and design rationale.
+
 ## Setup
 
 ```
@@ -35,12 +43,23 @@ cp config/rules.example.yaml config/rules.yaml
 crmdedup init
 ```
 
+Leave `.env`'s Pipedrive fields blank to run against the bundled zero-network
+sandbox demo — no credentials needed to try it.
+
 ## CLI
 
 - `crmdedup init` — create the database
 - `crmdedup status` — recent merge runs
 - `crmdedup rules` — show the loaded scoring weights and known-false-positive skip list
-- `crmdedup discover` — find duplicate groups (not yet implemented)
+- `crmdedup discover` — find duplicate groups (dry run — plans only, writes nothing to the CRM)
+- `crmdedup discover --execute` — actually perform each planned merge
+
+## Development
+
+```
+uv pip install -e ".[dev]"
+pytest -v
+```
 
 ## License
 

docs/ARCHITECTURE.md

@@ -0,0 +1,78 @@
+# Architecture
+
+## Overview
+
+The toolkit finds and safely merges duplicate CRM records in three stages:
+**discover** (group candidates by a normalized key), **score** (pick which
+member of each group survives), and **merge** (actually combine them, with
+an audit trail). Every mutating step is dry-run by default — nothing is
+written to the CRM unless `--execute` is passed explicitly.
+
+## Data flow
+
+```mermaid
+flowchart TD
+    A[CRMClient.list_organizations / list_contacts] --> B[normalize_domain / lowercase email]
+    B --> C[group by key]
+    C --> D{group size >= 2?}
+    D -- no --> Z[discarded, not a duplicate]
+    D -- yes --> E{in skip_pairs?}
+    E -- yes --> Y[skipped, known false positive]
+    E -- no --> F[get_related_counts per member]
+    F --> G[score = weighted sum of counts]
+    G --> H[pick_primary: richest wins, lowest-id tiebreak]
+    H --> I{--execute?}
+    I -- no --> J[write MergeLog, status=dry_run]
+    I -- yes --> K[merge_organizations / merge_contacts per absorbed id]
+    K --> L{merge succeeded?}
+    L -- yes --> M[add to absorbed_ok]
+    L -- no --> N[record failure, continue — never abort the group]
+    M --> O[post one audit note on primary]
+    N --> P[write MergeLog, status=merged/partial/failed]
+    O --> P
+```
+
+## Components
+
+- **`crm/base.py`** — the `CRMClient` interface every backend implements:
+  list organizations/contacts, get related-record counts, native merge for
+  both entity types, post an audit note.
+- **`crm/sandbox_client.py`** — zero-network default backend, seeded with
+  real duplicate groups so `discover`/`--execute` have something genuine to
+  find and merge without any credentials.
+- **`crm/pipedrive_client.py`** — a real Pipedrive API v1 implementation.
+  Honest about its own limitations (no native `notes_count` field, no live
+  account tested against in this project) rather than hiding them.
+- **`dedup.py`** — pure logic, no I/O: key normalization, grouping,
+  `skip_pairs` filtering, scoring, primary selection. Fully unit-testable
+  without a database or network.
+- **`pipeline/discover.py`** — orchestrates the CRM client and `dedup.py`
+  into one run: dry-run by default, `execute=True` actually merges.
+
+## Design decisions worth explaining
+
+- **Re-verification is the merge call itself, not a separate pre-check.**
+  Both CRM backends raise when a source record is already gone (merged by
+  someone else since the plan was made). Catching that at the merge call is
+  the re-verification checkpoint — a redundant separate "does it still
+  exist" check would just be the same information fetched twice.
+- **A failure on one group member never aborts the group.** If 3 of 5
+  duplicates merge cleanly and 2 fail, that's `status=partial` with 3
+  recorded as absorbed, not a rolled-back no-op.
+- **Score-based primary selection, one formula for both entity types.** A
+  weighted sum of linked-record counts (contacts, deals, notes, open
+  activities) — the richest record survives a merge, not "lowest id" or
+  "most recently touched."
+- **`skip_pairs` is data, not code.** Known false positives (a shared
+  mailbox, an ambiguous same-key pair) are listed in `config/rules.yaml`
+  and consulted by the matching logic, not hand-written as special cases
+  inside `dedup.py`.
+
+## What this deliberately does NOT do
+
+- No scheduled/automated merging — `discover --execute` is a command you
+  run, not a background job. Duplicate merging is high-stakes enough to
+  warrant a human deciding when to run it.
+- No fuzzy/partial-name matching — only exact-key duplicates (same
+  normalized domain, same email) are found automatically. Partial variants
+  need a human to catch, same as the reference system this was informed by.