diff --git a/CHANGELOG.md b/CHANGELOG.md index 7ce2cbb..1fc6fb8 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -2,6 +2,25 @@ Postmaster MCP follows Semantic Versioning for stable releases. Every stable release should update `VERSION`, this changelog, and publish an immutable Git tag/release named `vX.Y.Z`. +## 9.4.1 - 2026-08-20 + +### Added +- Query-time qualitative classification for link-click telemetry. Every returned `link` event can now include `provider_likelihood` (0-100), `provider_classification` (`likely_human`, `uncertain`, `likely_email_provider`, `known_email_proxy`), `provider_guess` (`google`, `microsoft`, `yahoo`, `other` or null) and human-readable `classification_reasons`. +- Combined-evidence heuristic model: explicit `GoogleImageProxy`/Gmail proxy signatures are treated as known proxy evidence; near-simultaneous second requests on the same `delivery_id + link_id`, changed fingerprints, country/browser/User-Agent changes and proxy source metadata increase provider likelihood, while a fingerprint seen consistently across multiple links in the same delivery lowers it. +- Qualitative unique-click summary fields including `likely_provider_unique_clicks`, `likely_human_or_unclassified_unique_clicks`, `uncertain_unique_clicks`, provider suspects and potential-provider share while preserving the original fingerprint unique count beside them. +- Tracking dashboard qualitative summary plus per-link-event provider score, classification, provider guess and reasons. +- `build_status.provider_qualitative_classification=true` and `tracking_status.link_tracking.provider_classification_query_time=true` capability reporting. +- Ground-truth regression fixtures for the observed Gmail duplicate-fetch pattern, stable human fingerprints across multiple links, same-fingerprint Libero-style clicks and explicit `GoogleImageProxy` traffic. + +### Changed +- Provider classification is recalculated from stored click evidence on every query instead of being persisted as authoritative database state, so historical events automatically benefit from future heuristic refinements. +- `get_tracking_summary`, `get_tracking_campaign` and `list_tracking_events` expose the qualitative interpretation layer without removing or renaming the v9.4 analytics fields. + +### Compatibility / deployment +- The v9.4 unique-click definition remains exactly `delivery_id + link_id + client_fingerprint`; suspicious events are never deleted, hidden or rewritten in `tracking_clicks`. +- No tracking schema migration is required and the v9.3/v9.4 single-YAML Portainer bootstrap remains unchanged. +- No new public HTTP endpoint is introduced. Cloudflare Access public bypass requirements remain `/track/open/*`, `/api/amp/*` and `/t/c/*` according to the features in use. `/files/*` remains the separate pre-existing signed file-handoff concern. + ## 9.4.0 - 2026-08-19 ### Added @@ -95,4 +114,4 @@ Postmaster MCP follows Semantic Versioning for stable releases. Every stable rel - CI coverage for runtime import, bootstrap, model provisioning, MIME regressions and knowledge operations. ### Changed -- Public project naming and configuration became provider-agnostic. +- Public project naming and configuration became provider-agnostic. \ No newline at end of file diff --git a/VERSION b/VERSION index 8148c55..ccfb75e 100644 --- a/VERSION +++ b/VERSION @@ -1 +1 @@ -9.4.0 +9.4.1 diff --git a/docs/LINK_TRACKING.md b/docs/LINK_TRACKING.md index 8ac5d3e..da6aefd 100644 --- a/docs/LINK_TRACKING.md +++ b/docs/LINK_TRACKING.md @@ -1,6 +1,6 @@ -# Link tracking and clean Sent copies (v9.4) +# Link tracking, qualitative provider classification and clean Sent copies (v9.4 / v9.4.1) -Postmaster v9.4 adds per-link click telemetry to the existing per-recipient analytics pipeline. The existing open-tracking pixel remains unchanged: its route, event format, enrichment, client fingerprint, country/source, browser/OS parsing, campaign/delivery correlation and dashboard continue to work as before. +Postmaster v9.4 adds per-link click telemetry to the existing per-recipient analytics pipeline. v9.4.1 adds a second, explicitly qualitative interpretation layer that estimates whether a unique click context may have been generated by mailbox-provider infrastructure. The existing open-tracking pixel, raw click records and stable fingerprint unique-click definition remain unchanged. ## Recipient versus Sent architecture @@ -24,7 +24,7 @@ Link tracking follows the existing `track_opens` tracking opt-in (including the ## Public click endpoint -The only new public callback path required by v9.4 is: +The public click callback is: ```text GET /t/c/ @@ -34,6 +34,8 @@ The public base is resolved exactly like existing mail callbacks: `PUBLIC_EMAIL_ The token is random and opaque; it does not encode recipient data or the destination. `/t/c/` resolves a server-side `tracking_links` record, records a `link` event and issues a `302` to the stored `original_url`. Request/query parameters are never used as the destination, so `?url=https://...` cannot create an open redirect. Invalid tokens return `404` without a destination hint. +v9.4.1 does **not** add another public endpoint. Provider classification runs only when protected analytics/MCP/dashboard queries read already stored click evidence. + ## HTML rewriting Recipient HTML rewrites eligible `http://` and `https://` anchors only. Postmaster does not rewrite `mailto:`, `tel:`, `cid:`, `data:`, `javascript:`, `#fragment` or other non-web URLs. A URL already pointing to this deployment's `/t/c/` path is not wrapped twice. A legitimate third-party URL whose path happens to contain `/t/c/` remains eligible. @@ -50,23 +52,117 @@ Existing pixel tables are not replaced. v9.4 adds: The click fingerprint reuses the existing analytics HMAC derivation; raw IP is not stored. Country/source/browser/OS parsing reuses the pixel enrichment helpers. The browser field continues to contain the parsed browser/version label used by the existing pipeline. -## Unique click +v9.4.1 deliberately adds **no provider-classification columns** to `tracking_clicks`. The score is derived at read time, so old events automatically receive the current classifier and future heuristic improvements do not require rewriting historical data. + +## Stable unique click -A v9.4 unique click is: +The authoritative unique-click definition remains: ```text delivery_id + link_id + client_fingerprint ``` -If both IP and User-Agent are unavailable, the existing keyed fingerprint pipeline produces the stable HMAC of the empty pair; repeated unknown fetches for that delivery/link therefore collapse consistently. v9.4 does not classify human/bot/scanner clicks. User-Agent, source and fingerprint are retained for a future evidence-based classifier. +If both IP and User-Agent are unavailable, the existing keyed fingerprint pipeline produces the stable HMAC of the empty pair; repeated unknown fetches for that delivery/link therefore collapse consistently. + +The qualitative layer never removes a suspicious event from this count. A report can therefore show both: + +```text +unique_clicks: 24 +likely_provider_unique_clicks: 3 +likely_human_or_unclassified_unique_clicks: 21 +``` + +## v9.4.1 qualitative provider classification + +Every returned link event can include: + +```text +provider_likelihood: 0..100 +provider_classification: + likely_human + uncertain + likely_email_provider + known_email_proxy +provider_guess: + null + google + microsoft + yahoo + other +classification_reasons: + - "second request on same delivery/link after 3.64s" + - "fingerprint changed" + - "country changed IT → US" + - "browser changed Chrome 151.0.0.0 → Chrome 149.0.0.0" +``` + +This is a heuristic interpretation, not ground truth. Timing, geography, browser version or fingerprint change alone are not considered proof. Signals are combined. -## Analytics, MCP and dashboard +Current `heuristic-v1-query-time` behavior includes: + +- explicit `GoogleImageProxy`, Gmail image-proxy source or parsed Google Image Proxy evidence -> known proxy, score 100, provider `google`; +- a second request for the same `delivery_id + link_id` within 5 seconds with a different fingerprint -> strong provider signal; +- the same pattern within 15 seconds -> medium signal; +- fast country, browser and User-Agent changes -> additional independent signals; +- source metadata that explicitly indicates a proxy -> additional signal; +- a fingerprint that appears consistently across multiple different links in the same delivery -> negative provider evidence because it is consistent with one human browser context; +- a single anomaly-free click -> low provider score. + +Thresholds are intentionally simple and inspectable: + +```text +0..34 likely_human +35..69 uncertain +70..99 likely_email_provider +100 known_email_proxy when an explicit known signature matched +``` + +For provider-like rows without a vendor-specific signature, `provider_guess` may be `other`. Microsoft/Yahoo guesses are only hints when their names/signatures are present in the stored client metadata; they are not treated as known proxies by name alone. + +### Ground-truth fixture examples + +The v9.4.1 regression tests encode the first observed real-world patterns discussed during development: + +```text +Gmail A — IT / Chrome 151 low (~0-10) +Gmail A — US / Chrome 149, +3.637 s high (~85-100) +Same human fingerprint on another link low +Libero-style same fingerprint across links low +GoogleImageProxy explicit 100 +``` -Available analytics include total clicks, unique clicks, unique recipients, first/last click, campaign/delivery/link filtering, event detail, top links and destination host. +These fixtures are deliberately evidence-based and can be extended as more Gmail, Outlook, Yahoo, Libero and other mailbox-provider cases are observed. -Existing `tracking_status` and `get_tracking_campaign` are extended. v9.4 adds read-only `get_tracking_summary`, `list_tracking_links` and `list_tracking_events`. Listing/event tools never return opaque click tokens. +## Qualitative campaign summary -The existing pixel dashboard is preserved. v9.4 adds **Top links** and a unified **Tracking events** table distinguishing `pixel`, `amp_xhr` and `link`; link rows include label, `link_id`, destination host and position. +`get_tracking_summary` and the link-tracking block inside `get_tracking_campaign` retain the v9.4 counters and additionally expose a qualitative estimate such as: + +```text +Observed click events: 37 +Unique fingerprint clicks: 24 +Likely human: 18 +Uncertain: 3 +Likely provider/proxy: 3 +Potential provider share: 3 / 24 (12.5%) +Provider suspects: google 1, other 2 +Confidence: high +``` + +`confidence` describes the strength of the provider-like rows that were found; it does not mean Postmaster can prove which requests were human. + +## MCP and dashboard + +Existing `tracking_status` and `get_tracking_campaign` remain available. v9.4 adds read-only `get_tracking_summary`, `list_tracking_links` and `list_tracking_events`. + +In v9.4.1: + +- `get_tracking_summary` adds `qualitative_estimate`, `likely_provider_unique_clicks`, `likely_human_or_unclassified_unique_clicks`, `uncertain_unique_clicks`, `potential_provider_share`, `provider_suspects` and `provider_classification_model`; +- `get_tracking_campaign` receives those fields inside its existing link-tracking summary; +- `list_tracking_events` adds the provider fields to `link` rows while existing `pixel` and `amp_xhr` rows remain structurally compatible; +- `build_status.provider_qualitative_classification=true` reports the capability; +- the dashboard adds a qualitative summary and event-level provider score/classification/reasons beside the unchanged raw fingerprint data. + +Opaque click tokens are never returned by listing/event tools. ## Clean Sent behavior and historical finding @@ -76,41 +172,74 @@ v9.4 builds outbound and Sent MIME independently from the same canonical body/at This is the primary self-open/self-click defense; Postmaster does not guess sender identity from IP, country, User-Agent, browser or fingerprint. -## Cloudflare Access — manual deployment requirement +## Cloudflare Access — exactly which endpoints to open + +Keep Postmaster protected by default. The public callback surface for tracking/AMP is intentionally narrow. + +Required Access bypasses according to enabled features: -Keep Postmaster protected by default. Existing public callback paths remain: +```text +GET /track/open/* # open-tracking pixel +GET /api/amp/* # AMP callback/status path family; current route is /api/amp/status +GET /t/c/* # per-link click redirect +``` + +For a deployment using all three features, configure public Bypass rules for exactly: ```text /track/open/* /api/amp/* +/t/c/* ``` -v9.4 adds exactly one new required public path: +v9.4.1 introduces no additional anonymous route. + +Keep these protected by Cloudflare Access: ```text -/t/c/* +/ +/mcp +/dashboard/* +mail/account/task/memory/skill APIs +file-management administration +tracking analytics and MCP read tools +all other private/general application routes ``` -**Cloudflare Access must bypass `/t/c/*`.** Without this bypass a recipient reaches the Access login/challenge instead of Postmaster and the redirect fails. +Do not disable Access globally and do not expose the raw Docker port directly to the Internet. -Keep `/mcp`, dashboard/admin/private APIs, mail/task/memory/skill/file-management routes and tracking analytics protected. Do not disable Access globally. +### Separate signed file-handoff endpoint + +`GET`/`HEAD /files/{file_id}` is a pre-existing v9.3 signed file-handoff route. It is **not** a v9.4/v9.4.1 tracking bypass requirement. If the deployment intentionally relies on signed public HTTP file handoff, the operator must separately decide to make `/files/*` reachable through the external proxy policy; authorization is then provided by the route's expiry/HMAC signature. If the deployment uses MCP resource handoff instead, `/files/*` does not need to be opened merely for tracking. + +## Portainer deployment + +`postmaster-mcp.yml` remains unchanged. With: + +```yaml +POSTMASTER_VERSION: latest +POSTMASTER_CHECK_UPDATES_ON_START: "true" +``` -### Separate v9.3 file-handoff note +a stack/container restart after the stable v9.4.1 release is published is sufficient for the bootstrap to resolve and cache the new release. No compose variable, database migration or new volume is required for the qualitative classifier. -`/files/{file_id}` is a pre-existing v9.3 signed HTTP file-handoff route. v9.4 does not add `/files/*` to its Cloudflare bypass or change that policy. If a deployment intentionally uses signed HTTP file handoff, the operator must separately ensure that pre-existing route is reachable according to the deployment's proxy policy. +Cloudflare Access is external/manual configuration and is never changed automatically by the container. -## Live preflight +## Live preflight for v9.4.1 -After release/deploy and after the operator adds the `/t/c/*` bypass: +After release/deploy: -1. `build_status` reports `9.4.0`, `link_tracking=true`, `sent_copy_tracking_sanitized=true`. -2. Send a tracked email with at least two distinct HTTP/HTTPS URLs. -3. Recipient MIME contains the unchanged `/track/open/...gif` pixel and distinct `/t/c/` URLs. -4. Sent MIME contains neither recipient pixel nor `/t/c/`, and contains the original URLs. -5. Anonymous `GET https:///t/c/` reaches Postmaster without Cloudflare login, records a `link` event and redirects to the exact stored destination. -6. Invalid token plus `?url=https://evil.example/` does not redirect. -7. Link #1/#2 are independently visible in total/unique/top-link analytics with enrichment fields. -8. Clicking a URL extracted from Sent goes directly to the original destination and creates no Postmaster click event. -9. `/mcp`, dashboard and all other private paths remain protected by Cloudflare Access. +1. `build_status` reports `9.4.1`, `link_tracking=true`, `sent_copy_tracking_sanitized=true`, `provider_qualitative_classification=true`. +2. `tracking_status.link_tracking.provider_classification_query_time=true`. +3. Send a tracked email with at least two distinct HTTP/HTTPS URLs. +4. Recipient MIME contains the unchanged `/track/open/...gif` pixel and distinct `/t/c/` URLs. +5. Sent MIME contains neither recipient pixel nor `/t/c/`, and contains the original URLs. +6. Anonymous `GET https:///t/c/` reaches Postmaster without Cloudflare login, records a `link` event and redirects to the exact stored destination. +7. Invalid token plus `?url=https://evil.example/` does not redirect. +8. `list_tracking_events(event_type="link")` exposes provider fields while the underlying database schema still contains no provider score/classification columns. +9. `get_tracking_summary` returns both the unchanged `unique_clicks` and the new qualitative counters. +10. A known `GoogleImageProxy` fixture/query classifies as `known_email_proxy` with score 100; the Gmail duplicate-fetch ground-truth fixture remains high and the stable human fingerprint fixtures remain low. +11. Clicking a URL extracted from Sent goes directly to the original destination and creates no Postmaster click event. +12. `/mcp`, dashboard and all other private paths remain protected by Cloudflare Access. -The container cannot prove Cloudflare policy by itself. v9.4 is fully operational only after the external `/t/c/*` bypass is configured and the anonymous live redirect test succeeds. +The container cannot prove Cloudflare policy by itself. Tracking is fully operational only after the external callback bypasses required by the features in use are configured and anonymous live callback tests succeed. diff --git a/src/postmaster/link_tracking_queries.py b/src/postmaster/link_tracking_queries.py index 72e026c..7932b24 100644 --- a/src/postmaster/link_tracking_queries.py +++ b/src/postmaster/link_tracking_queries.py @@ -2,8 +2,42 @@ from typing import Any +from .provider_classification import classify_click_events, summarize_click_classification + class LinkTrackingQueriesMixin: + def _classification_events( + self, + *, + campaign_id: str | None = None, + delivery_id: str | None = None, + link_id: str | None = None, + account_id: str | None = None, + ) -> list[dict[str, Any]]: + clauses: list[str] = [] + params: list[Any] = [] + for column, value in ( + ("c.campaign_id", campaign_id), + ("c.delivery_id", delivery_id), + ("c.link_id", link_id), + ("c.account_id", account_id), + ): + if value: + clauses.append(f"{column}=?") + params.append(value) + where = (" WHERE " + " AND ".join(clauses)) if clauses else "" + with self._connect() as conn: + rows = conn.execute( + f""" + SELECT c.* + FROM tracking_clicks c + {where} + ORDER BY c.observed_at ASC,c.id ASC + """, + params, + ).fetchall() + return [dict(row) for row in rows] + def list_links( self, *, @@ -90,7 +124,7 @@ def list_click_events( """, params, ).fetchall() - return [dict(row) for row in rows] + return classify_click_events([dict(row) for row in rows]) def summary( self, @@ -134,6 +168,21 @@ def summary( "If IP and User-Agent are unavailable, the existing keyed fingerprint pipeline " "uses the stable HMAC of the empty pair, collapsing unknown repeat fetches for that delivery/link." ) + qualitative = summarize_click_classification( + self._classification_events( + campaign_id=campaign_id, + delivery_id=delivery_id, + link_id=link_id, + account_id=account_id, + ) + ) + out["qualitative_estimate"] = qualitative + out["likely_provider_unique_clicks"] = qualitative["likely_provider_unique_clicks"] + out["likely_human_or_unclassified_unique_clicks"] = qualitative["likely_human_or_unclassified_unique_clicks"] + out["uncertain_unique_clicks"] = qualitative["uncertain_unique_clicks"] + out["potential_provider_share"] = qualitative["potential_provider_share"] + out["provider_suspects"] = qualitative["provider_suspects"] + out["provider_classification_model"] = qualitative["classification_model"] return out def top_links( diff --git a/src/postmaster/provider_classification.py b/src/postmaster/provider_classification.py new file mode 100644 index 0000000..0fb75ea --- /dev/null +++ b/src/postmaster/provider_classification.py @@ -0,0 +1,234 @@ +from __future__ import annotations + +from collections import Counter, defaultdict +from datetime import datetime, timezone +from typing import Any + + +PROVIDER_CLASSIFICATIONS = { + "likely_human", + "uncertain", + "likely_email_provider", + "known_email_proxy", +} + + +def _parse_time(value: Any) -> float | None: + text = str(value or "").strip() + if not text: + return None + try: + parsed = datetime.fromisoformat(text.replace("Z", "+00:00")) + except ValueError: + return None + if parsed.tzinfo is None: + parsed = parsed.replace(tzinfo=timezone.utc) + return parsed.timestamp() + + +def _provider_hint(event: dict[str, Any]) -> str | None: + ua = str(event.get("user_agent") or "").lower() + source = str(event.get("client_source") or "").lower() + browser = str(event.get("browser") or "").lower() + combined = " ".join((ua, source, browser)) + if "googleimageproxy" in combined or "gmail_image_proxy" in combined: + return "google" + if any(token in combined for token in ("microsoft", "outlook", "office 365", "office/")): + return "microsoft" + if "yahoo" in combined: + return "yahoo" + + recipient = str(event.get("recipient") or "").strip().lower() + domain = recipient.rsplit("@", 1)[-1] if "@" in recipient else "" + if domain in {"gmail.com", "googlemail.com"}: + return "google" + if domain in {"outlook.com", "hotmail.com", "live.com", "msn.com"}: + return "microsoft" + if domain == "yahoo.com" or domain.startswith("yahoo."): + return "yahoo" + return None + + +def _known_proxy(event: dict[str, Any]) -> tuple[str | None, str | None]: + ua = str(event.get("user_agent") or "").lower() + source = str(event.get("client_source") or "").lower() + browser = str(event.get("browser") or "").lower() + if "googleimageproxy" in ua or "gmail_image_proxy" in source or "google image proxy" in browser: + return "google", "known GoogleImageProxy/Gmail image-proxy signature" + return None, None + + +def _changed_nonempty(left: Any, right: Any) -> bool: + a = str(left or "").strip() + b = str(right or "").strip() + return bool(a and b and a != b) + + +def classify_click_events(events: list[dict[str, Any]]) -> list[dict[str, Any]]: + """Return click events enriched with a reversible, query-time provider likelihood. + + The raw event rows are never mutated or discarded. The heuristic intentionally combines + multiple weak signals instead of treating timing, geography or User-Agent as proof on its own. + """ + source_rows = [dict(event) for event in events] + + fingerprint_links: dict[tuple[str, str], set[str]] = defaultdict(set) + for event in source_rows: + delivery_id = str(event.get("delivery_id") or "") + fingerprint = str(event.get("client_fingerprint") or "") + link_id = str(event.get("link_id") or "") + if delivery_id and fingerprint and link_id: + fingerprint_links[(delivery_id, fingerprint)].add(link_id) + + grouped: dict[tuple[str, str], list[tuple[int, dict[str, Any]]]] = defaultdict(list) + for index, event in enumerate(source_rows): + grouped[(str(event.get("delivery_id") or ""), str(event.get("link_id") or ""))].append((index, event)) + + classifications: dict[int, dict[str, Any]] = {} + for rows in grouped.values(): + ordered = sorted( + rows, + key=lambda item: ( + _parse_time(item[1].get("observed_at")) + if _parse_time(item[1].get("observed_at")) is not None + else float("-inf"), + int(item[1].get("id") or 0), + ), + ) + prior: list[dict[str, Any]] = [] + for index, event in ordered: + score = 5 + reasons: list[str] = [] + provider_guess = _provider_hint(event) + known_provider, known_reason = _known_proxy(event) + + if known_provider: + score = 100 + provider_guess = known_provider + reasons.append(str(known_reason)) + classification = "known_email_proxy" + else: + current_time = _parse_time(event.get("observed_at")) + nearest: dict[str, Any] | None = None + delta_seconds: float | None = None + if current_time is not None: + for candidate in reversed(prior): + candidate_time = _parse_time(candidate.get("observed_at")) + if candidate_time is None: + continue + delta = current_time - candidate_time + if delta < 0: + continue + if delta <= 15: + nearest = candidate + delta_seconds = delta + break + + if nearest is not None and delta_seconds is not None: + if _changed_nonempty(event.get("client_fingerprint"), nearest.get("client_fingerprint")): + if delta_seconds <= 5: + score += 55 + else: + score += 35 + reasons.append(f"second request on same delivery/link after {delta_seconds:.2f}s") + reasons.append("fingerprint changed") + if _changed_nonempty(event.get("country_code"), nearest.get("country_code")): + score += 15 + reasons.append( + f"country changed {str(nearest.get('country_code') or '')} → {str(event.get('country_code') or '')}" + ) + if _changed_nonempty(event.get("browser"), nearest.get("browser")): + score += 10 + reasons.append( + f"browser changed {str(nearest.get('browser') or '')} → {str(event.get('browser') or '')}" + ) + if _changed_nonempty(event.get("user_agent"), nearest.get("user_agent")): + score += 10 + reasons.append("User-Agent changed") + + delivery_id = str(event.get("delivery_id") or "") + fingerprint = str(event.get("client_fingerprint") or "") + consistent_links = len(fingerprint_links.get((delivery_id, fingerprint), set())) if fingerprint else 0 + if consistent_links >= 2: + score -= 25 + reasons.append(f"fingerprint observed consistently across {consistent_links} links in same delivery") + + source = str(event.get("client_source") or "").lower() + if "proxy" in source and source != "direct_or_unknown": + score += 25 + reasons.append(f"client source indicates proxy: {event.get('client_source')}") + + score = max(0, min(100, int(score))) + if score >= 70: + classification = "likely_email_provider" + provider_guess = provider_guess or "other" + elif score >= 35: + classification = "uncertain" + else: + classification = "likely_human" + provider_guess = None + + classifications[index] = { + "provider_likelihood": int(score), + "provider_classification": classification, + "provider_guess": provider_guess, + "classification_reasons": reasons, + } + prior.append(event) + + return [{**event, **classifications[index]} for index, event in enumerate(source_rows)] + + +def summarize_click_classification(events: list[dict[str, Any]]) -> dict[str, Any]: + classified = classify_click_events(events) + unique: dict[tuple[str, str, str], dict[str, Any]] = {} + for event in classified: + key = ( + str(event.get("delivery_id") or ""), + str(event.get("link_id") or ""), + str(event.get("client_fingerprint") or ""), + ) + previous = unique.get(key) + if previous is None or int(event.get("provider_likelihood") or 0) > int(previous.get("provider_likelihood") or 0): + unique[key] = event + + unique_rows = list(unique.values()) + classes = Counter(str(row.get("provider_classification") or "") for row in unique_rows) + likely_provider_rows = [ + row for row in unique_rows + if row.get("provider_classification") in {"likely_email_provider", "known_email_proxy"} + ] + provider_suspects = Counter( + str(row.get("provider_guess") or "other") for row in likely_provider_rows + ) + total_unique = len(unique_rows) + likely_provider_unique = len(likely_provider_rows) + provider_scores = [int(row.get("provider_likelihood") or 0) for row in likely_provider_rows] + if not provider_scores: + confidence = "low" + elif min(provider_scores) >= 85: + confidence = "high" + else: + confidence = "medium" + + return { + "classification_model": "heuristic-v1-query-time", + "observed_click_events": len(classified), + "unique_clicks_classified": total_unique, + "likely_human_unique_clicks": int(classes.get("likely_human", 0)), + "uncertain_unique_clicks": int(classes.get("uncertain", 0)), + "likely_provider_unique_clicks": likely_provider_unique, + "known_email_proxy_unique_clicks": int(classes.get("known_email_proxy", 0)), + "likely_human_or_unclassified_unique_clicks": max(0, total_unique - likely_provider_unique), + "potential_provider_share": { + "numerator": likely_provider_unique, + "denominator": total_unique, + "percent": round((likely_provider_unique / total_unique) * 100.0, 1) if total_unique else 0.0, + }, + "provider_suspects": dict(sorted(provider_suspects.items())), + "confidence": confidence, + "note": ( + "Qualitative estimate only. Raw events and the stable unique-click definition are unchanged; " + "classification is recalculated from stored evidence on every query." + ), + } diff --git a/src/postmaster/runtime.py b/src/postmaster/runtime.py index bf79bbd..2862a2c 100644 --- a/src/postmaster/runtime.py +++ b/src/postmaster/runtime.py @@ -37,6 +37,7 @@ def build_status(): status["native_file_resource_handoff"] = True status["link_tracking"] = True status["sent_copy_tracking_sanitized"] = True + status["provider_qualitative_classification"] = True return status mcp.remove_tool("build_status") @@ -62,6 +63,7 @@ def tracking_status(): base = _legacy_tracking_status() if isinstance(base, dict) and base.get("ok"): base["link_tracking"] = link_store().status() + base["link_tracking"]["provider_classification_query_time"] = True base["event_types"] = ["pixel", "amp_xhr", "link"] return base @@ -93,7 +95,7 @@ def get_tracking_summary( link_id: str | None = None, account_id: str | None = None, ): - """Read-only click summary. Unique click = delivery_id + link_id + client_fingerprint.""" + """Read-only click summary with stable fingerprint uniques plus query-time provider estimates.""" return _base._safe_call( link_store().summary, campaign_id=campaign_id, delivery_id=delivery_id, link_id=link_id, account_id=account_id, @@ -127,7 +129,7 @@ def list_tracking_events( event_type: str | None = None, limit: int = 500, ): - """Read-only unified tracking events. event_type may be all, pixel, amp_xhr or link.""" + """Read-only unified tracking events. Link rows include query-time provider classification fields.""" return _base._safe_call( link_store().unified_events, delivery_id=delivery_id, campaign_id=campaign_id, link_id=link_id, @@ -174,6 +176,9 @@ async def tracking_click(request: Request): def _tracking_dashboard_fragment(account_id: str | None = None) -> str: + summary = link_store().summary(account_id=account_id) + qualitative = summary.get("qualitative_estimate") or {} + share = qualitative.get("potential_provider_share") or {} top = link_store().top_links(account_id=account_id, limit=20) events = link_store().unified_events(account_id=account_id, limit=100) top_rows = [] @@ -199,6 +204,10 @@ def _tracking_dashboard_fragment(account_id: str | None = None) -> str: browser_os = " / ".join(x for x in (str(row.get("browser") or ""), str(row.get("os") or "")) if x) label = str(row.get("anchor_text") or row.get("destination_host") or "") ua = str(row.get("user_agent") or "")[:180] + provider_class = str(row.get("provider_classification") or "") + provider_likelihood = str(row.get("provider_likelihood") if row.get("provider_likelihood") is not None else "") + provider_guess = str(row.get("provider_guess") or "") + reasons = "; ".join(str(x) for x in (row.get("classification_reasons") or [])) event_rows.append( "" f"{escape(str(row.get('event_type') or ''))}" @@ -207,23 +216,39 @@ def _tracking_dashboard_fragment(account_id: str | None = None) -> str: f"{escape(source)}{escape(browser_os)}" f"{escape(str(row.get('campaign_id') or ''))}
{escape(str(row.get('delivery_id') or ''))}" f"{escape(str(row.get('client_fingerprint') or ''))}" + f"{escape(provider_likelihood)}{escape(provider_class)}{escape(provider_guess)}" + f"{escape(reasons[:180])}" f"{escape(label)}{escape(str(row.get('link_id') or ''))}" f"{escape(str(row.get('destination_host') or ''))}" f"{escape(str(row.get('position') if row.get('position') is not None else ''))}" f"{escape(ua)}" ) if not event_rows: - event_rows.append('No tracking events recorded yet.') + event_rows.append('No tracking events recorded yet.') + + unique_clicks = int(summary.get("unique_clicks") or 0) + likely_provider = int(qualitative.get("likely_provider_unique_clicks") or 0) + human_or_unclassified = int(qualitative.get("likely_human_or_unclassified_unique_clicks") or 0) + uncertain = int(qualitative.get("uncertain_unique_clicks") or 0) + share_percent = float(share.get("percent") or 0.0) + suspects = qualitative.get("provider_suspects") or {} + suspect_text = ", ".join(f"{key}: {value}" for key, value in suspects.items()) or "none" return f"""
+

Qualitative click estimate

v9.4.1 query-time heuristic
+

Stable fingerprint uniques remain unchanged. Provider likelihood is recalculated from stored evidence and never deletes or rewrites raw events.

+

{unique_clicks} unique fingerprint clicks  ·  {likely_provider} likely provider/proxy  ·  {human_or_unclassified} likely human or unclassified  ·  {uncertain} uncertain

+

Potential provider share: {likely_provider}/{unique_clicks} ({share_percent:.1f}%). Provider suspects: {escape(suspect_text)}. Confidence: {escape(str(qualitative.get('confidence') or 'low'))}.

+
+

Top links

v9.4 click analytics
-

Unique click = delivery_id + link_id + client_fingerprint. Fetches are telemetry; v9.4 does not classify human vs scanner.

+

Unique click = delivery_id + link_id + client_fingerprint. The qualitative classifier is an additive interpretation layer and does not change these totals.

{''.join(top_rows)}
Link IDLabelDestination hostTotalUniqueRecipientsFirst clickLast click

Tracking events

pixel / AMP / link
-
{''.join(event_rows)}
TypeRecipientObserved UTCCountry / sourceBrowser / OSCampaign / deliveryClient fingerprintLink labelLink IDDestinationPositionUser-Agent
+
{''.join(event_rows)}
TypeRecipientObserved UTCCountry / sourceBrowser / OSCampaign / deliveryClient fingerprintProvider %ClassificationProvider guessReasonsLink labelLink IDDestinationPositionUser-Agent
""" diff --git a/tests/test_v9_4_1_provider_classification.py b/tests/test_v9_4_1_provider_classification.py new file mode 100644 index 0000000..f686ce5 --- /dev/null +++ b/tests/test_v9_4_1_provider_classification.py @@ -0,0 +1,161 @@ +from __future__ import annotations + +import os +import tempfile +import unittest +from pathlib import Path + +from postmaster.email_analytics import EmailAnalyticsStore +from postmaster.link_tracking import LinkTrackingStore +from postmaster.provider_classification import classify_click_events, summarize_click_classification + + +class ProviderClassificationTests(unittest.TestCase): + @staticmethod + def _event( + event_id: int, + *, + observed_at: str, + link_id: str, + fingerprint: str, + country: str, + browser: str, + user_agent: str, + source: str = "direct_or_unknown", + delivery_id: str = "del_ground_truth", + recipient: str = "reader@example.test", + ) -> dict: + return { + "id": event_id, + "delivery_id": delivery_id, + "link_id": link_id, + "recipient": recipient, + "client_fingerprint": fingerprint, + "country_code": country, + "browser": browser, + "user_agent": user_agent, + "client_source": source, + "observed_at": observed_at, + } + + def test_ground_truth_combines_signals_without_changing_unique_definition(self) -> None: + chrome151 = "Mozilla/5.0 (Windows NT 10.0) Chrome/151.0.0.0 Safari/537.36" + chrome149 = "Mozilla/5.0 (Windows NT 10.0) Chrome/149.0.0.0 Safari/537.36" + android151 = "Mozilla/5.0 (Linux; Android 15) Chrome/151.0.0.0 Mobile Safari/537.36" + events = [ + # Gmail A: the user click we know was made manually. + self._event(1, observed_at="2026-08-20T10:00:00+00:00", link_id="gmail-a", fingerprint="fp-human", country="IT", browser="Chrome 151.0.0.0", user_agent=chrome151, recipient="reader@gmail.com"), + # Gmail A: additional request not made by the user, 3.637 seconds later. + self._event(2, observed_at="2026-08-20T10:00:03.637000+00:00", link_id="gmail-a", fingerprint="fp-provider", country="US", browser="Chrome 149.0.0.0", user_agent=chrome149, recipient="reader@gmail.com"), + # Same human fingerprint later on another link in the same delivery. + self._event(3, observed_at="2026-08-20T10:00:30+00:00", link_id="gmail-b", fingerprint="fp-human", country="IT", browser="Chrome 151.0.0.0", user_agent=chrome151, recipient="reader@gmail.com"), + # Libero-style pair: same fingerprint across two explicit human clicks. + self._event(4, observed_at="2026-08-20T10:01:00+00:00", link_id="libero-a", fingerprint="fp-libero", country="IT", browser="Chrome 151.0.0.0", user_agent=android151, recipient="reader@libero.it"), + self._event(5, observed_at="2026-08-20T10:01:08+00:00", link_id="libero-b", fingerprint="fp-libero", country="IT", browser="Chrome 151.0.0.0", user_agent=android151, recipient="reader@libero.it"), + # Explicit known provider signature is deterministic and does not need timing heuristics. + self._event(6, observed_at="2026-08-20T10:02:00+00:00", link_id="proxy", fingerprint="fp-google-proxy", country="US", browser="Google Image Proxy", user_agent="Mozilla/5.0 (via ggpht.com GoogleImageProxy)", source="gmail_image_proxy", recipient="reader@gmail.com"), + ] + + classified = {row["id"]: row for row in classify_click_events(events)} + + self.assertLessEqual(classified[1]["provider_likelihood"], 10) + self.assertEqual(classified[1]["provider_classification"], "likely_human") + self.assertGreaterEqual(classified[2]["provider_likelihood"], 85) + self.assertLessEqual(classified[2]["provider_likelihood"], 100) + self.assertEqual(classified[2]["provider_classification"], "likely_email_provider") + self.assertEqual(classified[2]["provider_guess"], "google") + self.assertIn("second request on same delivery/link after 3.64s", classified[2]["classification_reasons"]) + self.assertIn("fingerprint changed", classified[2]["classification_reasons"]) + self.assertIn("country changed IT → US", classified[2]["classification_reasons"]) + self.assertIn("browser changed Chrome 151.0.0.0 → Chrome 149.0.0.0", classified[2]["classification_reasons"]) + self.assertLessEqual(classified[4]["provider_likelihood"], 5) + self.assertLessEqual(classified[5]["provider_likelihood"], 5) + self.assertEqual(classified[6]["provider_likelihood"], 100) + self.assertEqual(classified[6]["provider_classification"], "known_email_proxy") + self.assertEqual(classified[6]["provider_guess"], "google") + + summary = summarize_click_classification(events) + self.assertEqual(summary["unique_clicks_classified"], 6) + self.assertEqual(summary["likely_provider_unique_clicks"], 2) + self.assertEqual(summary["known_email_proxy_unique_clicks"], 1) + self.assertEqual(summary["likely_human_or_unclassified_unique_clicks"], 4) + self.assertEqual(summary["potential_provider_share"], {"numerator": 2, "denominator": 6, "percent": 33.3}) + self.assertEqual(summary["provider_suspects"], {"google": 2}) + + def test_one_weak_signal_does_not_become_provider_proof(self) -> None: + events = [ + self._event(1, observed_at="2026-08-20T11:00:00+00:00", link_id="weak", fingerprint="fp-a", country="IT", browser="Chrome 151", user_agent="Chrome/151"), + self._event(2, observed_at="2026-08-20T11:00:10+00:00", link_id="weak", fingerprint="fp-b", country="IT", browser="Chrome 151", user_agent="Chrome/151"), + ] + classified = {row["id"]: row for row in classify_click_events(events)} + self.assertEqual(classified[2]["provider_likelihood"], 40) + self.assertEqual(classified[2]["provider_classification"], "uncertain") + self.assertIsNone(classified[2]["provider_guess"]) + + +class ProviderClassificationStoreIntegrationTests(unittest.TestCase): + def setUp(self) -> None: + self.tmp = tempfile.TemporaryDirectory() + root = Path(self.tmp.name) + self.old_public = {k: os.environ.get(k) for k in ("PUBLIC_EMAIL_BASE_URL", "PUBLIC_MCP_HOST")} + os.environ["PUBLIC_EMAIL_BASE_URL"] = "https://postmaster.example.test" + os.environ["PUBLIC_MCP_HOST"] = "" + self.analytics = EmailAnalyticsStore(db_path=str(root / "analytics.db"), key_path=str(root / "analytics.key")) + self.links = LinkTrackingStore(self.analytics) + self.campaign = self.analytics.create_campaign(account_id="acct", sender="sender@example.test", subject="v9.4.1", track_opens=True, amp_used=False) + self.delivery = self.analytics.create_delivery(campaign_id=self.campaign["id"], account_id="acct", recipient="reader@gmail.com", recipient_role="to") + + def tearDown(self) -> None: + for key, value in self.old_public.items(): + if value is None: + os.environ.pop(key, None) + else: + os.environ[key] = value + self.tmp.cleanup() + + def test_store_queries_classify_at_read_time_without_schema_columns(self) -> None: + _, meta = self.links.instrument_html(body_html='Project', delivery=self.delivery) + occurrence_id = meta[0]["occurrence_id"] + with self.links._connect() as conn: + token = str(conn.execute("SELECT tracking_token FROM tracking_links WHERE id=?", (occurrence_id,)).fetchone()[0]) + link = self.links.get_by_token(token) + + first = self.links.record_click( + link, + user_agent="Mozilla/5.0 (Windows NT 10.0) Chrome/151.0.0.0 Safari/537.36", + client_ip="203.0.113.10", + country_code="IT", + ) + second = self.links.record_click( + link, + user_agent="Mozilla/5.0 (Windows NT 10.0) Chrome/149.0.0.0 Safari/537.36", + client_ip="203.0.113.11", + country_code="US", + ) + with self.links._connect() as conn: + conn.execute("UPDATE tracking_clicks SET observed_at=? WHERE id=?", ("2026-08-20T12:00:00+00:00", first["id"])) + conn.execute("UPDATE tracking_clicks SET observed_at=? WHERE id=?", ("2026-08-20T12:00:03.637000+00:00", second["id"])) + columns = {str(row["name"]) for row in conn.execute("PRAGMA table_info(tracking_clicks)").fetchall()} + + self.assertNotIn("provider_likelihood", columns) + self.assertNotIn("provider_classification", columns) + + events = self.links.list_click_events(delivery_id=self.delivery["id"]) + by_id = {row["id"]: row for row in events} + self.assertEqual(by_id[first["id"]]["provider_classification"], "likely_human") + self.assertGreaterEqual(by_id[second["id"]]["provider_likelihood"], 85) + self.assertEqual(by_id[second["id"]]["provider_classification"], "likely_email_provider") + self.assertEqual(by_id[second["id"]]["provider_guess"], "google") + + summary = self.links.summary(delivery_id=self.delivery["id"]) + self.assertEqual(summary["unique_clicks"], 2) + self.assertEqual(summary["unique_click_definition"], "delivery_id + link_id + client_fingerprint") + self.assertEqual(summary["likely_provider_unique_clicks"], 1) + self.assertEqual(summary["likely_human_or_unclassified_unique_clicks"], 1) + self.assertEqual(summary["potential_provider_share"], {"numerator": 1, "denominator": 2, "percent": 50.0}) + self.assertEqual(summary["provider_suspects"], {"google": 1}) + self.assertEqual(summary["provider_classification_model"], "heuristic-v1-query-time") + + +if __name__ == "__main__": + unittest.main()