diff --git a/CHANGELOG.md b/CHANGELOG.md index a451687..708dc40 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -4,6 +4,39 @@ All notable changes to MAPLEAD. Format follows [Keep a Changelog](https://keepac ## [Unreleased] +## [1.3.0] — 2026-11-07 + +Linux (AppImage) only for now — the Windows build and the terminal wizard are +unchanged in this release. + +### Added +- **Bulk WhatsApp sender** — a new **Bulk send** button in the leads panel sends + your saved WhatsApp message to every lead of the current filter (for example + *No website*) one by one, with no clicks per lead. + 1. A checklist shows the leads that have a WhatsApp number. Untick anyone you + want to leave out and click a row to preview its message. + 2. **Start sending** opens WhatsApp Web inside MAPLEAD. Scan the QR code once; + the session is saved for next time. + 3. Each chat is opened with the message filled in and sent. Numbers that are + not on WhatsApp are detected and skipped. **Pause**, **Skip lead** and + **Stop** work at any time, and an activity log shows every step. +- **Safety pauses** against WhatsApp blocks, all editable before each run: + random 45–120 s pause between messages, a 10 min rest every 10 messages, a + *"WhatsApp may block the number after this many messages — continue? (Yes/No)"* + prompt every 25 messages, and a 50-a-day limit that asks before going over. +- **Contact history** — every number messaged is saved with its date and result + (sent / not on WhatsApp / failed / skipped) in + `~/.local/share/maplead/whatsapp-history.json`. Leads already contacted come + unticked, show a **Contacted** chip on their card, and the new + **Not contacted** filter hides them. +- `tools/bulk_test.py` — try the sender on your own numbers without a search + (`--forget` removes them from the history afterwards). + +### Changed +- The AppImage now bundles QtWebEngine (the browser engine that runs WhatsApp + Web inside the app), so the file is bigger. `build-appimage.sh` installs the + full `PySide6` package instead of `PySide6-Essentials`. + ## [1.2.0] — 2026-10-22 ### Added diff --git a/README.md b/README.md index ccf0e28..4258048 100644 --- a/README.md +++ b/README.md @@ -18,6 +18,7 @@ lead-scraper API and writes them to CSV / Excel / JSON / TXT / TSV / HTML. | 🧭 **65 countries / 1044 cities** | typed, filtered, auto-completed — no spelling games | | 🏷️ **68 business types · 35 fields** | you choose exactly which columns come out | | 💬 **Leads panel + WhatsApp** | cards with one-click links and a first message built from the lead | +| 📨 **Bulk WhatsApp** | sends the message to a whole filter through WhatsApp Web, with safety pauses and a contact history | | 📄 **6 export formats** | csv · xlsx · json · txt · tsv · html |
@@ -32,8 +33,8 @@ lead-scraper API and writes them to CSV / Excel / JSON / TXT / TSV / HTML. **Linux x86_64, AppImage** — one file, no packages, no root: ```bash -chmod +x MAPLEAD-1.2.0-x86_64.AppImage -./MAPLEAD-1.2.0-x86_64.AppImage +chmod +x MAPLEAD-1.3.0-x86_64.AppImage +./MAPLEAD-1.3.0-x86_64.AppImage ``` Building it yourself (also fine for arm64/other distros with a rebuild): @@ -114,6 +115,18 @@ folder — `~/Documents/maplead/` on Linux,  +## 5 · Message leads in bulk (Linux) + +Open the **Leads panel**, pick a filter (e.g. *No website*) and press **Bulk send**: + +1. untick the leads you want to skip — the message preview shows exactly what each one gets; +2. **Start sending** opens WhatsApp Web inside MAPLEAD (scan the QR code once); +3. it sends one message at a time, with random pauses, rests and a + *"may get blocked — continue?"* question every 25 messages. + +Everyone you message is remembered, so the next search skips them. +Details and the safety settings: [docs/GUIDE.md › Bulk WhatsApp](docs/GUIDE.md#bulk-whatsapp). + --- ## Terminal wizard @@ -165,9 +178,9 @@ On Linux the wizard is inside the AppImage — the same file as the desktop app, nothing has to be installed first either: ```bash -./MAPLEAD-1.2.0-x86_64.AppImage --cli --demo -./MAPLEAD-1.2.0-x86_64.AppImage --cli --country DE --city Berlin --business dentist -./MAPLEAD-1.2.0-x86_64.AppImage --cli --doctor +./MAPLEAD-1.3.0-x86_64.AppImage --cli --demo +./MAPLEAD-1.3.0-x86_64.AppImage --cli --country DE --city Berlin --business dentist +./MAPLEAD-1.3.0-x86_64.AppImage --cli --doctor ``` You can add `~/.local/bin/maplead-cli` yourself (`#!/bin/sh` + `exec "$HOME/Apps"/MAPLEAD-*-x86_64.AppImage --cli "$@"`) @@ -194,7 +207,8 @@ The terminal wizard runs anywhere Python 3.9+ does. macOS packaging is on the ro Lead data returned by these APIs is public business information. Respect each provider's terms and rate limits, and your GDPR/LGPD/PCCB obligations before -cold-contacting anyone. MAPLEAD never edits Google Maps — it only reads what the +cold-contacting anyone. Automated WhatsApp messages go against WhatsApp's terms of +service: the bulk sender's pauses lower the risk of a ban, they do not remove it. MAPLEAD never edits Google Maps — it only reads what the provider APIs already expose. ## License diff --git a/build-appimage.sh b/build-appimage.sh index f3a6e31..c7e7df0 100755 --- a/build-appimage.sh +++ b/build-appimage.sh @@ -77,9 +77,10 @@ if [[ ! -x "$PY" ]]; then rm -rf "$tmp" fi fi -if ! "$PY" -c "import PySide6" >/dev/null 2>&1; then +if ! "$PY" -c "import PyInstaller; from PySide6 import QtWebEngineWidgets" >/dev/null 2>&1; then "$PY" -m pip install -q --upgrade pip - "$PY" -m pip install -q "PySide6-Essentials>=6.8" pyinstaller + # full PySide6: the bulk WhatsApp sender needs QtWebEngine + "$PY" -m pip install -q "PySide6>=6.8" pyinstaller fi "$PY" -c "import PySide6; from PySide6 import QtCore; print(' PySide6', QtCore.__version__)" diff --git a/docs/DEVELOPMENT.md b/docs/DEVELOPMENT.md index cabfd5f..e43d06f 100644 --- a/docs/DEVELOPMENT.md +++ b/docs/DEVELOPMENT.md @@ -70,14 +70,17 @@ maplead/ app.py entry point, theme bootstrap, --selftest main_window.py window: form, stats, table, activity log lead_panel.py card view of the leads + WhatsApp message template + bulk_sender.py bulk WhatsApp: lead picker, safety settings, WhatsApp Web sender widgets.py cards, searchable combo, field picker, table model keys.py API key manager (paste · verify live · save) worker.py harvest + key validation on a QThread icons.py inline SVG icon set (no image files) theme.py dark/light palettes and the stylesheet contacts.py WhatsApp number normalisation + {placeholder} messages + outreach.py bulk-send queue, safety pacing, contact history (no Qt) packaging/ icon (svg+png+ico), .desktop file, PyInstaller specs, launchers tools/screenshots.py regenerates docs/screenshots/* +tools/bulk_test.py try the bulk WhatsApp sender on your own numbers build-appimage.sh one-command AppImage build (+ optional --install) build-windows.ps1 one-command Windows build: one portable .exe docs/ user guide, development notes and screenshots @@ -86,7 +89,7 @@ docs/ user guide, development notes and screenshots ## Running from source ```bash -python3 -m venv .venv-gui && .venv-gui/bin/python -m pip install PySide6-Essentials +python3 -m venv .venv-gui && .venv-gui/bin/python -m pip install PySide6 # full: QtWebEngine for the bulk sender ./maplead.py --demo # terminal wizard (needs no venv at all) ./maplead-gui.py # desktop @@ -116,7 +119,7 @@ MAPLEAD_OUTDIR=~/bin ./build-appimage.sh # somewhere else What it does, in order: -1. creates/updates `.venv-gui` (PySide6-Essentials + PyInstaller); +1. creates/updates `.venv-gui` (PySide6 with QtWebEngine + PyInstaller); 2. renders the PNG icons from `packaging/maplead.svg` (Qt does the rasterising — no ImageMagick, no Pillow); 3. generates the `.desktop` entry (`Exec=AppRun`, replaced at install time); diff --git a/docs/GUIDE.md b/docs/GUIDE.md index c070da1..120ec8b 100644 --- a/docs/GUIDE.md +++ b/docs/GUIDE.md @@ -20,11 +20,11 @@ New here? Read this once, then the [README](../README.md) is just a reminder. ### Linux x86_64 -Download `MAPLEAD-1.2.0-x86_64.AppImage`, then: +Download `MAPLEAD-1.3.0-x86_64.AppImage`, then: ```bash -chmod +x MAPLEAD-1.2.0-x86_64.AppImage # first time only -./MAPLEAD-1.2.0-x86_64.AppImage +chmod +x MAPLEAD-1.3.0-x86_64.AppImage # first time only +./MAPLEAD-1.3.0-x86_64.AppImage ``` After `./build-appimage.sh --install` you can also just type `maplead`, and the app @@ -36,7 +36,7 @@ only things it expects from the system are a display and the usual C library. The same file runs the terminal wizard too — no Python needed: ```bash -./MAPLEAD-1.2.0-x86_64.AppImage --cli --demo +./MAPLEAD-1.3.0-x86_64.AppImage --cli --demo ``` **No FUSE?** Older AppImages mount themselves through `libfuse.so.2`. If your system @@ -169,12 +169,41 @@ reading rather than for a spreadsheet: Unknown placeholders are highlighted in the preview instead of silently disappearing. The template is remembered between launches. - **Chips filter the list**: *Has WhatsApp*, *Has phone*, *Has website*, - *No website*, *Has e-mail*. The search box matches every visible field, and + *No website*, *Has e-mail*, *Not contacted*. The search box matches every visible field, and the sort menu orders by rating, reviews or name. - **Copy** on a card puts the whole row (TSV) on the clipboard. - **Right-click a grid row** also has *Send WhatsApp message* and *Open in leads panel*. The panel stays open while you work and refreshes after the next run. +### Bulk WhatsApp + +**Bulk send** in the panel header sends the saved message to every lead of the +current filter and search (for example *No website*) that has a WhatsApp number. + +1. A checklist opens: untick anyone you want to leave out (leads already + messaged come unticked), click a row to preview its message, and check the + **Safety** settings. +2. **Start sending** opens WhatsApp Web inside MAPLEAD. The first time, scan the + QR code (phone → WhatsApp → *Linked devices*); the session is kept for later. +3. Each lead is opened with the message filled in and sent; numbers without + WhatsApp are marked and skipped. **Pause**, **Skip lead** and **Stop** work at + any time. Keep the window open (minimising is fine). + +| Safety setting | default | what it does | +|---|---|---| +| Pause from / up to | 45 – 120 s | random wait between two messages | +| Rest every / for | 10 msgs / 10 min | longer break after each burst | +| Ask after | 25 msgs | stops and asks *"WhatsApp may block the number — continue?"* | +| Daily limit | 50 msgs | asks before going over today's total | + +Every number is saved with date and status in `whatsapp-history.json` in the +data folder (`~/.local/share/maplead/` on Linux, `%LOCALAPPDATA%\maplead\` on +Windows); contacted leads show a *Contacted* chip and are skipped by default. + +> Automated messaging goes against WhatsApp's terms of service. The pauses lower +> the risk of the number being restricted; they do not remove it. Use a number +> dedicated to prospecting. + ## 7 · The export file A search **never writes a file by itself**. Results stay in the grid and in the @@ -209,7 +238,7 @@ The original interface, still fully supported and dependency-free: Without Python, the packaged builds carry the same wizard: ```bash -./MAPLEAD-1.2.0-x86_64.AppImage --cli --demo # Linux: the AppImage is both +./MAPLEAD-1.3.0-x86_64.AppImage --cli --demo # Linux: the AppImage is both .\MAPLEAD-1.2.0-windows-x64.exe --cli --demo # Windows: same trick, one .exe ``` diff --git a/maplead/art.py b/maplead/art.py index 6837393..55ddbe1 100644 --- a/maplead/art.py +++ b/maplead/art.py @@ -10,7 +10,7 @@ from . import host -VERSION = "1.2.0" +VERSION = "1.3.0" # --------------------------------------------------------------------------- # # colour plumbing diff --git a/maplead/gui/app.py b/maplead/gui/app.py index 21088b3..751a3ee 100644 --- a/maplead/gui/app.py +++ b/maplead/gui/app.py @@ -154,6 +154,11 @@ def create_app(argv: Optional[Sequence[str]] = None): """Boot QApplication and paint the first palette.""" args = build_parser().parse_args(argv) + if QApplication.instance() is None: + # QtWebEngine (bulk WhatsApp sender) needs shared GL contexts set + # before the application object exists + from PySide6.QtCore import Qt as _Qt + QApplication.setAttribute(_Qt.AA_ShareOpenGLContexts, True) app = QApplication.instance() or QApplication(list(sys.argv[:1])) app.setApplicationName("MAPLEAD") app.setApplicationDisplayName("MAPLEAD") diff --git a/maplead/gui/bulk_sender.py b/maplead/gui/bulk_sender.py new file mode 100644 index 0000000..8d8df6a --- /dev/null +++ b/maplead/gui/bulk_sender.py @@ -0,0 +1,881 @@ +""" +Bulk WhatsApp sender: pick leads, then let WhatsApp Web send them one by one. + +Two windows: + +* :class:`BulkSendDialog` — the leads from the panel's current filter with a + checkbox each (numbers already messaged come unticked), a preview of the + message every lead will get, and the safety settings (pauses, rests, the + "continue?" threshold, the daily cap). +* :class:`WhatsAppSenderWindow` — WhatsApp Web inside the app (QtWebEngine, its + own persistent profile, so the QR code is scanned once). A one-second state + machine opens ``web.whatsapp.com/send?phone=…&text=…`` for each lead, waits for + the send button, clicks it, confirms the composer emptied, records the result + in the history and waits the pause the :class:`~maplead.outreach.Pacer` picks. + +Everything WhatsApp-Web specific (selectors, texts) is in the ``_JS_*`` scripts +below — when WhatsApp changes its page, that is the only place to adjust. +QtWebEngine is imported lazily: a build without it still runs, and the panel +explains why bulk sending is unavailable. +""" + +from __future__ import annotations + +import json +import os +import re +from typing import Any, Dict, List, Optional, Sequence +from urllib.parse import quote + +from PySide6.QtCore import QEvent, QSettings, Qt, QTimer, QUrl, Signal +from PySide6.QtGui import QKeyEvent +from PySide6.QtWidgets import (QApplication, QDialog, QFrame, QGridLayout, + QHBoxLayout, QHeaderView, QLabel, QMessageBox, + QPlainTextEdit, QProgressBar, QPushButton, + QSpinBox, QSplitter, QTableWidget, + QTableWidgetItem, QVBoxLayout, QWidget) + +from .. import contacts, outreach, paths +from . import icons +from .theme import Palette +from .widgets import LogPane, card, field_label + +_SETTINGS_ORG = "maplead" +_SETTINGS_APP = "maplead-desktop" +_SAFETY_KEY = "whatsapp/bulk_safety" + +WHATSAPP_WEB = "https://web.whatsapp.com/" +WHATSAPP_SEND = WHATSAPP_WEB + "send" + +# seconds +OPEN_TIMEOUT = 60 # chat must open (or be reported invalid) by then +VERIFY_TIMEOUT = 20 # the composer must empty after clicking send +ENTER_FALLBACK_AT = 5 # press Enter ourselves if the click did nothing + + +# --------------------------------------------------------------------------- # +# settings +# --------------------------------------------------------------------------- # + +def load_safety() -> outreach.SafetySettings: + value = QSettings(_SETTINGS_ORG, _SETTINGS_APP).value(_SAFETY_KEY, "") + try: + data = json.loads(str(value)) if value else None + except ValueError: + data = None + return outreach.SafetySettings.from_dict(data) + + +def save_safety(settings: outreach.SafetySettings) -> None: + store = QSettings(_SETTINGS_ORG, _SETTINGS_APP) + store.setValue(_SAFETY_KEY, json.dumps(settings.normalized().to_dict())) + store.sync() + + +def webengine_available() -> bool: + import importlib + try: + importlib.import_module("PySide6.QtWebEngineCore") + importlib.import_module("PySide6.QtWebEngineWidgets") + except ImportError: + return False + return True + + +# --------------------------------------------------------------------------- # +# WhatsApp Web scripts (the only page-specific part) +# --------------------------------------------------------------------------- # + +_JS_HELPERS = r""" +var mlq = function (list) { + for (var i = 0; i < list.length; i++) { + var el = document.querySelector(list[i]); + if (el) return el; + } + return null; +}; +var mlSend = function () { + return mlq(['[data-icon="send"]', '[data-icon="wds-ic-send-filled"]', + 'span[data-icon*="send"]', 'button[aria-label="Send"]', + 'button[aria-label="Enviar"]', 'div[aria-label="Send"]', + 'div[aria-label="Enviar"]']); +}; +var mlComposer = function () { + return mlq(['footer div[contenteditable="true"]', + 'div[contenteditable="true"][data-tab="10"]', + '#main div[contenteditable="true"][role="textbox"]']); +}; +""" + +_JS_PROBE = _JS_HELPERS + r""" +(function () { + var side = mlq(['#pane-side', '[data-testid="chat-list"]', + 'div[aria-label="Chat list"]', 'div[aria-label="Lista de conversas"]', + 'div[aria-label="Lista de chats"]']); + var qr = mlq(['canvas[aria-label*="QR"]', 'canvas[aria-label*="qr"]', + 'div[data-ref] canvas', '[data-testid="qrcode"]']); + var dialog = mlq(['[data-animate-modal-popup="true"]', 'div[role="dialog"]']); + var composer = mlComposer(); + return JSON.stringify({ + logged: !!side || !!composer, + qr: !!qr, + dialog: dialog ? (dialog.innerText || '').slice(0, 400) : '', + send: !!mlSend(), + composer: composer ? (composer.innerText || '').trim().length : -1 + }); +})(); +""" + +_JS_CLICK_SEND = _JS_HELPERS + r""" +(function () { + var el = mlSend(); + if (!el) return false; + var btn = el.closest('button') || el.closest('[role="button"]') || el; + btn.click(); + return true; +})(); +""" + +_JS_FOCUS_COMPOSER = _JS_HELPERS + r""" +(function () { + var c = mlComposer(); + if (!c) return false; + c.focus(); + return true; +})(); +""" + +# texts of the popup WhatsApp shows for a number that has no account +_INVALID_RE = re.compile(r"invalid|inv[áa]lid|no v[áa]lido|ung[üu]ltig|non valide|" + r"not on whatsapp|n[ãa]o est[áa] no whatsapp", re.I) +# popups that are only a "please wait" +_LOADING_RE = re.compile(r"starting chat|iniciando|loading|carregando|cargando", re.I) + + +def _profile(): + """The persistent WhatsApp Web profile (shared by every sender window).""" + from PySide6.QtWebEngineCore import QWebEngineProfile + + app = QApplication.instance() + existing = getattr(app, "_maplead_wa_profile", None) + if existing is not None: + return existing + profile = QWebEngineProfile("maplead-whatsapp", app) + storage = paths.ensure_dir(os.path.join(paths.data_dir(), "whatsapp-web")) + profile.setPersistentStoragePath(storage) + profile.setCachePath(os.path.join(storage, "cache")) + profile.setPersistentCookiesPolicy( + QWebEngineProfile.PersistentCookiesPolicy.ForcePersistentCookies) + # WhatsApp Web refuses browsers it does not recognise; drop the Qt token so + # the engine presents itself as the Chromium it really is. + agent = re.sub(r"\s*QtWebEngine/\S+", "", profile.httpUserAgent()) + profile.setHttpUserAgent(agent) + app._maplead_wa_profile = profile + return profile + + +# --------------------------------------------------------------------------- # +# setup dialog: pick the leads and the safety settings +# --------------------------------------------------------------------------- # + +class BulkSendDialog(QDialog): + """Review the queue, tick who gets the message, then start.""" + + COL_CHECK, COL_NAME, COL_NUMBER, COL_STATUS = range(4) + + def __init__(self, leads: Sequence[Dict[str, Any]], palette: Palette, *, + dial: str = "", country_code: str = "", + filter_label: str = "", parent: Optional[QWidget] = None): + super().__init__(parent) + self.p = palette + self._dial = dial + self._iso = (country_code or "").lower() + self._leads = list(leads) + self.history = outreach.History() + self.safety = load_safety() + self.items: List[outreach.OutreachItem] = [] + self.selected: List[outreach.OutreachItem] = [] + + self.setWindowTitle("MAPLEAD — bulk WhatsApp") + self.setMinimumSize(860, 640) + self.resize(980, 760) + self.setSizeGripEnabled(True) + + root = QVBoxLayout(self) + root.setContentsMargins(16, 16, 16, 12) + root.setSpacing(12) + + # header ------------------------------------------------------------- + header = QFrame() + header.setObjectName("panelHeader") + hl = QHBoxLayout(header) + hl.setContentsMargins(16, 12, 16, 12) + titles = QVBoxLayout() + titles.setSpacing(1) + title = QLabel("BULK WHATSAPP") + title.setObjectName("panelTitle") + titles.addWidget(title) + self.sub_label = QLabel("") + self.sub_label.setObjectName("panelSub") + titles.addWidget(self.sub_label) + hl.addLayout(titles) + hl.addStretch(1) + message_btn = QPushButton(" Edit message") + message_btn.setIcon(icons.icon("chat", self.p.text_dim, 15)) + message_btn.clicked.connect(self._edit_template) + hl.addWidget(message_btn) + root.addWidget(header) + self._filter_label = filter_label + + # leads + preview ------------------------------------------------------ + split = QSplitter(Qt.Horizontal) + split.setChildrenCollapsible(False) + + leads_card, lc = card("Leads to message") + tools = QHBoxLayout() + tools.setSpacing(6) + for label, mode in (("All", "all"), ("None", "none"), + ("Not contacted", "new")): + button = QPushButton(label) + button.setProperty("flat", True) + button.clicked.connect(lambda _=False, m=mode: self._tick(m)) + tools.addWidget(button) + tools.addStretch(1) + lc.addLayout(tools) + + self.table = QTableWidget(0, 4) + self.table.setHorizontalHeaderLabels(["", "Business", "WhatsApp", "History"]) + self.table.verticalHeader().setVisible(False) + self.table.setSelectionBehavior(QTableWidget.SelectRows) + self.table.setSelectionMode(QTableWidget.SingleSelection) + self.table.setEditTriggers(QTableWidget.NoEditTriggers) + self.table.setShowGrid(False) + head = self.table.horizontalHeader() + head.setSectionResizeMode(self.COL_CHECK, QHeaderView.ResizeToContents) + head.setSectionResizeMode(self.COL_NAME, QHeaderView.Stretch) + head.setSectionResizeMode(self.COL_NUMBER, QHeaderView.ResizeToContents) + head.setSectionResizeMode(self.COL_STATUS, QHeaderView.ResizeToContents) + self.table.itemChanged.connect(self._on_item_changed) + self.table.currentCellChanged.connect(lambda *_: self._sync_preview()) + lc.addWidget(self.table, 1) + split.addWidget(leads_card) + + side = QWidget() + sl = QVBoxLayout(side) + sl.setContentsMargins(0, 0, 0, 0) + sl.setSpacing(12) + + preview_card, pc = card("Message preview") + self.preview = QPlainTextEdit() + self.preview.setReadOnly(True) + self.preview.setMinimumHeight(110) + pc.addWidget(self.preview) + sl.addWidget(preview_card, 1) + + safety_card, sc = card("Safety") + grid = QGridLayout() + grid.setHorizontalSpacing(10) + grid.setVerticalSpacing(6) + s = self.safety + self.min_delay = self._spin(5, 3600, s.min_delay, " s") + self.max_delay = self._spin(5, 3600, s.max_delay, " s") + self.burst_size = self._spin(0, 500, s.burst_size, " msgs") + self.burst_rest = self._spin(0, 240, s.burst_rest // 60, " min") + self.warn_after = self._spin(0, 1000, s.warn_after, " msgs") + self.daily_limit = self._spin(0, 2000, s.daily_limit, " msgs") + rows = ( + ("Pause from", self.min_delay, "Shortest random pause between two messages"), + ("Pause up to", self.max_delay, "Longest random pause between two messages"), + ("Rest every", self.burst_size, "After this many messages, take a longer rest (0 = never)"), + ("Rest for", self.burst_rest, "Length of that rest"), + ("Ask after", self.warn_after, + "Stop and ask whether to continue every time this many messages went out (0 = never)"), + ("Daily limit", self.daily_limit, + "Ask before going over this many messages today (0 = no limit)"), + ) + for i, (label, spin, tip) in enumerate(rows): + grid.addWidget(field_label(label, tip), i // 2, (i % 2) * 2) + spin.setToolTip(tip) + spin.valueChanged.connect(self._sync_footer) + grid.addWidget(spin, i // 2, (i % 2) * 2 + 1) + sc.addLayout(grid) + risk = QLabel("Automated messages go against WhatsApp's terms. The pauses " + "lower the risk of the number being restricted, they do not " + "remove it — prefer a number used only for prospecting.") + risk.setObjectName("hint") + risk.setWordWrap(True) + sc.addWidget(risk) + sl.addWidget(safety_card) + split.addWidget(side) + split.setStretchFactor(0, 3) + split.setStretchFactor(1, 2) + root.addWidget(split, 1) + + # footer --------------------------------------------------------------- + footer = QHBoxLayout() + footer.setSpacing(10) + self.summary = QLabel("") + self.summary.setObjectName("hint") + self.summary.setWordWrap(True) + footer.addWidget(self.summary, 1) + cancel = QPushButton("Cancel") + cancel.clicked.connect(self.reject) + footer.addWidget(cancel) + self.start_btn = QPushButton(" Start sending") + self.start_btn.setProperty("accent", True) + self.start_btn.setIcon(icons.icon("play", self.p.accent_text, 13)) + self.start_btn.clicked.connect(self.accept) + footer.addWidget(self.start_btn) + root.addLayout(footer) + + self._fill() + + # -- building ------------------------------------------------------------- + @staticmethod + def _spin(lo: int, hi: int, value: int, suffix: str) -> QSpinBox: + spin = QSpinBox() + spin.setRange(lo, hi) + spin.setValue(max(lo, min(hi, int(value)))) + spin.setSuffix(suffix) + return spin + + def _template(self) -> str: + from .lead_panel import load_message_template + return load_message_template() + + def _fill(self) -> None: + self.items = outreach.build_queue( + self._leads, self._template(), dial=self._dial, iso=self._iso, + history=self.history, skip_contacted=False) + self.table.blockSignals(True) + self.table.setRowCount(len(self.items)) + for row, item in enumerate(self.items): + check = QTableWidgetItem() + check.setFlags(Qt.ItemIsUserCheckable | Qt.ItemIsEnabled | Qt.ItemIsSelectable) + check.setCheckState(Qt.Unchecked if item.contacted else Qt.Checked) + self.table.setItem(row, self.COL_CHECK, check) + self.table.setItem(row, self.COL_NAME, QTableWidgetItem(item.name)) + self.table.setItem(row, self.COL_NUMBER, QTableWidgetItem("+" + item.number)) + entry = self.history.get(item.number) + if entry and item.contacted: + when = (entry.last_sent or entry.first_sent or entry.at)[:10] + text = f"Contacted {when}" + elif entry: + text = outreach.STATUS_LABELS.get(entry.status, entry.status) + else: + text = "New" + status = QTableWidgetItem(text) + if item.contacted: + status.setForeground(self.p.c("warning")) + elif entry: + status.setForeground(self.p.c("text_dim")) + self.table.setItem(row, self.COL_STATUS, status) + self.table.blockSignals(False) + if self.items: + self.table.setCurrentCell(0, self.COL_NAME) + skipped = len(self._leads) - len(self.items) + contacted = sum(1 for i in self.items if i.contacted) + parts = [self._filter_label or "current filter", + f"{len(self.items)} with WhatsApp"] + if contacted: + parts.append(f"{contacted} already contacted") + if skipped: + parts.append(f"{skipped} without a usable number / duplicates") + self.sub_label.setText(" · ".join(parts)) + self._sync_preview() + self._sync_footer() + + # -- interaction ------------------------------------------------------------ + def _tick(self, mode: str) -> None: + self.table.blockSignals(True) + for row, item in enumerate(self.items): + on = (mode == "all") or (mode == "new" and not item.contacted) + self.table.item(row, self.COL_CHECK).setCheckState( + Qt.Checked if on else Qt.Unchecked) + self.table.blockSignals(False) + self._sync_footer() + + def _on_item_changed(self, cell: QTableWidgetItem) -> None: + if cell.column() == self.COL_CHECK: + self._sync_footer() + + def _checked(self) -> List[outreach.OutreachItem]: + return [item for row, item in enumerate(self.items) + if self.table.item(row, self.COL_CHECK).checkState() == Qt.Checked] + + def _sync_preview(self) -> None: + row = self.table.currentRow() + if 0 <= row < len(self.items): + item = self.items[row] + self.preview.setPlainText( + f"To: {item.name} (+{item.number})\n\n" + + (item.message or "(empty message)")) + else: + self.preview.setPlainText("No lead with a WhatsApp number in this filter.") + + def current_safety(self) -> outreach.SafetySettings: + return outreach.SafetySettings( + min_delay=self.min_delay.value(), + max_delay=self.max_delay.value(), + burst_size=self.burst_size.value(), + burst_rest=self.burst_rest.value() * 60, + warn_after=self.warn_after.value(), + daily_limit=self.daily_limit.value(), + ).normalized() + + def _sync_footer(self, *_: Any) -> None: + chosen = self._checked() + safety = self.current_safety() + today = self.history.sent_today() + limit = f"/{safety.daily_limit}" if safety.daily_limit else "" + eta = outreach.format_duration(outreach.estimate_seconds(len(chosen), safety)) + text = (f"{len(chosen)} selected · sent today {today}{limit} · " + f"about {eta} with these pauses") + if safety.daily_limit and today + len(chosen) > safety.daily_limit: + text += " · goes over the daily limit (you will be asked)" + self.summary.setText(text) + self.start_btn.setEnabled(bool(chosen)) + + def _edit_template(self) -> None: + from .lead_panel import WhatsAppTemplateDialog + dialog = WhatsAppTemplateDialog(self._leads, self.p, parent=self) + if dialog.exec() == QDialog.Accepted: + ticks = {item.number: self.table.item(row, self.COL_CHECK).checkState() + for row, item in enumerate(self.items)} + self._fill() + self.table.blockSignals(True) + for row, item in enumerate(self.items): + if item.number in ticks: + self.table.item(row, self.COL_CHECK).setCheckState(ticks[item.number]) + self.table.blockSignals(False) + self._sync_footer() + + def accept(self) -> None: # noqa: D102 (Qt API) + self.selected = self._checked() + if not self.selected: + return + self.safety = self.current_safety() + save_safety(self.safety) + super().accept() + + +# --------------------------------------------------------------------------- # +# the sender +# --------------------------------------------------------------------------- # + +class WhatsAppSenderWindow(QDialog): + """WhatsApp Web inside MAPLEAD, sending the queue with safety pauses.""" + + history_changed = Signal() + + # states of the one-second loop + IDLE, LOGIN, OPENING, VERIFYING, WAITING, DONE = ( + "idle", "login", "opening", "verifying", "waiting", "done") + + def __init__(self, items: Sequence[outreach.OutreachItem], + safety: outreach.SafetySettings, palette: Palette, *, + history: Optional[outreach.History] = None, + parent: Optional[QWidget] = None): + super().__init__(parent) + from PySide6.QtWebEngineCore import QWebEnginePage + from PySide6.QtWebEngineWidgets import QWebEngineView + + self.p = palette + self.items: List[outreach.OutreachItem] = list(items) + for item in self.items: + item.status = "" + self.safety = safety.normalized() + self.pacer = outreach.Pacer(self.safety) + self.history = history or outreach.History() + + self._state = self.IDLE + self._index = -1 # item being handled + self._elapsed = 0 # seconds in the current state + self._remaining = 0 # countdown while WAITING + self._wait_reason = "" + self._sent_run = 0 + self._paused = False + self._probing = False + self._daily_ok = False # user agreed to go over the cap + self._enter_tried = False + self._loaded = False # the current lead's page finished loading + + self.setWindowTitle("MAPLEAD — WhatsApp sender") + self.setModal(False) + self.setMinimumSize(1100, 700) + self.resize(1320, 860) + self.setSizeGripEnabled(True) + self.setWindowFlags(self.windowFlags() | Qt.WindowMinMaxButtonsHint + | Qt.WindowCloseButtonHint) + + root = QHBoxLayout(self) + root.setContentsMargins(12, 12, 12, 12) + root.setSpacing(12) + + self.view = QWebEngineView(self) + self.page = QWebEnginePage(_profile(), self.view) + self.view.setPage(self.page) + self.page.loadStarted.connect(lambda: setattr(self, "_loaded", False)) + self.page.loadFinished.connect(lambda ok: setattr(self, "_loaded", True)) + root.addWidget(self.view, 1) + + side = QWidget() + side.setFixedWidth(340) + sl = QVBoxLayout(side) + sl.setContentsMargins(0, 0, 0, 0) + sl.setSpacing(12) + + status_card, stc = card("Bulk send") + self.state_label = QLabel("") + self.state_label.setObjectName("panelTitle") + self.state_label.setWordWrap(True) + stc.addWidget(self.state_label) + self.detail_label = QLabel("") + self.detail_label.setObjectName("hint") + self.detail_label.setWordWrap(True) + stc.addWidget(self.detail_label) + self.progress = QProgressBar() + self.progress.setRange(0, max(1, len(self.items))) + self.progress.setValue(0) + self.progress.setTextVisible(False) + stc.addWidget(self.progress) + self.counts_label = QLabel("") + self.counts_label.setObjectName("hint") + stc.addWidget(self.counts_label) + + buttons = QHBoxLayout() + buttons.setSpacing(6) + self.pause_btn = QPushButton("Pause") + self.pause_btn.clicked.connect(self._toggle_pause) + buttons.addWidget(self.pause_btn) + self.skip_btn = QPushButton("Skip lead") + self.skip_btn.setToolTip("Do not message the current lead; go to the next one") + self.skip_btn.clicked.connect(self._skip) + buttons.addWidget(self.skip_btn) + self.stop_btn = QPushButton("Stop") + self.stop_btn.setProperty("danger", True) + self.stop_btn.clicked.connect(self._confirm_stop) + buttons.addWidget(self.stop_btn) + stc.addLayout(buttons) + sl.addWidget(status_card) + + log_card, lc = card("Activity") + self.log = LogPane(self.p) + lc.addWidget(self.log) + sl.addWidget(log_card, 1) + + hint = QLabel("Keep this window open while it sends. You can minimise " + "it; closing it stops the run. The WhatsApp session is " + "kept for next time.") + hint.setObjectName("hint") + hint.setWordWrap(True) + sl.addWidget(hint) + root.addWidget(side) + + self.timer = QTimer(self) + self.timer.setInterval(1000) + self.timer.timeout.connect(self._tick) + + self._sync_counts() + self.log.write(f"{len(self.items)} lead(s) queued · pause " + f"{self.safety.min_delay}-{self.safety.max_delay} s · rest " + f"{self.safety.burst_rest // 60} min every " + f"{self.safety.burst_size or '∞'} · ask after " + f"{self.safety.warn_after or '∞'}", "info") + + # -- lifecycle ------------------------------------------------------------ + def start(self) -> None: + self._set_state(self.LOGIN, "Opening WhatsApp Web…", + "If a QR code shows up, scan it with WhatsApp on your " + "phone (Linked devices). This is needed only once.") + self.page.load(QUrl(WHATSAPP_WEB)) + self.timer.start() + + def _set_state(self, state: str, title: str = "", detail: str = "") -> None: + self._state = state + self._elapsed = 0 + if title: + self.state_label.setText(title) + self.detail_label.setText(detail) + running = state not in (self.DONE, self.IDLE) + self.pause_btn.setEnabled(running) + self.skip_btn.setEnabled(state in (self.OPENING, self.VERIFYING, self.WAITING)) + self.stop_btn.setEnabled(running) + + def _sync_counts(self) -> None: + c = outreach.counts(self.items) + done = sum(c.values()) + self.progress.setValue(done) + self.counts_label.setText( + f"{done}/{len(self.items)} done · sent {c[outreach.SENT]} · " + f"not on WhatsApp {c[outreach.INVALID]} · failed {c[outreach.FAILED]}" + + (f" · skipped {c[outreach.SKIPPED]}" if c[outreach.SKIPPED] else "")) + + def _current(self) -> Optional[outreach.OutreachItem]: + if 0 <= self._index < len(self.items): + return self.items[self._index] + return None + + # -- the loop --------------------------------------------------------------- + def _tick(self) -> None: + if self._state == self.WAITING: + if self._paused: + self.detail_label.setText( + f"Paused · next lead in {outreach.format_duration(self._remaining)}") + return + self._remaining -= 1 + nxt = self._next_index() + who = self.items[nxt].name if nxt is not None else "" + self.detail_label.setText( + f"{self._wait_reason.capitalize()} · next: {who} in " + f"{outreach.format_duration(self._remaining)}") + if self._remaining <= 0: + self._open_next() + return + if self._state in (self.LOGIN, self.OPENING, self.VERIFYING): + self._elapsed += 1 + # never probe the previous lead's page while the next one loads + if self._state == self.OPENING and not self._loaded: + if self._elapsed >= OPEN_TIMEOUT: + self._finish(outreach.FAILED, "WhatsApp Web did not load") + return + if not self._probing: + self._probing = True + self.page.runJavaScript(_JS_PROBE, 0, self._on_probe) + + def _on_probe(self, raw: Any) -> None: + self._probing = False + try: + info = json.loads(raw) if isinstance(raw, str) else {} + except ValueError: + info = {} + state = self._state + + if state == self.LOGIN: + if info.get("logged"): + self.log.write("WhatsApp Web is ready", "ok") + self._open_next() + elif info.get("qr"): + self.state_label.setText("Scan the QR code") + self.detail_label.setText( + "On your phone: WhatsApp → Settings → Linked devices → " + "Link a device. Sending starts by itself once you are in.") + return + + item = self._current() + if item is None: + return + + if state == self.OPENING: + dialog = str(info.get("dialog") or "") + if dialog and _INVALID_RE.search(dialog) and not _LOADING_RE.search(dialog): + self._finish(outreach.INVALID) + elif info.get("send"): + self.page.runJavaScript(_JS_CLICK_SEND, 0, self._on_clicked) + elif info.get("qr") and not info.get("logged"): + self.log.write("WhatsApp Web logged out — scan the QR code again", "warn") + self._index -= 1 + self._set_state(self.LOGIN, "Scan the QR code", "Session expired.") + elif self._elapsed >= OPEN_TIMEOUT: + self._finish(outreach.FAILED, "the chat did not open in time") + return + + if state == self.VERIFYING: + composer = info.get("composer", -1) + if composer == 0 and not info.get("send"): + self._finish(outreach.SENT) + elif self._elapsed >= VERIFY_TIMEOUT: + self._finish(outreach.FAILED, "WhatsApp did not confirm the send") + elif self._elapsed >= ENTER_FALLBACK_AT and not self._enter_tried: + self._enter_tried = True + self.page.runJavaScript(_JS_FOCUS_COMPOSER, 0, + lambda ok: self._press_enter() if ok else None) + + def _on_clicked(self, ok: Any) -> None: + if self._state != self.OPENING: + return + if ok: + self._enter_tried = False + item = self._current() + self._set_state(self.VERIFYING, f"Sending to {item.name if item else ''}…", + "Waiting for WhatsApp to confirm.") + + def _press_enter(self) -> None: + target = self.view.focusProxy() or self.view + for kind in (QEvent.KeyPress, QEvent.KeyRelease): + QApplication.sendEvent(target, QKeyEvent(kind, Qt.Key_Return, + Qt.NoModifier, "\r")) + + def _next_index(self) -> Optional[int]: + nxt = self._index + 1 + return nxt if nxt < len(self.items) else None + + def _open_next(self) -> None: + nxt = self._next_index() + if nxt is None: + self._done("All leads handled") + return + # daily cap: ask once per run before going over it + if (not self._daily_ok + and self.pacer.over_daily_limit(self.history.sent_today())): + self.timer.stop() + answer = QMessageBox.question( + self, "Daily limit reached", + f"{self.history.sent_today()} messages were already sent today " + f"(your limit is {self.safety.daily_limit}).\n\n" + "Going over it raises the chance of WhatsApp restricting or " + "blocking this number.\n\nContinue anyway?", + QMessageBox.Yes | QMessageBox.No, QMessageBox.No) + if answer != QMessageBox.Yes: + self._done("Stopped at the daily limit") + return + self._daily_ok = True + self.log.write("Daily limit passed — continuing on your confirmation", "warn") + self.timer.start() + + self._index = nxt + item = self.items[nxt] + from .lead_panel import load_message_template + # rendered now, so {saudacao} matches the time it is actually sent + item.message = contacts.render_template(load_message_template(), item.lead) + url = (f"{WHATSAPP_SEND}?phone={item.number}" + f"&text={quote(item.message, safe='')}") + self._set_state(self.OPENING, f"Opening {item.name}", + f"+{item.number} · lead {nxt + 1} of {len(self.items)}") + self.log.write(f"→ {item.name} (+{item.number})", "query") + self._loaded = False + self.page.load(QUrl(url)) + + def _finish(self, status: str, error: str = "") -> None: + item = self._current() + if item is None: + return + item.status = status + try: + self.history.record(item.number, status, name=item.name, + error=error, lead=item.lead) + except OSError as exc: + self.log.write(f"Could not save the history: {exc}", "error") + self.history_changed.emit() + label = outreach.STATUS_LABELS.get(status, status) + level = {outreach.SENT: "ok", outreach.INVALID: "warn", + outreach.SKIPPED: "info"}.get(status, "error") + self.log.write(f" {label}" + (f" — {error}" if error else ""), level) + self._sync_counts() + + if self._next_index() is None: + self._done("All leads handled") + return + + if status == outreach.SENT: + self._sent_run += 1 + if self.pacer.needs_warning(self._sent_run): + self.timer.stop() + answer = QMessageBox.question( + self, "Keep sending?", + f"{self._sent_run} messages were sent in this run.\n\n" + "After this many messages WhatsApp may restrict or block " + "the number. Do you want to continue?", + QMessageBox.Yes | QMessageBox.No, QMessageBox.No) + if answer != QMessageBox.Yes: + self._done(f"Stopped after {self._sent_run} messages") + return + self.log.write("Continuing on your confirmation", "warn") + self.timer.start() + delay, reason = self.pacer.delay_after(self._sent_run) + else: + delay, reason = self.pacer.short_delay(), "short pause" + self._remaining = delay + self._wait_reason = reason + self._set_state(self.WAITING, "Waiting" if not self._paused else "Paused", "") + if reason.startswith("rest"): + self.log.write(f"Resting {outreach.format_duration(delay)} " + f"({reason})", "info") + + def _done(self, title: str) -> None: + self.timer.stop() + c = outreach.counts(self.items) + self._set_state(self.DONE, title, + f"Sent {c[outreach.SENT]} · not on WhatsApp " + f"{c[outreach.INVALID]} · failed {c[outreach.FAILED]}. " + "The history is saved; these numbers are skipped next time.") + self.log.write(title, "ok") + self.pause_btn.setText("Pause") + self._paused = False + + # -- controls ----------------------------------------------------------------- + def _toggle_pause(self) -> None: + self._paused = not self._paused + self.pause_btn.setText("Resume" if self._paused else "Pause") + if self._paused: + self.log.write("Paused — the current lead finishes, then it waits", "warn") + if self._state == self.WAITING: + self.state_label.setText("Paused") + else: + self.log.write("Resumed", "info") + if self._state == self.WAITING: + self.state_label.setText("Waiting") + + def _skip(self) -> None: + if self._state in (self.OPENING, self.VERIFYING): + self._finish(outreach.SKIPPED, "skipped by you") + elif self._state == self.WAITING: + nxt = self._next_index() + if nxt is None: + return + self._index = nxt + self._finish(outreach.SKIPPED, "skipped by you") + + def is_running(self) -> bool: + return self._state not in (self.IDLE, self.DONE) + + def _confirm_stop(self) -> bool: + if not self.is_running(): + return True + answer = QMessageBox.question( + self, "Stop sending?", + "Stop the bulk send? Leads not reached yet stay unmarked and can be " + "sent later.", QMessageBox.Yes | QMessageBox.No, QMessageBox.No) + if answer == QMessageBox.Yes: + self._done("Stopped by you") + return True + return False + + def closeEvent(self, event) -> None: # noqa: N802 (Qt API) + if self.is_running() and not self._confirm_stop(): + event.ignore() + return + self.timer.stop() + self.page.load(QUrl("about:blank")) + super().closeEvent(event) + + +def open_bulk_sender(leads: Sequence[Dict[str, Any]], palette: Palette, *, + dial: str = "", country_code: str = "", + filter_label: str = "", + parent: Optional[QWidget] = None + ) -> Optional[WhatsAppSenderWindow]: + """Setup dialog → sender window. Returns the window (or None if cancelled).""" + if not webengine_available(): + QMessageBox.warning( + parent, "Bulk WhatsApp", + "This build of MAPLEAD has no web engine (QtWebEngine), which bulk " + "sending needs to run WhatsApp Web.\n\nInstall PySide6-WebEngine or " + "use a release build that includes it.") + return None + dialog = BulkSendDialog(leads, palette, dial=dial, country_code=country_code, + filter_label=filter_label, parent=parent) + if not dialog.items: + QMessageBox.information( + parent, "Bulk WhatsApp", + "None of the leads in this filter has a phone number that can be " + "opened in WhatsApp.") + return None + if dialog.exec() != QDialog.Accepted or not dialog.selected: + return None + window = WhatsAppSenderWindow(dialog.selected, dialog.safety, palette, + history=dialog.history, parent=parent) + window.setAttribute(Qt.WA_DeleteOnClose, True) + window.show() + window.start() + return window diff --git a/maplead/gui/lead_panel.py b/maplead/gui/lead_panel.py index a31ec75..97fb03f 100644 --- a/maplead/gui/lead_panel.py +++ b/maplead/gui/lead_panel.py @@ -28,7 +28,7 @@ QMessageBox, QPlainTextEdit, QPushButton, QScrollArea, QVBoxLayout, QWidget) -from .. import contacts +from .. import contacts, outreach from ..data import fields as FD from . import icons from .theme import Palette @@ -45,6 +45,7 @@ ("site", "Has website"), ("nos", "No website"), ("email", "Has e-mail"), + ("new", "Not contacted"), ) SORTS: Tuple[Tuple[str, str], ...] = ( @@ -336,6 +337,9 @@ def __init__(self, leads: Sequence[Dict[str, Any]], columns: Sequence[str], self._has_wa: List[bool] = [] self._has_site: List[bool] = [] self._has_email: List[bool] = [] + self._contacted: List[bool] = [] + self._history = outreach.History() + self._sender = None self._rendered = 0 # typing in the search box rebuilds the cards: debounce it self._search_timer = QTimer(self) @@ -401,6 +405,14 @@ def _build_header(self) -> QWidget: message_btn.clicked.connect(self.edit_template) row.addWidget(message_btn) + bulk_btn = QPushButton(" Bulk send") + bulk_btn.setIcon(icons.icon("play", self.p.accent_text, 13)) + bulk_btn.setProperty("accent", True) + bulk_btn.setToolTip("Send the saved message to every lead in the current " + "filter through WhatsApp Web, with safety pauses") + bulk_btn.clicked.connect(self.open_bulk_sender) + row.addWidget(bulk_btn) + close_btn = QPushButton("Close") close_btn.clicked.connect(self.close) row.addWidget(close_btn) @@ -471,6 +483,9 @@ def _precompute(self) -> None: for lead in self._leads] self._has_email = [bool(str(lead.get("email") or "").strip()) for lead in self._leads] + self._contacted = [self._history.was_contacted( + contacts.whatsapp_number(lead, dial, iso) or "-") + for lead in self._leads] def set_leads(self, leads: Sequence[Dict[str, Any]], columns: Optional[Sequence[str]] = None) -> None: @@ -518,6 +533,8 @@ def _passes(self, index: int, needle: str) -> bool: return not self._has_site[index] if key == "email": return self._has_email[index] + if key == "new": + return self._has_wa[index] and not self._contacted[index] return True @staticmethod @@ -620,7 +637,11 @@ def _presence_chips(self, lead: Dict[str, Any]) -> List[QLabel]: if self._has("phone") and str(lead.get("phone") or "").strip(): chips.append(chip("Phone", "ok")) if self._show_wa(lead): - chips.append(chip("WhatsApp", "wa")) + number = contacts.whatsapp_number(lead, self._dial, self._iso) + if self._history.was_contacted(number): + chips.append(chip("Contacted", "warn")) + else: + chips.append(chip("WhatsApp", "wa")) if self._has("email") and str(lead.get("email") or "").strip(): chips.append(chip("E-mail", "ok")) status = str(lead.get("status") or "").upper() @@ -915,5 +936,31 @@ def edit_template(self) -> None: self._template = load_message_template() self._status("WhatsApp message saved — it opens with every chat") + def open_bulk_sender(self) -> None: + """Queue the leads of the current filter for bulk WhatsApp sending.""" + if self._sender is not None and self._sender.is_running(): + self._sender.showNormal() + self._sender.raise_() + self._sender.activateWindow() + return + from .bulk_sender import open_bulk_sender + label = dict(FILTERS).get(self._filter_key, "") + if self.search.text().strip(): + label += f" · “{self.search.text().strip()}”" + window = open_bulk_sender(self._visible, self.p, dial=self._dial, + country_code=self._iso, filter_label=label, + parent=self) + if window is None: + return + self._sender = window + window.history_changed.connect(self._reload_history) + window.destroyed.connect(lambda *_: setattr(self, "_sender", None)) + self._status(f"Bulk send started for {len(window.items)} lead(s)") + + def _reload_history(self) -> None: + self._history.load() + self._precompute() + self._apply_filters(reset=False) + def _status(self, text: str) -> None: self.status_label.setText(text) diff --git a/maplead/outreach.py b/maplead/outreach.py new file mode 100644 index 0000000..18dd48c --- /dev/null +++ b/maplead/outreach.py @@ -0,0 +1,326 @@ +""" +Bulk WhatsApp outreach: the queue, the safety pacing and the contact history. + +Pure standard library, like ``contacts.py`` — the GUI drives the actual sending +(WhatsApp Web inside a Qt window), but every decision about *who* gets a message, +*when* the next one goes out and *whether to ask the user first* lives here, so +it can be tested without a display. + +Three pieces: + +* :class:`SafetySettings` — random delay between messages, a longer rest after + every burst, the "this many messages may get the number blocked, continue?" + threshold and a daily cap. Defaults are conservative on purpose. +* :class:`History` — every number ever messaged, with status and date, stored as + JSON in the user data folder. Numbers already *sent* are skipped next time. +* :func:`build_queue` — turns the leads the user picked into + :class:`OutreachItem` s (number + rendered message), dropping duplicates and + numbers already contacted. +""" + +from __future__ import annotations + +import csv +import datetime +import json +import os +import random +import tempfile +from dataclasses import asdict, dataclass, field, fields +from typing import Any, Dict, Iterable, List, Mapping, Optional, Sequence, Tuple + +from . import contacts, paths + +HISTORY_FILE = "whatsapp-history.json" + +# delivery outcomes recorded in the history +SENT = "sent" +FAILED = "failed" +INVALID = "invalid" # the number is not on WhatsApp +SKIPPED = "skipped" # the user skipped it during the run + +STATUS_LABELS: Dict[str, str] = { + SENT: "Sent", + FAILED: "Failed", + INVALID: "Not on WhatsApp", + SKIPPED: "Skipped", +} + + +# --------------------------------------------------------------------------- # +# safety +# --------------------------------------------------------------------------- # + +@dataclass +class SafetySettings: + """How fast the bulk sender may go. All times in seconds.""" + + min_delay: int = 45 # random pause between two messages ... + max_delay: int = 120 # ... somewhere in [min_delay, max_delay] + burst_size: int = 10 # after this many messages ... + burst_rest: int = 600 # ... rest this long + warn_after: int = 25 # ask "continue?" after this many in one run + daily_limit: int = 50 # ask again once today's total reaches this + + def normalized(self) -> "SafetySettings": + """A copy with every value clamped into a sane range.""" + lo = max(5, int(self.min_delay)) + hi = max(lo, int(self.max_delay)) + return SafetySettings( + min_delay=lo, + max_delay=hi, + burst_size=max(0, int(self.burst_size)), + burst_rest=max(0, int(self.burst_rest)), + warn_after=max(0, int(self.warn_after)), + daily_limit=max(0, int(self.daily_limit)), + ) + + def to_dict(self) -> Dict[str, int]: + return asdict(self) + + @classmethod + def from_dict(cls, data: Optional[Mapping[str, Any]]) -> "SafetySettings": + base = cls() + if not data: + return base + values: Dict[str, int] = {} + for f in fields(cls): + raw = data.get(f.name, getattr(base, f.name)) + try: + values[f.name] = int(raw) + except (TypeError, ValueError): + values[f.name] = getattr(base, f.name) + return cls(**values).normalized() + + +class Pacer: + """Decides the wait after each message and when to ask the user. + + ``sent`` counts messages that actually went out in this run (failures and + invalid numbers do not count towards bursts or the warning — they never + reached anyone). + """ + + def __init__(self, settings: SafetySettings, + rng: Optional[random.Random] = None): + self.s = settings.normalized() + self.rng = rng or random.Random() + + def delay_after(self, sent: int) -> Tuple[int, str]: + """Seconds to wait before the next message, and why.""" + s = self.s + if s.burst_size and sent > 0 and sent % s.burst_size == 0 and s.burst_rest: + return s.burst_rest, f"rest after {sent} messages" + return self.rng.randint(s.min_delay, s.max_delay), "random pause" + + def short_delay(self) -> int: + """Pause after a number that was not messaged (invalid / failed).""" + return self.rng.randint(5, 12) + + def needs_warning(self, sent: int) -> bool: + """True when the run just crossed the "may get blocked" threshold.""" + w = self.s.warn_after + return bool(w) and sent > 0 and sent % w == 0 + + def over_daily_limit(self, sent_today: int) -> bool: + limit = self.s.daily_limit + return bool(limit) and sent_today >= limit + + +# --------------------------------------------------------------------------- # +# history +# --------------------------------------------------------------------------- # + +def _now() -> datetime.datetime: + return datetime.datetime.now().replace(microsecond=0) + + +@dataclass +class HistoryEntry: + number: str + name: str = "" + status: str = SENT + at: str = "" # ISO timestamp of the latest attempt + first_sent: str = "" # ISO timestamp of the first successful send + last_sent: str = "" # ISO timestamp of the latest successful send + attempts: int = 0 + error: str = "" + city: str = "" + category: str = "" + + +class History: + """Every number the bulk sender touched, keyed by ``wa.me`` digits.""" + + def __init__(self, path: Optional[str] = None): + self.path = path or paths.state_file(HISTORY_FILE) + self.entries: Dict[str, HistoryEntry] = {} + self.load() + + # -- storage ---------------------------------------------------------- + def load(self) -> None: + self.entries = {} + try: + with open(self.path, "r", encoding="utf-8") as fh: + data = json.load(fh) + except (OSError, ValueError): + return + known = {f.name for f in fields(HistoryEntry)} + for row in data.get("entries", []) if isinstance(data, dict) else []: + if not isinstance(row, dict) or not row.get("number"): + continue + entry = HistoryEntry(**{k: v for k, v in row.items() if k in known}) + self.entries[entry.number] = entry + + def save(self) -> None: + folder = os.path.dirname(self.path) or "." + os.makedirs(folder, exist_ok=True) + payload = {"version": 1, + "entries": [asdict(e) for e in self.entries.values()]} + fd, tmp = tempfile.mkstemp(prefix=".history-", dir=folder) + try: + with os.fdopen(fd, "w", encoding="utf-8") as fh: + json.dump(payload, fh, ensure_ascii=False, indent=1) + os.replace(tmp, self.path) + except OSError: + try: + os.unlink(tmp) + except OSError: + pass + raise + + # -- queries ---------------------------------------------------------- + def get(self, number: str) -> Optional[HistoryEntry]: + return self.entries.get(contacts.digits_only(number)) + + def was_contacted(self, number: str) -> bool: + """True once a message to this number actually went out.""" + entry = self.get(number) + return bool(entry and (entry.status == SENT or entry.first_sent)) + + def sent_on(self, day: datetime.date) -> int: + prefix = day.isoformat() + return sum(1 for e in self.entries.values() + if e.last_sent.startswith(prefix)) + + def sent_today(self) -> int: + return self.sent_on(_now().date()) + + # -- updates ---------------------------------------------------------- + def record(self, number: str, status: str, *, name: str = "", + error: str = "", lead: Optional[Mapping[str, Any]] = None, + when: Optional[datetime.datetime] = None, + save: bool = True) -> HistoryEntry: + digits = contacts.digits_only(number) + stamp = (when or _now()).isoformat() + entry = self.entries.get(digits) or HistoryEntry(number=digits) + entry.name = name or entry.name + entry.attempts += 1 + entry.at = stamp + entry.error = error + if lead: + entry.city = str(lead.get("city") or entry.city or "") + entry.category = str(lead.get("category") or entry.category or "") + # a later failure never hides that the lead was already messaged + if status == SENT: + entry.status = SENT + entry.first_sent = entry.first_sent or stamp + entry.last_sent = stamp + elif not entry.first_sent: + entry.status = status + self.entries[digits] = entry + if save: + self.save() + return entry + + def forget(self, number: str) -> bool: + removed = self.entries.pop(contacts.digits_only(number), None) is not None + if removed: + self.save() + return removed + + def export_csv(self, path: str) -> str: + cols = [f.name for f in fields(HistoryEntry)] + with open(path, "w", encoding="utf-8-sig", newline="") as fh: + writer = csv.writer(fh) + writer.writerow(cols) + for entry in sorted(self.entries.values(), key=lambda e: e.at): + writer.writerow([getattr(entry, c) for c in cols]) + return path + + +# --------------------------------------------------------------------------- # +# queue +# --------------------------------------------------------------------------- # + +@dataclass +class OutreachItem: + number: str # wa.me digits + name: str + message: str + lead: Dict[str, Any] = field(repr=False, default_factory=dict) + contacted: bool = False # already in the history as sent + status: str = "" # filled in while the run goes + + +def build_queue(leads: Iterable[Mapping[str, Any]], template: str, *, + dial: str = "", iso: str = "", + history: Optional[History] = None, + skip_contacted: bool = True, + when: Optional[datetime.datetime] = None) -> List[OutreachItem]: + """One item per distinct WhatsApp number, in the order given. + + Leads without a usable number are dropped. With ``skip_contacted`` the + numbers already messaged (per ``history``) are dropped too; otherwise they + stay in, flagged ``contacted=True`` so the UI can leave them unticked. + """ + seen = set() + queue: List[OutreachItem] = [] + for lead in leads: + number = contacts.whatsapp_number(lead, dial, iso) + if not number or number in seen: + continue + seen.add(number) + contacted = bool(history and history.was_contacted(number)) + if contacted and skip_contacted: + continue + queue.append(OutreachItem( + number=number, + name=str(lead.get("name") or "").strip() or number, + message=contacts.render_template(template, lead, when=when), + lead=dict(lead), + contacted=contacted, + )) + return queue + + +def estimate_seconds(count: int, settings: SafetySettings) -> int: + """Rough duration of a run of `count` messages (average pauses + rests).""" + if count <= 0: + return 0 + s = settings.normalized() + per_message = 15 # load + send + pauses = (count - 1) * (s.min_delay + s.max_delay) / 2 + rests = 0 + if s.burst_size: + rests = ((count - 1) // s.burst_size) * s.burst_rest + return int(count * per_message + pauses + rests) + + +def format_duration(seconds: float) -> str: + seconds = max(0, int(round(seconds))) + hours, rest = divmod(seconds, 3600) + minutes, secs = divmod(rest, 60) + if hours: + return f"{hours} h {minutes:02d} min" + if minutes: + return f"{minutes} min {secs:02d} s" + return f"{secs} s" + + +def counts(items: Sequence[OutreachItem]) -> Dict[str, int]: + out = {SENT: 0, FAILED: 0, INVALID: 0, SKIPPED: 0} + for item in items: + if item.status in out: + out[item.status] += 1 + return out diff --git a/packaging/maplead.spec b/packaging/maplead.spec index f5a95e3..61f5408 100644 --- a/packaging/maplead.spec +++ b/packaging/maplead.spec @@ -17,8 +17,10 @@ EXCLUDES = [ "PySide6.QtQml", "PySide6.QtQuick", "PySide6.QtQuickWidgets", "PySide6.QtQuick3D", "PySide6.Qt3DCore", "PySide6.Qt3DRender", "PySide6.Qt3DInput", "PySide6.Qt3DLogic", "PySide6.Qt3DAnimation", "PySide6.Qt3DExtras", "PySide6.QtCharts", "PySide6.QtDataVisualization", - "PySide6.QtMultimedia", "PySide6.QtMultimediaWidgets", "PySide6.QtWebEngineCore", - "PySide6.QtWebEngineWidgets", "PySide6.QtWebChannel", "PySide6.QtWebSockets", + "PySide6.QtMultimedia", "PySide6.QtMultimediaWidgets", + # QtWebEngineCore/Widgets + QtWebChannel stay: the bulk WhatsApp sender + # runs WhatsApp Web inside the app + "PySide6.QtWebSockets", "PySide6.QtBluetooth", "PySide6.QtNfc", "PySide6.QtPositioning", "PySide6.QtLocation", "PySide6.QtSensors", "PySide6.QtSerialPort", "PySide6.QtTest", "PySide6.QtHelp", "PySide6.QtDesigner", "PySide6.QtUiTools", "PySide6.QtSql", "PySide6.QtPdf", @@ -44,6 +46,10 @@ a = Analysis( # noqa: F821 "maplead.gui.icons", "maplead.gui.theme", "maplead.contacts", + "maplead.outreach", + "maplead.gui.bulk_sender", + "PySide6.QtWebEngineCore", + "PySide6.QtWebEngineWidgets", # `--cli` imports the wizard lazily, so PyInstaller cannot see it "maplead.cli", "PySide6.QtSvg", diff --git a/tools/bulk_test.py b/tools/bulk_test.py new file mode 100644 index 0000000..c60d830 --- /dev/null +++ b/tools/bulk_test.py @@ -0,0 +1,63 @@ +#!/usr/bin/env python3 +""" +Try the bulk WhatsApp sender on your OWN numbers, without a search. + + .venv-gui/bin/python tools/bulk_test.py # asks for the numbers + .venv-gui/bin/python tools/bulk_test.py +351912345678 +351934567890 + +Never test with demo leads: their numbers are made up and may belong to real +people. Each number becomes a fake lead ("Teste 1", "Teste 2"…) so the saved +message template is rendered exactly like it would be for a real lead. The +history is the real one, so the numbers show as "Contacted" afterwards; use +``--forget`` to remove them from the history again. +""" + +from __future__ import annotations + +import os +import sys + +ROOT = os.path.dirname(os.path.dirname(os.path.abspath(__file__))) +sys.path.insert(0, ROOT) + + +def main(argv) -> int: + forget = "--forget" in argv + numbers = [a for a in argv if not a.startswith("--")] + + from maplead import outreach + if forget: + history = outreach.History() + for number in numbers: + print(number, "removed" if history.forget(number) else "not in history") + return 0 + + from maplead.gui import app as app_mod + qapp, _ = app_mod.create_app([]) + from PySide6.QtWidgets import QInputDialog + from maplead.gui import bulk_sender, theme + + if not numbers: + text, ok = QInputDialog.getText( + None, "Bulk WhatsApp test", + "Your own test numbers, with country code, separated by spaces\n" + "(e.g. +351912345678 +351934567890):") + if not ok: + return 0 + numbers = text.replace(",", " ").split() + if not numbers: + return 0 + + leads = [{"name": f"Teste {i}", "phone": n, "city": "Sintra", + "category": "Teste"} for i, n in enumerate(numbers, 1)] + palette = theme.PALETTES[app_mod.preferred_theme()] + window = bulk_sender.open_bulk_sender(leads, palette, + filter_label="test numbers") + if window is None: + return 0 + window.finished.connect(qapp.quit) + return qapp.exec() + + +if __name__ == "__main__": + sys.exit(main(sys.argv[1:]))