feat(integration-whatsapp): core ingest do coletor de WhatsApp#273
Draft
Clintonrocha98 wants to merge 7 commits into
Draft
feat(integration-whatsapp): core ingest do coletor de WhatsApp#273Clintonrocha98 wants to merge 7 commits into
Clintonrocha98 wants to merge 7 commits into
Conversation
📝 WalkthroughWalkthroughThis PR introduces a complete WhatsApp event ingestion module for the Laravel monolith. The implementation follows a data-lake schema-on-read approach with HMAC-authenticated webhook endpoints, idempotency checks via Suggested reviewers
🚥 Pre-merge checks | ✅ 4 | ❌ 1❌ Failed checks (1 warning)
✅ Passed checks (4 passed)
✏️ Tip: You can configure your own custom pre-merge checks in the settings. Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
![coderabbitai[bot]](https://avatars.githubusercontent.com/in/347564?s=60&v=4){kind=small}
There was a problem hiding this comment.
Actionable comments posted: 1
🧹 Nitpick comments (3)
app-modules/integration-whatsapp/docs/spec.md (1)
63-78: ⚡ Quick winAdd language tags to fenced diagrams to satisfy markdown linting.
Both fenced blocks should declare a language (e.g.,
text) to avoid MD040 warnings and keep docs CI clean.Suggested diff
-``` +```text WhatsApp servers ↓ WebSocket (Baileys) [ wpp-tui ] ← repo Node separado · runtime · sessão · envia todos os eventos (sem filtro) @@ -``` +``` -``` +```text whatsapp_groups ├─ id uuid PK @@ -``` +```Also applies to: 109-141
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@app-modules/integration-whatsapp/docs/spec.md` around lines 63 - 78, The two fenced diagram blocks in the spec (the block starting with "WhatsApp servers ↓ WebSocket (Baileys) [ wpp-tui ]" and the block listing "whatsapp_groups ├─ id uuid PK ...") are missing language tags; update their opening ``` fences to include a language (e.g., ```text) so both diagrams and the other affected block range (lines around the second diagram referenced) are annotated and satisfy markdown linting (MD040).app-modules/integration-whatsapp/docs/plans/0001-ingest-implementation.md (1)
32-54: ⚡ Quick winAdd language identifiers to all unlabeled fenced blocks.
Several fenced blocks are missing a language tag; adding
text,bash, orenvas appropriate will clear MD040 warnings and improve readability.Also applies to: 58-69, 73-86, 90-96, 102-130, 226-228
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@app-modules/integration-whatsapp/docs/plans/0001-ingest-implementation.md` around lines 32 - 54, The Markdown file contains multiple unlabeled fenced code blocks (e.g., the architecture diagram and payload examples surrounding symbols like VerifyWhatsAppSignature, WhatsAppWebhookController, ProcessWhatsAppEvent, and the POST body schema) which trigger MD040; add appropriate language identifiers to each fence — use "text" for the ASCII diagram block, "json" for the request body example ({ type, group_jid, ... }), "bash" for any CLI snippets, and "env" for environment samples — so every fenced block has a language tag to clear the lint warnings and improve readability.app-modules/integration-whatsapp/CONTEXT.md (1)
25-38: ⚡ Quick winDeclare a language for the architecture fence block.
Use a language like
texton this fenced diagram to satisfy markdownlint MD040.🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@app-modules/integration-whatsapp/CONTEXT.md` around lines 25 - 38, The fenced architecture diagram at the top of the file is missing a language tag which triggers markdownlint MD040; update the opening triple-backtick of that diagram (the top-level architecture fence block) to include a language identifier such as text so the fence becomes ```text, ensuring the markdown linter recognizes it as a code block with an explicit language.
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Inline comments:
In `@app-modules/integration-whatsapp/docs/adr/0001-data-lake-approach.md`:
- Around line 70-77: Update the ADR text to replace the vague “revisit when the
exploration phase ends” with a concrete privacy review trigger: add a specific
calendar deadline (e.g., "Review by YYYY-MM-DD") and an accountable owner (e.g.,
"Owner: Data Privacy Lead / <name or role>"). Also document where the review
will evaluate the listed items (aggregating signals, TTL for whatsapp_events,
phone hashing/content minimization, and presence.update) so the owner and date
are clearly tied to those symbols (`whatsapp_events`, `presence.update`) in the
same paragraph.
---
Nitpick comments:
In `@app-modules/integration-whatsapp/CONTEXT.md`:
- Around line 25-38: The fenced architecture diagram at the top of the file is
missing a language tag which triggers markdownlint MD040; update the opening
triple-backtick of that diagram (the top-level architecture fence block) to
include a language identifier such as text so the fence becomes ```text,
ensuring the markdown linter recognizes it as a code block with an explicit
language.
In `@app-modules/integration-whatsapp/docs/plans/0001-ingest-implementation.md`:
- Around line 32-54: The Markdown file contains multiple unlabeled fenced code
blocks (e.g., the architecture diagram and payload examples surrounding symbols
like VerifyWhatsAppSignature, WhatsAppWebhookController, ProcessWhatsAppEvent,
and the POST body schema) which trigger MD040; add appropriate language
identifiers to each fence — use "text" for the ASCII diagram block, "json" for
the request body example ({ type, group_jid, ... }), "bash" for any CLI
snippets, and "env" for environment samples — so every fenced block has a
language tag to clear the lint warnings and improve readability.
In `@app-modules/integration-whatsapp/docs/spec.md`:
- Around line 63-78: The two fenced diagram blocks in the spec (the block
starting with "WhatsApp servers ↓ WebSocket (Baileys) [ wpp-tui ]" and the block
listing "whatsapp_groups ├─ id uuid PK ...") are missing language tags; update
their opening ``` fences to include a language (e.g., ```text) so both diagrams
and the other affected block range (lines around the second diagram referenced)
are annotated and satisfy markdown linting (MD040).
🪄 Autofix (Beta)
Fix all unresolved CodeRabbit comments on this PR:
- Push a commit to this branch (recommended)
- Create a new PR with the fixes
ℹ️ Review info
⚙️ Run configuration
Configuration used: Repository YAML (base), Central YAML (inherited)
Review profile: CHILL
Plan: Pro
Run ID: b6fb1b2d-3870-4d40-9845-c6d4061b98d8
⛔ Files ignored due to path filters (1)
composer.lockis excluded by!**/*.lock
📒 Files selected for processing (26)
.env.exampleapp-modules/integration-whatsapp/CONTEXT.mdapp-modules/integration-whatsapp/composer.jsonapp-modules/integration-whatsapp/config/whatsapp.phpapp-modules/integration-whatsapp/database/factories/WhatsAppEventFactory.phpapp-modules/integration-whatsapp/database/factories/WhatsAppGroupFactory.phpapp-modules/integration-whatsapp/database/factories/WhatsAppParticipantFactory.phpapp-modules/integration-whatsapp/database/migrations/2026_05_20_120000_create_whatsapp_groups_table.phpapp-modules/integration-whatsapp/database/migrations/2026_05_20_120001_create_whatsapp_participants_table.phpapp-modules/integration-whatsapp/database/migrations/2026_05_20_120002_create_whatsapp_events_table.phpapp-modules/integration-whatsapp/docs/adr/0001-data-lake-approach.mdapp-modules/integration-whatsapp/docs/plans/0001-ingest-implementation.mdapp-modules/integration-whatsapp/docs/spec.mdapp-modules/integration-whatsapp/routes/whatsapp-routes.phpapp-modules/integration-whatsapp/src/Ingest/Http/Controllers/WhatsAppWebhookController.phpapp-modules/integration-whatsapp/src/Ingest/Http/Middleware/VerifyWhatsAppSignature.phpapp-modules/integration-whatsapp/src/Ingest/Http/Requests/IngestEventRequest.phpapp-modules/integration-whatsapp/src/Ingest/Jobs/ProcessWhatsAppEvent.phpapp-modules/integration-whatsapp/src/IntegrationWhatsappServiceProvider.phpapp-modules/integration-whatsapp/src/Models/WhatsAppEvent.phpapp-modules/integration-whatsapp/src/Models/WhatsAppGroup.phpapp-modules/integration-whatsapp/src/Models/WhatsAppParticipant.phpapp-modules/integration-whatsapp/tests/Feature/Ingest/ProcessWhatsAppEventTest.phpapp-modules/integration-whatsapp/tests/Feature/Ingest/WebhookIngestTest.phpapp-modules/integration-whatsapp/tests/Feature/Models/WhatsAppModelsTest.phpcomposer.json
Comment on lines
+70
to
+77
| This decision **must be revisited when the exploration phase ends** — concretely, once the data team | ||
| has defined the metrics worth keeping. At that point, evaluate: | ||
|
|
||
| - Aggregating useful signals into a persistent metrics table and **dropping or truncating raw payloads**. | ||
| - Introducing a **retention TTL** for `whatsapp_events`. | ||
| - Re-introducing **phone hashing / content minimization** (with a shared pepper if hashing, to keep | ||
| identity linking working). | ||
| - Whether `presence.update` is still worth collecting at volume. |
There was a problem hiding this comment.
Make the privacy review trigger concrete (date + owner).
“Revisit when exploration phase ends” is too open-ended for a decision that explicitly stores raw phone/content with no TTL. Add a hard deadline and accountable owner to avoid indefinite retention by drift.
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
In `@app-modules/integration-whatsapp/docs/adr/0001-data-lake-approach.md` around
lines 70 - 77, Update the ADR text to replace the vague “revisit when the
exploration phase ends” with a concrete privacy review trigger: add a specific
calendar deadline (e.g., "Review by YYYY-MM-DD") and an accountable owner (e.g.,
"Owner: Data Privacy Lead / <name or role>"). Also document where the review
will evaluate the listed items (aggregating signals, TTL for whatsapp_events,
phone hashing/content minimization, and presence.update) so the owner and date
are clearly tied to those symbols (`whatsapp_events`, `presence.update`) in the
same paragraph.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
O que é
Primeira fatia do módulo
integration-whatsapp: a camada de ingest (lado Laravel) do coletor de métricas dos grupos de WhatsApp da He4rt. Um webhook autenticado por HMAC recebe os eventos crus emitidos pelo coletor externo (wpp-tui, repo Node/Baileys separado) e os persiste num data lake (3 tabelas), para a equipe de dados explorar depois.Arquitetura
O controller responde rápido (202) e enfileira o processamento pesado no Horizon. A idempotência é garantida em duas camadas: short-circuit por
event_idno controller efirstOrCreate(event_id)no Job (seguro sob corrida/retry).Modelo de dados (data lake)
3 tabelas com UUID PK (UUIDv7 ordenado via
HasUuids) e a colunapayload(jsonb) guardando o evento Baileys cru. Sótype, FKs e timestamps são materializados como colunas (para indexação).whatsapp_groups—external_jidUNIQUE,tenant_id?,display_name,internal_name,payload,first/last_seen_atwhatsapp_participants— global por número (external_jidUNIQUE, 1 linha por pessoa),push_name,identity_id?(→users, para vínculo futuro),payloadwhatsapp_events—event_idUNIQUE (idempotência),type(indexado),group_id?,participant_id?,tenant_id?,occurred_at,received_at,payload. Índices compostos(type, occurred_at),(group_id, occurred_at),(participant_id, occurred_at)Decisões de design
users; UUIDv7 é ordenado por tempo → sem fragmentação de índice no insert da tabela de alto volume.participants.X-Event-Id), via middlewareVerifyWhatsAppSignature.collection_policydescartada nesta fase: o bot envia tudo e o Laravel salva tudo cru — mais aderente ao data lake e remove o subsistema de polling/cache. Revisitar só se uma fase de endurecimento exigir.typecomo string (não enum): o conjunto de eventos do Baileys é aberto e muda entre versões; um enum rejeitaria tipos novos e quebraria o mapeamento.Privacidade — postura consciente (fase exploratória)
Testes
12 testes Pest (37 asserts), todos verdes; Pint limpo.
event_id, não-duplicação de grupoX-Event-Id→ 401; body inválido → 422; duplicata → 202 sem dispatchFora de escopo (PRs futuros)
Recurso Filament para administração, flow de vínculo de identidade, wiring do webhook no
wpp-tui, e operação (deploy, observabilidade, retenção, onboarding/consentimento dos membros).Config
Requer
WHATSAPP_WEBHOOK_SECRET(adicionado ao.env.example) — o mesmo segredo usado pelowpp-tuipara assinar o corpo.Documentação incluída
CONTEXT.md,docs/spec.md,docs/adr/0001-data-lake-approach.mdedocs/plans/0001-ingest-implementation.md.