Skip to content

fix: batch notification queries to eliminate N+1 cascading#1679

Open
ionfwsrijan wants to merge 1 commit into
utksh1:mainfrom
ionfwsrijan:fix/1625-notification-n-plus-1
Open

fix: batch notification queries to eliminate N+1 cascading#1679
ionfwsrijan wants to merge 1 commit into
utksh1:mainfrom
ionfwsrijan:fix/1625-notification-n-plus-1

Conversation

@ionfwsrijan

Copy link
Copy Markdown
Contributor

Description

The notification delivery pipeline performed a cascading N+1 query pattern: for each finding, it fetched all rules, and for each rule it checked delivery history and recorded it — resulting in 1 + N×(1 + 2M) DB queries. With 50 findings and 5 rules, this was 551 DB queries plus 250 outbound HTTP calls in a single synchronous chain.

Changes

  1. backend/secuscan/notification_service.py:process_task_notifications() — Rewritten to batch all findings and all active rules upfront (2 queries). Evaluates (finding, rule) pairs in-memory. Dedup check uses a single batched query for all existing delivery history.
  2. backend/secuscan/notification_service.py:_deliver_rule_batch() — New helper that sends a single webhook per (rule, task) containing an array of findings instead of one webhook per (finding, rule). Reduces HTTP calls from N×M to M.
  3. backend/secuscan/executor.py:execute_task() — Notification dispatch is offloaded to asyncio.create_task() so slow webhook targets don't block task finalization.
  4. process_finding_notifications() and deliver_via_rule() are preserved for backward compatibility.

Impact

  • DB queries reduced from 1 + N×(1 + 2M) to 3 + M (findings + rules + dedup + M deliveries)
  • Webhook HTTP calls reduced from N×M to M (one per rule for all matching findings)
  • Task completion is no longer blocked by slow webhook targets
  • New webhook payload uses event: finding.alert.batch with a findings array

Files Changed

  • backend/secuscan/notification_service.py — Major rewrite of process_task_notifications, new _deliver_rule_batch helper
  • backend/secuscan/executor.py — 1 line changed (async dispatch via create_task)

Closes #1625

@ionfwsrijan ionfwsrijan force-pushed the fix/1625-notification-n-plus-1 branch from 9c0ac17 to b074126 Compare July 5, 2026 20:28
@ionfwsrijan

Copy link
Copy Markdown
Contributor Author

@utksh1 Please review this. The failing check is pre-exisiting

@utksh1 utksh1 left a comment

Copy link
Copy Markdown
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The N+1 query batching is a solid improvement, but backend-unit CI is currently failing. Please run pytest locally, fix the failing tests, and push again.

@utksh1 utksh1 added level:intermediate 35 pts difficulty label for moderate contributor PRs type:performance Performance work category bonus label type:bug Bug fix work category bonus label area:backend Backend API, database, or service work labels Jul 9, 2026
@ionfwsrijan ionfwsrijan force-pushed the fix/1625-notification-n-plus-1 branch from b074126 to 6821f78 Compare July 10, 2026 02:45
@ionfwsrijan

Copy link
Copy Markdown
Contributor Author

@utksh1 Please review this now

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

area:backend Backend API, database, or service work level:intermediate 35 pts difficulty label for moderate contributor PRs type:bug Bug fix work category bonus label type:performance Performance work category bonus label

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[BUG] Notification Processing Causes Cascading N+1 Database Queries and Webhook Bursts

2 participants