Repository navigation
feat(security): a Detections page for audit log detections - #2534
Merged
Merged
Conversation
…wn events One warehouse table for audit events from every connected platform, normalized to who did what to which target with what outcome. This first layer adds the table, the read path and the first writer. - audit_trail_events (migration 0064, local schema v56): ReplacingMergeTree keyed on a hash of provider, account and the provider's event id, so a redelivery collapses. A version column makes the first write win, so a redelivered event keeps its first IngestedAt. 400-day TTL, minmax index on IngestedAt. Written through ingest only, pinned to the managed warehouse and hidden from raw SQL like audit_log. - AuditTrailService: write (what every provider adapter calls) and list. - GET /internal/audit-trail/events: session only, behind audit_log:read, keyset paginated on (OccurredAt, Id), filters for provider, account, category, outcome, action, actor and time. A Maple member's email also finds their events. - Maple's own changes and refusals are copied in by the audit events consumer and the direct write path. Allowed reads, tool calls and SQL runs stay in audit_log. A failed copy is logged and dropped so it never sends stored entries to the DLQ.
…edential actions, review tests - The direct write path drops a failed audit_log insert at Warn; its trail copy now does the same instead of leaving an Error span. - Removing a status page password, revoking a break-glass token and unlinking a chat identity are filed like the actions that set them. - An email actor filter is resolved against Maple's members only when Maple's events can be in the result. - Tests: a trail copy that dies still acks, one that is interrupted stays unacked, the directory is read once per request, an empty batch never reaches the warehouse.
…, show the first write - Maple's events are stored under the user id, so an email filter needs the member directory. A directory outage now answers 503 instead of a page that silently leaves the member's Maple events out. A deployment with no directory is unchanged. - Unmerged copies of one event are read oldest write first, so the list shows the IngestedAt the table will keep.
/security/audit-log lists the unified audit trail, newest first, for org admins. - A new Activity section in the Security rail links to it. The page reads the session's grants first: without audit_log:read it says who can read the log and asks the API for nothing. - Events are grouped by day. A row shows when, the platform, who acted, the platform's action name with its category, the target and a mark for a refusal. Opening a row shows everything recorded about the event and its payload. - Category, outcome and actor filters live in the URL. The platform filter appears once a second platform writes events. - Load more continues from the last event shown. A failed page keeps the list and offers a retry. - jsonBlock moves from the settings audit log into the page kit, and both pages use it. - Six lab scenarios under /lab/security?state=audit-log* and a scene test.
…tries under their public id - OutcomeReason holds why an event failed or was refused, in the platform's words, as its own column instead of inside the payload. Maple's denial reason goes there. - A Maple entry's provider event id is its public alog_ id, the one the Settings audit log shows and looks up, so an id copied from one page finds it in the other. The table has no rows anywhere yet, so the column lands in migration 0064 itself.
… and message Both become public tagged errors, like the integrations group's: an outage now reads "The audit log is temporarily unavailable" on the page instead of a generic failure, and a refusal carries a code the page can tell apart from an outage.
- A next page is neither asked for nor accepted while other filters are loading, and an answer has to match the cursor it was asked from. - A session read that fails no longer locks an admin out: the API is asked and its refusal is what shows the admins-only state. - The rows dim while other filters load. - Nothing the row cuts off is lost: full values on hover, and the detail lists the action, actor, actor type and target. The refusal reason shows under the action. - Anything but a success is marked, so an unknown outcome does not read as one. - The outcome filter offers what a platform writes today. A footnote says allowed reads are in the Settings audit log, not here. - Kit: tableGroupRow (the releases list moves onto it), dayEyebrow, refusedBadge and expandChevron (the settings audit log moves onto them), prettyJson, and a controlled value on nativeSelect so a filter change keeps keyboard focus. - The page entry takes the auth mode as an argument so the hosted gate is tested.
Its label replaces its content for a screen reader, and the content was the only place the time was. The label now carries the action and the timestamp.
A screen reader heard "All outcomes, Denied": the select was named after its clear option. They are now Category and Outcome. The scene test's second page also carries a cursor on its own last event instead of the first page's.
Found testing on the preview: a key refused for a missing scope was filed under authentication with an empty target, so filtering refusals by permission found nothing and the row did not say what was asked for. It is now a permission event whose target is the request's method and path.
- An empty answer to the previous filters no longer shows as "no matches" while the new filters load: the list shows loading rows until they answer. - Row times tick: the clock is read every minute. - A Maple event's id links to the same entry in the Settings audit log, and the Settings lookup field now opens an entry when its alog_ id is pasted. - The footnote opens the Data access tab, where the reads it names are. - The lab fixture records a refused request the way the API now does.
… audit log filter The field was rebuilt from the URL on every route change, so typing an actor and then picking a category threw the text away. It now follows the URL only when it holds nothing unapplied, which keeps Back clearing an applied actor.
… window Retesting on the preview showed the Settings lookup change did not work: that page cannot fetch an entry by id alone, so pasting an alog_ id set the URL and opened nothing. The lookup is back to what it was. The link from the Security page now goes through the same builder as the Settings page's own entry links (the minute ending at the entry, changes and reads both shown), so it finds old entries too.
…t to logs An audit event names a person, their email and their IP address. As a log it was readable by every member and writable by anything holding an ingest key. The gateway now writes these to audit_trail_events, behind audit_log:read. Gateway (apps/ingest): - New audit_trail module: Vercel audit drain events and GCP Admin Activity and Policy Denied entries as rows, with the same key derivation as the TypeScript writer. GCP System Event entries are Google's own actions and stay logs. - The Vercel audit drain has its own handler. It refuses the public ingest key, which ships in browser bundles, and acks and drops while the integration's source is paused. - A GCP audit entry is stored ahead of the plan check on logs and is not metered. - A gateway that only forwards has no such table: there both feeds stay logs. Web: - The audit log page gets its platform filter, column and detail row back, with Vercel and Google Cloud named. Target shows on wide screens only, so Outcome stays on screen beside the rail. - /infra/vercel no longer reads audit events from logs. Its rail points at the Security audit log when the drain is on. The query, endpoint and row schema behind the old panel are removed. - The Vercel and GCP setup copy and the public docs say where these events land.
…ded and off by default Review of the Vercel and GCP layer: - The Vercel audit drain accepted any private key or source token. It now takes only the drain Maple creates: the private key naming one of the org's integration sources. - A delivery is decoded inside the shared decode memory budget and capped at 4 MiB; a small gzip could otherwise parse into gigabytes. - Audit rows are unmetered, so each org has a delivery rate, every column a cap, and a timestamp outside the table's retention is replaced by the time of receipt. - Kubernetes API writes stay logs. Categories are matched on whole words, a federated workload is a named service, and Vercel rows carry a target. - INGEST_AUDIT_TRAIL_ENABLED (default off) turns the trail on, so the gateway can deploy before the table exists and is switched off rather than rolled back past the new WAL frame tag. - The handler records request metrics like the other ingest routes. - Tests read the stored row instead of the returned count.
… to retry verify_source_claims left a claim unverified when the store could not be asked, and the audit handler read that as an unknown sender (401). The drain then looked misconfigured during a Postgres blip. The lookup now reports the failure and the handler answers 503, as the key lookup does.
…n and the count query An audit log detection is a managed alert rule with signal type audit: N or more matching events arrived in the last W minutes. This adds the wire schemas and endpoints on the audit trail API, the nullable audit_filters_json column on alert_rules, and auditTrailCountQuery, which counts distinct event ids by IngestedAt and bounds OccurredAt so partitions prune.
…ents A detection is a hidden alert rule with signal type audit, written only through the audit trail API by callers who hold audit_log:read. AuditDetectionsService owns the rule row the way an SLO owns its burn-rate rule: destinations and the name are checked under the org's alert rule lock, an edit ends the tick's claim, and a delete takes the rule's state, deliveries and incidents with it. The evaluator counts the events that arrived in the rule's window through AuditTrailService.count. A count is a fact even at zero, so an empty window is healthy and resolves, and a count that could not be read is an evaluation failure that leaves an open incident open. Notifications carry the count as their detail and never an event.
The rule fields and the row read back; the detections service over PGlite (create, list, update, remove, the cap, an unknown destination, a taken name, another org's ids); the count query through the fake warehouse; a scheduler tick that stays healthy below the count, opens at it, resolves below it and holds an open incident while the count cannot be read; and the routes, where a caller without audit_log:read is refused on every endpoint.
The Security hub lists the org's detections under Activity, with a switch that saves on toggle, edit and a confirmed delete. With none set up the page offers five ready-made ones. The editor is a form page in the shape of the SLO editor, with the same destination picker, and the audit log opens it prefilled from its platform, category and outcome filters. The audit log's label maps and access check move to the hub's kit so both pages share them.
Lab states for the list, the empty on-ramp, the member view, a failed load, and the editor new, from a template, on an existing detection and with a server error, plus a filtered audit log. Scene tests cover the three access states, what each row says, the on-ramp, create and edit against the API's answers, the toggle, the confirmed delete, the invalid error's message and the audit log's prefill link.
|
Warning Review limit reachedYou've used all free OSS reviews for now. Wait for the free limit to reset to keep reviewing this public repository. Next included review available in 42 minutes. View limit details
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. |
This comment has been minimized.
This comment has been minimized.
Makisuo
added
the
preview:web
This comment has been minimized.
This comment has been minimized.
…ot a failed write The consumer drops a failed copy and acks the entry, but wrote it without bestEffort, so the insert closed its spans as Error for a copy that was discarded on purpose. The direct write path already passed it.
This comment has been minimized.
This comment has been minimized.
… filtered empty state A phone drops the Platform column, so another platform's name now leads the line under the action. The empty state for a filter that matched nothing offers the platform filter too. The drains note says where audit events are when the gateway does not store the trail.
This comment has been minimized.
This comment has been minimized.
The label and its actor caveat were separate flex items, so from the width where the caveat shows, the button's gap fell before the comma. They are one text run now.
# Conflicts: # packages/domain/src/http/index.ts
# Conflicts: # apps/web/src/page/alerts/incident/view.ts # apps/web/src/page/kit/session-grants.ts # packages/views/src/ui/INDEX.md
This comment has been minimized.
This comment has been minimized.
…tion An id that is not one showed "Detection not found" under the heading "New detection". The heading now says it is a link to a detection, as it does for one the org no longer has.
This comment has been minimized.
This comment has been minimized.
This comment has been minimized.
This comment has been minimized.
Maple review🟢 Confidence 10/10 · safe to merge. Only the delta since the previous review was new, and it is empty: both Adds the Detections list and editor for audit-log detections, its templates and the audit-log "alert on these events" link. Nothing new to review since the last pass:
Production impactOpen errors in the changed files (2)
After this merges, Maple checks whether they stop.
|
Maple: after this shipped✅ Shipped clean. Nothing this change touches got worse in the hour after it deployed. Deployed to Errors in the changed files:
Maple looks once, an hour after the first deploy that carries the merge. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Fifth layer of the unified audit log, on top of #2532: the Detections page. An org admin can now create the alerts the previous layer made possible.
What it adds
/security/detections). Admins only, with the same three access states as the audit log: asking, allowed, and an admins-only notice.The page states once what the backend guarantees: every member can see a detection's name and that it fired; only org admins see the events it counted.
Screenshots
Audit log with filters applied. The "before" is this build with the control switched off, so its rail already shows the new entry.
Verified
bun run --cwd apps/web typecheck,ui:index:checkandui:patterns:checkpass.oxlint -c .oxlintrc.effect.jsonis clean on the changed files.Tested on this PR's
preview:webdeploy by a separate tester, signed in as an org admin, against the real API:9b9aaf6bef: the list hid its row controls below about 1340 px, the audit log control ran off a phone screen, "A blank detection" did not blank the form, and the server's refusal stayed after the form was corrected.A second pass on the preview at
8d7ab1f7afconfirmed the two layout fixes live: at 1280 x 800 the switch, Edit and Delete sit inside the panel with no sideways scroll, and at 390 wide the audit log control fits with a category and an actor applied.Not testable on a web-only preview (no live sync): picking destinations in the editor, the Alerts hub and the incident page. The destination picker is the one the SLO and uptime editors use. A non-admin's view was not tested either: the preview has one account.
Known rough edges, left as they are: a failed load still shows the "New detection" button; a name that differs only in case from an existing one is accepted; an org with no alert destinations gets no link to where they are made.