Skip to content

Commit e8ec022

Browse files
icecrasher321claude
andcommitted
feat(usage): add enterprise organization usage monitoring
Enterprise org admins had no way to see where their pooled credits go. The billing page hides the usage-limit field and the credit-usage drill-down for enterprise, and the credit-usage view is personal-scope only — so the one audience that negotiates a pooled commitment was the one audience that could not watch it burn. Everything needed already existed in `usage_log`, which is the single source of cost truth and is stamped `billing_entity_type='organization'` at charge time. This adds the read surface over it, plus one narrow write so BYOK usage is captured rather than discarded. Settings → Organization → Usage tracking, gated on hosted + enterprise with a `USAGE_MONITORING_ENABLED` self-hosted override, matching the other enterprise features. Five tabs, each answering one question: Overview (how much, and what kind of work), Members, Workspaces (drill into one for its Sources and Workflows), Models, BYOK. Only the visible tab's dimension is fetched — half the dimensions heap-scan the ledger, so a tab nobody opens never pays for one. Every ranked list closes with an explicit `Other (N more)` row so it reconciles to the headline figure. Track unbilled (BYOK) model usage --------------------------------- BYOK spans already reached `costSummary` with real token counts and were discarded at a single `if (modelData.total > 0)` gate. A new `model_unbilled` usage_log category records them at zero cost, written only at the terminal execution boundary and only when billing attribution is already resolved, so a BYOK-only run with no billable target still bails safely instead of hitting the attribution requirement. Every billing read over `usage_log` is `SUM(cost)`, so zero-cost rows change no total anywhere. Collapse organization settings onto the workspace plane ------------------------------------------------------ `/organization/[id]/settings/*` was a second plane with no UI entry point anywhere, whose nine sections all already render on the workspace plane. It is deleted, along with its renderer and unavailable page. `planes.organization` is replaced by a single `unified.organizationSection` marker that now derives both `ORGANIZATION_PLANE_UNIFIED_SECTIONS` and the section map in the workspace gate, removing a hand-maintained duplicate. Usage-threshold emails now link to the workspace-scoped billing page, and the two org-provisioning paths that could strand an admin with zero workspaces now backfill one. Shared chart module ------------------- `LineChart` moves to `components/charts/` with a matching `BarChart` sibling built from the same geometry, tooltip, and theme modules. The move also breaks the import edge from the chart to `logs/utils` → `@/blocks/registry`, which would otherwise have pulled the entire block registry into the settings chunk. Logs dashboard renders identically. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
1 parent 2795922 commit e8ec022

94 files changed

Lines changed: 25027 additions & 707 deletions

File tree

Some content is hidden

Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.

apps/docs/content/docs/en/platform/enterprise/index.mdx

Lines changed: 6 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -53,6 +53,12 @@ Track configuration and security-relevant actions across your organization for c
5353

5454
---
5555

56+
## Usage Tracking
57+
58+
See where your organization's credits go — by member, workspace, model, and platform feature. See the [usage tracking guide](/platform/enterprise/usage-tracking).
59+
60+
---
61+
5662
## Data Retention
5763

5864
Configure how long execution logs, soft-deleted resources, and Chat data are kept before permanent deletion. See the [data retention guide](/platform/enterprise/data-retention).

apps/docs/content/docs/en/platform/enterprise/meta.json

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -10,6 +10,7 @@
1010
"custom-blocks",
1111
"whitelabeling",
1212
"audit-logs",
13+
"usage-tracking",
1314
"data-retention",
1415
"data-drains",
1516
"forks"
Lines changed: 161 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,161 @@
1+
---
2+
title: Usage Tracking
3+
description: See where your organization's credits go, by member, workspace, and model
4+
---
5+
6+
import { Callout } from 'fumadocs-ui/components/callout'
7+
import { FAQ } from '@/components/ui/faq'
8+
import { Image } from '@/components/ui/image'
9+
10+
Usage tracking shows how your organization consumes credits across every part of the platform — which members, which workspaces, which models, and which product features. Use it to monitor spend against your commitment, find what is driving it, and export the underlying events for chargeback.
11+
12+
All figures are in **credits** (1 credit = $0.005). See [cost calculation](/platform/costs) for how credits are derived.
13+
14+
---
15+
16+
## Viewing usage
17+
18+
Go to **Settings → Organization → Usage tracking** in your workspace.
19+
20+
<Image src="/static/enterprise/usage-tracking-overview.png" alt="Usage tracking Overview tab showing the period selector, credits used against the organization limit, a daily usage chart, and a Sources breakdown listing Sim Chat, Workflow, and Agent block" width={900} height={578} />
21+
22+
The period selector applies to every tab:
23+
24+
| Period | What it covers |
25+
|--------|----------------|
26+
| **Current period** | Your organization's current billing period |
27+
| **Previous period** | The period immediately before it |
28+
| **Last 7 days** / **Last 30 days** | A rolling window ending now |
29+
| **Custom range** | Any range up to 92 days |
30+
31+
<Callout type="info">
32+
Daily bars are drawn in your browser's timezone, while a billing period begins at a fixed UTC instant. The first and last bar of a period can therefore be partial, and two admins in different timezones will see the same total split across slightly different days.
33+
</Callout>
34+
35+
### Tabs
36+
37+
| Tab | Answers |
38+
|-----|---------|
39+
| **Overview** | How much have we used, against what limit, and what kind of work was it |
40+
| **Members** | Which people are driving usage |
41+
| **Workspaces** | Which workspaces are driving usage — select one to drill in |
42+
| **Models** | Which models we are paying for |
43+
| **BYOK** | What ran on our own provider keys |
44+
45+
Selecting a workspace opens its detail view, which splits that workspace's usage into **Sources** (what kind of work) and **Workflows** (the individual workflow runs). **Open logs** jumps to that workspace's execution logs.
46+
47+
<Image src="/static/enterprise/usage-tracking-workspace-detail.png" alt="A workspace's detail view with a Sources section listing Sim Chat and Workflow, and a Workflows section ranking individual workflows by credits" width={900} height={598} />
48+
49+
The two sections answer different questions, and the difference is the point: **Sources** adds up to the workspace's total, while **Workflows** covers only the workflow-run part of it. In the example above, Sources totals 4,435 credits but the workflows list only accounts for the 161 credits under Workflow — the other 4,274 came from Chat, which no workflow produced.
50+
51+
---
52+
53+
## What each source means
54+
55+
A **source** is the part of the platform that consumed the credits. Every charge belongs to exactly one source, so the Sources breakdown always adds up to your total.
56+
57+
| Source | Platform features that bill to it |
58+
|--------|-----------------------------------|
59+
| **Workflow** | Every workflow run — model calls made on Sim's hosted keys, hosted-key tool calls (web search, scraping, and similar), and the per-run base charge |
60+
| **Sim Chat** | The Chat panel, agent calls made through the API, and the email Inbox |
61+
| **Agent block** | The Agent block running inside a workflow |
62+
| **Knowledge Base** | Embedding documents on upload or connector sync, and semantic search queries |
63+
| **Wand** | Inline AI generation in editors |
64+
| **Enrichment** | Table column enrichment |
65+
| **Voice input** | Voice sessions |
66+
| **Voice output** | Spoken responses in deployed chats |
67+
| **Sim Chat (MCP)** | Retired. Appears only for historical periods |
68+
69+
### Nuances worth knowing
70+
71+
These follow from how charges are recorded, and they explain most questions about why a number looks the way it does.
72+
73+
**Workflow covers the whole run, not just the base charge.** A workflow row includes three things: the models it called on Sim's hosted keys, any hosted-key tool calls, and the per-run base charge. Model usage typically dominates, but the per-run charge is not negligible — across a high-volume organization it commonly accounts for around a fifth of workflow spend, because it applies to every run whether or not the workflow calls a model.
74+
75+
**Agent block is billed separately from the workflow that contains it.** An Agent block runs inside a workflow but is metered on its own, so its credits appear under **Agent block** rather than **Workflow**. Nothing is counted twice — but it also means Workflow alone understates what your workflows cost. Read the two lines together.
76+
77+
**Sim Chat covers three surfaces.** The Chat panel, headless agent calls through the API, and the Inbox all record as **Sim Chat** and cannot be separated from one another. If you need per-surface attribution today, separate them by workspace instead.
78+
79+
**Only workflow runs carry workflow attribution.** Chat, Agent block, Wand, Knowledge Base, and voice usage are not produced by a workflow and have no workflow attached. That is the gap between a workspace's **Sources** and **Workflows** sections shown above.
80+
81+
**Some usage has no workspace.** Agent calls made through the API may not name a workspace, particularly from headless or self-hosted callers. That usage is grouped under **No workspace** on the Workspaces tab. It is still counted in your total.
82+
83+
**Models excludes non-model charges.** The Models tab covers model usage only, so it will read lower than your period total — the difference is per-run base charges, hosted-key tool charges, and fixed charges such as voice sessions.
84+
85+
<Callout type="warning">
86+
Each tab is a different slice of the same charges, not an additional set of them. Members, Workspaces, and Sources each add up to your period total on their own; adding two tabs together double-counts.
87+
</Callout>
88+
89+
---
90+
91+
## Bring your own keys (BYOK)
92+
93+
When a workspace or organization supplies its own provider key, Sim does not charge for that model usage. Those calls are still recorded so you can see the volume.
94+
95+
The **BYOK** tab groups this usage by provider and reports **tokens** rather than credits, because the credit cost is zero by definition. Tokens on your own keys are not included in the credit totals anywhere else in the panel.
96+
97+
<Callout type="info">
98+
Tool calls and Chat usage made on your own keys are not yet recorded. BYOK currently covers model usage in workflow runs.
99+
</Callout>
100+
101+
---
102+
103+
## Exporting
104+
105+
**Export** downloads the events behind the current period and filters as a CSV with columns `Date, Source, Description, Workflow, Credits`.
106+
107+
**All events** opens the full ledger — every credit-consuming event, newest first, with its own filters and export.
108+
109+
Very large exports are capped. When that happens the download still succeeds and Sim tells you it was truncated; narrow the date range to capture everything.
110+
111+
---
112+
113+
<FAQ items={[
114+
{
115+
question: "Who can view usage tracking?",
116+
answer: "Organization owners and admins. On Sim Cloud you must be on the Enterprise plan. Regular members cannot see organization usage, including their own."
117+
},
118+
{
119+
question: "Why doesn't the Models tab add up to my total?",
120+
answer: "The Models tab covers model usage only. Your total also includes per-run base charges, hosted-key tool charges, and fixed charges such as voice sessions. Use the Sources breakdown on the Overview tab for a figure that reconciles to the total."
121+
},
122+
{
123+
question: "Why is Sim Chat so much larger than Workflow?",
124+
answer: "Sim Chat covers the Chat panel, API agent calls, and the Inbox together, and conversational usage is typically token-heavy. It is common for Chat to be the largest line in an organization that uses it."
125+
},
126+
{
127+
question: "Can I see how much the Inbox or the API specifically costs?",
128+
answer: "Not directly — both record as Sim Chat. Where those surfaces run in dedicated workspaces, the Workspaces tab gives you the equivalent breakdown."
129+
},
130+
{
131+
question: "What is “No workspace”?",
132+
answer: "Usage that was not attributed to a workspace, most often agent calls made through the API without a workspace context. It is included in your organization total."
133+
},
134+
{
135+
question: "Is there an API for usage data?",
136+
answer: "Not currently. Usage tracking is a settings surface; use the CSV export to move the data into your own systems."
137+
},
138+
{
139+
question: "Why does a period show partial days at each end?",
140+
answer: "Daily buckets follow your browser's timezone while billing periods begin at a fixed UTC instant, so the first and last bucket of a period can cover part of a day."
141+
},
142+
{
143+
question: "Does this match my invoice?",
144+
answer: "The Overview total for the current period is the same figure your billing page reports. Rolling windows such as Last 7 days do not align with a billing period and will not match an invoice."
145+
}
146+
]} />
147+
148+
---
149+
150+
## Self-hosted setup
151+
152+
Self-hosted deployments use environment variables instead of the billing/plan check.
153+
154+
### Environment variables
155+
156+
```bash
157+
USAGE_MONITORING_ENABLED=true
158+
NEXT_PUBLIC_USAGE_MONITORING_ENABLED=true
159+
```
160+
161+
Once enabled, usage tracking is viewable in **Settings → Organization → Usage tracking** for organization owners and admins.
123 KB
Loading
122 KB
Loading

apps/sim/app/api/emails/preview/route.ts

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -160,7 +160,7 @@ const emailTemplates = {
160160
scope: 'organization',
161161
currentUsage: 500,
162162
limit: 500,
163-
ctaLink: 'https://sim.ai/organization/org_123/settings/billing',
163+
ctaLink: 'https://sim.ai/workspace/ws_123/settings/billing',
164164
}),
165165

166166
// Operational notification emails
Lines changed: 38 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,38 @@
1+
import { getOrganizationUsageBreakdownContract } from '@/lib/api/contracts/organization-usage'
2+
import {
3+
defineInternalJsonRoute,
4+
internalOrchestrationErrorPolicy,
5+
internalRateLimits,
6+
internalSessionAuth,
7+
} from '@/lib/api/server/routes'
8+
import { getOrganizationUsageBreakdown } from '@/lib/billing/application/organization-usage/get-organization-usage-breakdown'
9+
import { organizationUsageOperations } from '@/lib/billing/application/organization-usage/operations'
10+
11+
export const dynamic = 'force-dynamic'
12+
13+
/**
14+
* One route for all five dimensions: they share a scope, a window, a row shape, and
15+
* authorization, so five routes would be five copies of the same mapping. Separate
16+
* from the summary because three of the five heap-scan the ledger.
17+
*/
18+
export const GET = defineInternalJsonRoute({
19+
contract: getOrganizationUsageBreakdownContract,
20+
auth: internalSessionAuth,
21+
operation: organizationUsageOperations.readBreakdown,
22+
rateLimit: internalRateLimits.none({
23+
reason:
24+
'Authenticated org-admin settings read, gated on enterprise entitlement and billing authority',
25+
}),
26+
errorPolicy: internalOrchestrationErrorPolicy,
27+
mapInput: ({ params, query }) => ({
28+
organizationId: params.id,
29+
dimension: query.dimension,
30+
workspaceId: query.workspaceId,
31+
preset: query.preset,
32+
startDate: query.startDate ? new Date(query.startDate) : undefined,
33+
endDate: query.endDate ? new Date(query.endDate) : undefined,
34+
limit: query.limit,
35+
}),
36+
useCase: getOrganizationUsageBreakdown,
37+
present: (result) => result,
38+
})
Lines changed: 39 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,39 @@
1+
import { listOrganizationUsageEventsContract } from '@/lib/api/contracts/organization-usage'
2+
import {
3+
defineInternalJsonRoute,
4+
internalOrchestrationErrorPolicy,
5+
internalRateLimits,
6+
internalSessionAuth,
7+
} from '@/lib/api/server/routes'
8+
import { listOrganizationUsageEvents } from '@/lib/billing/application/organization-usage/list-organization-usage-events'
9+
import { organizationUsageOperations } from '@/lib/billing/application/organization-usage/operations'
10+
import type { InternalUsageLogSource } from '@/lib/billing/usage-sources'
11+
12+
export const dynamic = 'force-dynamic'
13+
14+
/**
15+
* The raw ledger, paged. Separate from the summary because it owns a cursor
16+
* lifecycle and its own staleness — folding it in would re-run the headline
17+
* aggregate on every scroll.
18+
*/
19+
export const GET = defineInternalJsonRoute({
20+
contract: listOrganizationUsageEventsContract,
21+
auth: internalSessionAuth,
22+
operation: organizationUsageOperations.listEvents,
23+
rateLimit: internalRateLimits.none({
24+
reason:
25+
'Authenticated org-admin settings read, gated on enterprise entitlement and billing authority',
26+
}),
27+
errorPolicy: internalOrchestrationErrorPolicy,
28+
mapInput: ({ params, query }) => ({
29+
organizationId: params.id,
30+
preset: query.preset,
31+
startDate: query.startDate ? new Date(query.startDate) : undefined,
32+
endDate: query.endDate ? new Date(query.endDate) : undefined,
33+
source: query.source as InternalUsageLogSource[] | undefined,
34+
limit: query.limit,
35+
cursor: query.cursor,
36+
}),
37+
useCase: listOrganizationUsageEvents,
38+
present: (result) => result,
39+
})
Lines changed: 81 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,81 @@
1+
import { createLogger } from '@sim/logger'
2+
import { getErrorMessage } from '@sim/utils/errors'
3+
import { type NextRequest, NextResponse } from 'next/server'
4+
import { exportOrganizationUsageContract } from '@/lib/api/contracts/organization-usage'
5+
import { getValidationErrorMessage, parseRequest } from '@/lib/api/server'
6+
import { getSession } from '@/lib/auth'
7+
import {
8+
exportOrganizationUsageEvents,
9+
type OrganizationUsageExportRow,
10+
} from '@/lib/billing/application/organization-usage/export-organization-usage-events'
11+
import { formatCreditsLabel } from '@/lib/billing/credits/conversion'
12+
import type { InternalUsageLogSource } from '@/lib/billing/usage-sources'
13+
import { ForbiddenOperationError } from '@/lib/core/application'
14+
import { formatCsvValue, toCsvRow } from '@/lib/core/utils/csv'
15+
import { withRouteHandler } from '@/lib/core/utils/with-route-handler'
16+
17+
const logger = createLogger('OrganizationUsageExportAPI')
18+
19+
const CSV_HEADER = toCsvRow(['Date', 'Source', 'Description', 'Workflow', 'Credits'])
20+
21+
/** `formatCsvValue` neutralizes formula injection — model and workflow names are user-controlled. */
22+
function toCsvLine(row: OrganizationUsageExportRow): string {
23+
return toCsvRow([
24+
formatCsvValue(row.createdAt),
25+
formatCsvValue(row.source),
26+
formatCsvValue(row.description),
27+
formatCsvValue(row.workflowName ?? ''),
28+
formatCsvValue(formatCreditsLabel(row.credits)),
29+
])
30+
}
31+
32+
/**
33+
* A raw handler rather than a JSON builder: the body is `text/csv`, and the response
34+
* carries `X-Export-Truncated` so the client can tell the user their range was capped
35+
* rather than silently handing them a partial file.
36+
*/
37+
export const GET = withRouteHandler(async (request: NextRequest, context) => {
38+
try {
39+
const session = await getSession()
40+
const sessionId = session?.session?.id
41+
if (!session?.user?.id || !sessionId) {
42+
return NextResponse.json({ error: 'Unauthorized' }, { status: 401 })
43+
}
44+
45+
const parsed = await parseRequest(exportOrganizationUsageContract, request, context, {
46+
validationErrorResponse: (error) =>
47+
NextResponse.json(
48+
{ error: getValidationErrorMessage(error, 'Invalid query parameters') },
49+
{ status: 400 }
50+
),
51+
})
52+
if (!parsed.success) return parsed.response
53+
54+
const { query, params } = parsed.data
55+
const result = await exportOrganizationUsageEvents.execute({
56+
principal: { kind: 'session', userId: session.user.id, sessionId },
57+
input: {
58+
organizationId: params.id,
59+
preset: query.preset,
60+
startDate: query.startDate ? new Date(query.startDate) : undefined,
61+
endDate: query.endDate ? new Date(query.endDate) : undefined,
62+
source: query.source as InternalUsageLogSource[] | undefined,
63+
},
64+
})
65+
66+
const csv = [CSV_HEADER, ...result.rows.map(toCsvLine)].join('\n')
67+
return new NextResponse(csv, {
68+
headers: {
69+
'Content-Type': 'text/csv; charset=utf-8',
70+
'Content-Disposition': `attachment; filename="organization-usage-${params.id}.csv"`,
71+
...(result.truncated ? { 'X-Export-Truncated': '1' } : {}),
72+
},
73+
})
74+
} catch (error) {
75+
if (error instanceof ForbiddenOperationError) {
76+
return NextResponse.json({ error: error.message }, { status: 403 })
77+
}
78+
logger.error('Failed to export organization usage', { error: getErrorMessage(error) })
79+
return NextResponse.json({ error: 'Failed to export usage' }, { status: 500 })
80+
}
81+
})
Lines changed: 36 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,36 @@
1+
import { getOrganizationUsageSummaryContract } from '@/lib/api/contracts/organization-usage'
2+
import {
3+
defineInternalJsonRoute,
4+
internalOrchestrationErrorPolicy,
5+
internalRateLimits,
6+
internalSessionAuth,
7+
} from '@/lib/api/server/routes'
8+
import { getOrganizationUsageSummary } from '@/lib/billing/application/organization-usage/get-organization-usage-summary'
9+
import { organizationUsageOperations } from '@/lib/billing/application/organization-usage/operations'
10+
11+
export const dynamic = 'force-dynamic'
12+
13+
/**
14+
* Everything above the fold in one round trip. Kept separate from the breakdown
15+
* route because every read here is index-covered, and folding in a dimension that
16+
* heap-scans would put that cost on first paint.
17+
*/
18+
export const GET = defineInternalJsonRoute({
19+
contract: getOrganizationUsageSummaryContract,
20+
auth: internalSessionAuth,
21+
operation: organizationUsageOperations.readSummary,
22+
rateLimit: internalRateLimits.none({
23+
reason:
24+
'Authenticated org-admin settings read, gated on enterprise entitlement and billing authority',
25+
}),
26+
errorPolicy: internalOrchestrationErrorPolicy,
27+
mapInput: ({ params, query }) => ({
28+
organizationId: params.id,
29+
preset: query.preset,
30+
startDate: query.startDate ? new Date(query.startDate) : undefined,
31+
endDate: query.endDate ? new Date(query.endDate) : undefined,
32+
timezone: query.timezone,
33+
}),
34+
useCase: getOrganizationUsageSummary,
35+
present: (result) => result,
36+
})

0 commit comments

Comments
 (0)