Feature/extended telemetry by sharppaul · Pull Request #130 · maris-development/beacon-studio

sharppaul · 2026-09-17T15:19:44Z

d2a2c70 Document the telemetry rules in AGENTS.md
cc10011 Report query shape and feature use at call sites
4f47c63 Carry query shape and client context in events
0b4c454 Add Telemetry settings group and query opt-out

"Send query telemetry" governs the table, column and filter values that a query event carries. The main switch keeps working on its own, so a user can report usage without reporting content.

The studio_telemetry table takes no new columns, so every new field goes in props. The server drops a props object above its cap instead of cutting it, so fitProps degrades the object in steps first.

Eleven of eighteen routes sent only a page view. The node, browser, builder and workbench events fill that gap, and every query event now carries the table, the columns and the filters that ran.

The event name list is closed on both sides, and the table takes no new columns. Both facts cost a silent data loss when missed.

Copilot

🟡 Changes recommended

Several telemetry paths misclassify automatic actions, omit events, or risk losing unload batches.

Get a fresh assessment by requesting another Copilot review.

Pull request overview

This PR extends telemetry with query details, client context, cache timing, user settings, and broader event coverage.

Changes:

Adds query-shape encoding, payload truncation, batching, error capture, and session metrics.
Records browser, builder, node, query, and visualisation events.
Documents telemetry rules and adds query-detail controls.

File summaries

File	Description
`src/routes/visualisations/table-explorer/+page.svelte`	Reports table visualisation telemetry.
`src/routes/data-browser/datasets/detail/+page.svelte`	Tracks dataset opens and searches.
`src/routes/data-browser/datasets/+page.svelte`	Tracks dataset searches.
`src/routes/data-browser/data-tables/detail/+page.svelte`	Tracks table opens and searches.
`src/lib/telemetry/types.ts`	Defines event names and query sources.
`src/lib/telemetry/README.md`	Documents telemetry events and limits.
`src/lib/telemetry/queue.ts`	Adds byte-aware event batching.
`src/lib/telemetry/query-shape.ts`	Encodes query details for telemetry.
`src/lib/telemetry/props.ts`	Limits telemetry property size.
`src/lib/telemetry/index.ts`	Applies settings, context, and session tracking.
`src/lib/telemetry/errors.ts`	Captures global errors and rejections.
`src/lib/telemetry/context.ts`	Collects client context.
`src/lib/telemetry/console.ts`	Deduplicates console events.
`src/lib/stores/stored-query.ts`	Tracks query sharing and opening.
`src/lib/stores/settings.ts`	Adds telemetry settings.
`src/lib/stores/saved-queries.ts`	Tracks saved queries.
`src/lib/stores/query-store.svelte.ts`	Adds query timing and cache telemetry.
`src/lib/services/beacon-node.ts`	Tracks node lifecycle actions.
`src/lib/services/beacon-node-health.ts`	Tracks health changes.
`src/lib/components/visualisation/MapViewController.svelte.ts`	Tracks map visualisation timing.
`src/lib/components/query-builder/QueryWorkspace.svelte.ts`	Tracks workbench blocks.
`src/lib/components/query-builder/QueryBuilderParameterBlock.svelte`	Tracks column additions.
`src/lib/components/query-builder/QueryBuilder.svelte`	Tracks table selections.
`src/lib/components/query-builder/AddFilterDropdown.svelte`	Tracks filter additions.
`src/lib/components/plots/ChartExplorerController.svelte.ts`	Tracks chart visualisation timing.
`src/lib/components/home/QuickStartExamples.svelte`	Tracks example starts.
`AGENTS.md`	Documents telemetry implementation rules.

Review details

Files reviewed: 27/27 changed files
Comments generated: 10
Review effort level: Lite

💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.

A loader, an import and a cache hit each raised an event that means a user action. The events now carry their source, or come from the picker itself. Also: keep the unload batch inside the browser keepalive quota, scan filter groups for the drawn area, and report an empty table result as a view.

Copilot

🟡 Changes recommended

Several telemetry metrics, cache timings, unload batches, and data type values are inaccurate.

Get a fresh assessment by requesting another Copilot review.

Review details

Suppressed comments (6)

Previously missed (6) — in code that hasn't changed since the last review.

src/lib/components/plots/ChartExplorerController.svelte.ts:451

syncPlotToColumns only changes plot state. The page schedules preparation through requestAnimationFrame and setTimeout, so this event runs before chart data is prepared. Report after preparation.
src/lib/components/query-builder/AddFilterDropdown.svelte:171
data_type uses the same object form for timestamp fields. This reports every object as Timestamp instead of its unit and timezone. Use the existing formatter.
src/lib/components/query-builder/QueryBuilderParameterBlock.svelte:331
DataType can be a TimestampDataType object. This fallback labels every timestamp as generic Timestamp, so telemetry loses its unit and timezone. Use the existing formatter.
src/lib/components/visualisation/MapViewController.svelte.ts:404
prepareTable catches grouping and display errors and returns normally. This line then counts a map view even when no layer was built. Return success from preparation and report only on success.
src/lib/stores/query-store.svelte.ts:575
restore also keeps the original query id and duration, although OPFS did not contact the node. Telemetry therefore reports the old network run as this cache read. Record restore time and omit the old query id.
src/lib/telemetry/queue.ts:77
The first event may exceed MAX_BODY_BYTES because track does not cap message. A network failure then requeues it forever and blocks later events. Drop or truncate oversized events before retry.

Files reviewed: 30/30 changed files
Comments generated: 2
Review effort level: Lite

+
+				// The cached entry holds the stats of its own run. This hit read no
+				// node and no disk, so the copy that every caller sees says `memory`.
+				const hit: DatasetEntry = { ...cached, stats: { ...cached.stats, tier: 'memory' } };


 	if (buffer.length === 0 || Date.now() < pausedUntil) return;

-	const events = buffer.splice(0, MAX_EVENTS_PER_BATCH);
+	const events = takeTail(MAX_KEEPALIVE_BYTES);


Copilot

🟡 Changes recommended

Cache durations, back-forward cache sessions, keepalive quotas, and receiver allow-list deployment need correction or confirmation.

Get a fresh assessment by requesting another Copilot review.

Review details

Suppressed comments (2)

src/lib/stores/query-store.svelte.ts:238

The memory hit reuses the original network duration. query.execute.duration_ms therefore misreports cache latency. Measure this ensure call or add separate original-run duration.

				const hit: DatasetEntry = { ...cached, stats: { ...cached.stats, tier: 'memory' } };

src/lib/telemetry/queue.ts:159

Both events call this function. A normal hide can send two 50 KB keepalive bodies and exceed the 65,536-byte aggregate quota. Share one budget.

	const events = takeTail(MAX_KEEPALIVE_BYTES);

	void post(events, true);

Files reviewed: 30/30 changed files
Comments generated: 3
Review effort level: Lite

+				stats: {
+					tier: 'opfs',
+					decodeMs: Math.round(performance.now() - start),
+					bytes: hit.bytes.byteLength
+				}


+	const onExit = () => {
+		if (ended) return;


 /** A user action, or a lifecycle moment of the app. */
 export type ActionName =
 	| 'app.start'
+	| 'app.error'


Copilot

🟡 Changes recommended

A duplicate Svelte handler breaks compilation, and telemetry omits required or accurate execution data.

Get a fresh assessment by requesting another Copilot review.

Review details

Suppressed comments (1)

src/lib/stores/query-store.svelte.ts:353

Cached entries reuse the original network duration from fetch or OPFS metadata. Memory and OPFS query.execute events therefore report stale network time, not cache lookup time. Record elapsed time for each ensure call.

			props: {
				...entry.stats,
				...describeQuery(entry.query)

Files reviewed: 30/30 changed files
Comments generated: 5
Review effort level: Lite

 			props: {
-				cacheHit,
-				columns: entry.query.query_parameters?.length ?? 0,
-				filters: entry.query.filters?.length ?? 0
+				...entry.stats,
+				...describeQuery(entry.query)


+	const kept: Props = {};
+
+	for (const [key, value] of Object.entries(props)) {
+		if (CONTENT_KEYS.includes(key)) continue;


+	/**
+	 * Reports one chart view. The call comes after the plot setup, so `renderMs`
+	 * holds the time that the chart itself took.
+	 */


+	/**
+	 * Reports one map view. The call comes after the render, so `renderMs` holds
+	 * the time that the map itself took: the dedup pass and the geometry build.
+	 */


Jasper-Maris

Fixed merge issue. now approved

Copilot

🔵 Needs a closer look

Several telemetry paths can report incorrect cache or visualisation data, and opt-out can race with an upload.

Review details

Suppressed comments (6)

Previously missed (3) — in code that hasn't changed since the last review.

src/lib/components/visualisation/MapViewController.svelte.ts:404

prepareTable catches map build errors and awaits worker work. A newer run can also replace this.entry before this line. This call can report a failed or stale view with the wrong row count and query id. Return a success result and check token before reporting.
src/lib/stores/settings.ts:316
stopTelemetry() clears buffered events but cannot cancel a post() that already waits for a token or fetches. That request can upload old events after opt-out, so this promise to send nothing is false. Abort or gate in-flight sends when the setting changes.
src/lib/telemetry/types.ts:51
QuerySource has no runtime use. Existing props.source writes carry a saved-record role or example label, so query execution events never carry these source values. Wire the union through query callers or remove this public type.

src/lib/telemetry/queue.ts:159

flushOnHide can run for both visibilitychange and pagehide while the first keepalive request remains pending. Two 50 KB requests can exceed the browser's shared 64 KB keepalive budget. Guard concurrent unload flushes or reserve one shared budget.

	const events = takeTail(MAX_KEEPALIVE_BYTES);

	void post(events, true);

src/lib/stores/query-store.svelte.ts:574

The OPFS entry keeps the original network duration while stats.tier says opfs. query.execute.duration_ms then reports stale network latency for a local restore. Measure restore time for telemetry while preserving the original duration for history.

				stats: {
					tier: 'opfs',
					decodeMs: Math.round(performance.now() - start),
					bytes: hit.bytes.byteLength

src/lib/stores/query-store.svelte.ts:238

Memory hits copy the previous tier's stats and the original fetch duration into this entry. query.execute then reports non-memory work for a local hit. Clear those fields and measure the cache lookup separately.

				const hit: DatasetEntry = { ...cached, stats: { ...cached.stats, tier: 'memory' } };

Files reviewed: 30/30 changed files
Comments generated: 0 new
Review effort level: Lite

sharppaul added 4 commits September 17, 2026 16:38

Add Telemetry settings group and query opt-out

0b4c454

"Send query telemetry" governs the table, column and filter values that a query event carries. The main switch keeps working on its own, so a user can report usage without reporting content.

Carry query shape and client context in events

4f47c63

The studio_telemetry table takes no new columns, so every new field goes in props. The server drops a props object above its cap instead of cutting it, so fitProps degrades the object in steps first.

Report query shape and feature use at call sites

cc10011

Eleven of eighteen routes sent only a page view. The node, browser, builder and workbench events fill that gap, and every query event now carries the table, the columns and the filters that ran.

Document the telemetry rules in AGENTS.md

d2a2c70

The event name list is closed on both sides, and the table takes no new columns. Both facts cost a silent data loss when missed.

sharppaul requested review from Jasper-Maris and a lite review from Copilot September 17, 2026 15:25

Copilot started reviewing on behalf of sharppaul September 17, 2026 15:25 View session

Copilot AI reviewed Sep 17, 2026

View reviewed changes

sharppaul requested a lite review from Copilot September 17, 2026 15:55

Copilot started reviewing on behalf of sharppaul September 17, 2026 15:56 View session

Copilot AI reviewed Sep 17, 2026

View reviewed changes

Jasper-Maris requested a lite review from Copilot September 18, 2026 10:04

Copilot started reviewing on behalf of Jasper-Maris September 18, 2026 10:04 View session

Copilot AI reviewed Sep 18, 2026

View reviewed changes

Merge branch 'main' into feature/extended-telemetry

5ead80b

Jasper-Maris requested a lite review from Copilot September 18, 2026 10:13

Copilot started reviewing on behalf of Jasper-Maris September 18, 2026 10:13 View session

Copilot AI reviewed Sep 18, 2026

View reviewed changes

Refactor onValueChange handler in QueryBuilderTableSelector

d1018f3

Jasper-Maris requested a lite review from Copilot September 18, 2026 10:24

Copilot started reviewing on behalf of Jasper-Maris September 18, 2026 10:24 View session

Jasper-Maris reviewed Sep 18, 2026

View reviewed changes

Jasper-Maris approved these changes Sep 18, 2026

View reviewed changes

Jasper-Maris merged commit 374eeb1 into main Sep 18, 2026
1 check passed

Copilot AI reviewed Sep 18, 2026

View reviewed changes

Navigation Menu

Feature/extended telemetry - #130

Feature/extended telemetry#130
Jasper-Maris merged 7 commits into
mainfrom
feature/extended-telemetry

sharppaul commented Sep 17, 2026

Uh oh!

Copilot AI left a comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Copilot AI left a comment

Uh oh!

Copilot AI left a comment

Uh oh!

Copilot AI left a comment

Uh oh!

Uh oh!

Jasper-Maris left a comment

Uh oh!

Uh oh!

Copilot AI left a comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

Conversation

sharppaul commented Sep 17, 2026

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

🟡 Changes recommended

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

🟡 Changes recommended

Suppressed comments (6)

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

🟡 Changes recommended

Suppressed comments (2)

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

🟡 Changes recommended

Suppressed comments (1)

Uh oh!

Uh oh!

Jasper-Maris left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

🔵 Needs a closer look

Suppressed comments (6)

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants