Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
52 changes: 52 additions & 0 deletions scripts/proposals/catalog-retained-selection/README.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,52 @@
# Retained-field selection and inner consumer — inactive source proposal

This implements the existing `catalog-field-consumer-2207.md` inner contract only.
It is outside app/lib and routes. Missing authority ports return unavailable.
No SQL wrapper, grant, source write, Context identity creation, environment flag,
UI schema-v1 extension or outer wire version is supplied. Frozen raw-report
API1290/Database142 are a different source path and are unchanged.

`readRetainedFieldSelection` calls an injected current command reconciler with the
original authenticated actor, workspace, request, catalog and retention key. The
intended backing function is the private, source-only Database proposal
[`stream_binding_proposal.reconcile_retention`](https://github.com/recoupable/database/blob/1b892dd2671ba7e15e0c44b690fbfbe515b59fbd/proposals/stream-retention-writer-2207/writer.sql#L3),
which rechecks `read_binding`. It is not implemented or connected as a live API
transport here; the injected callback alone supplies no authentication proof.
The adapter validates the exact receipt/scope/key and projects its
genuine Context identities. Null, pending, malformed, mismatched or ambiguous
results cannot create a ready selection; operational errors are redacted errors.
No retain/expand/provider call, latest lookup, re-key or raw-report fallback exists.
This projection supplies a selection, not a currentness lease.

`consumeRetainedField` accepts only the explicit retained_field request and server
actor/session/chat/assistant context. The injected transaction port must authorize
that complete scope, pre-lock the whole batch in a qualified stable order, read
each exact selection using the existing source family, and keep locks through
callback evaluation. No production port is implemented here. Passing mocked data
to that trusted port does not prove source authorization. The actor is never
substituted with owner/workspace. Maximum eight unique subjects; all-or-nothing
refusal, exact registered method/query, current_streams only, 32KiB output cap.
It calls the existing pinned evaluator, then strips raw observations and identifiers
outside the inner projection contract. Unknown has no amount key; zero stays zero.
Caller order is preserved; the port receives a separate subject-sorted lock list.

The generated `fixtures/fieldContext.json` is the ready inner response without
`state`, as specified by the consumer contract. It is synthetic evidence for UI's
explicitly versioned offline envelope; no outer version is invented here. UI owns
projection/collection/text agreement and capability negotiation. This increment
returns typed values but does not implement a renderer or any disclosure path.

Pure tests run the actual selection adapter and evaluator with synthetic authority
ports, including withheld values on stale/missing/withdrawn boundary simulations.
The Database companion extends the actual existing retained-field integration
harness with a deferred current reconcile/read consumer. Those PostgreSQL scenarios
are not run under the SQL hold. Simulated refusal labels do not prove actual
withdrawal, memberships, transaction liveness or concurrent batch correctness.

Remaining runtime gates: real selection-source transport, actor/session/chat scope,
reserved-family writer enforcement, callable held-transaction evaluator transport,
qualified batch lock order, UI-owned envelope, and current authorization on initial
delivery/replay/persistence/history/reconnect/text/export/model reads. A successful
journaled preparation cannot authorize later disclosure; deny both typed values and
numeric/text projections together. No recomputation or new version under the same
saved source revision. The ordinary report stays separate and is never a fallback.
Original file line number Diff line number Diff line change
@@ -0,0 +1,8 @@
/** Preserve contract failure classes while exposing no source details or values. */
export function classifyRetainedFieldRefusal(reason: string): {
state: "invalid" | "error" | "unavailable";
} {
if (reason === "size_limit") return { state: "invalid" };
if (reason === "helper_changed" || reason === "source_error") return { state: "error" };
return { state: "unavailable" };
}
Original file line number Diff line number Diff line change
@@ -0,0 +1,93 @@
import { z } from "zod";
Comment thread
sidneyswift marked this conversation as resolved.
Resolved
import { evaluateRetainedStreamField } from "../catalog-stream-fields/evaluate";
import { actorScopeSchema, requestSchema } from "./schemas";
import type { Item, Ready, Result, LockedRead } from "./retainedFieldTypes";
import { classifyRetainedFieldRefusal } from "./classifyRetainedFieldRefusal";
/** Inactive source consumer; no report fallback, disclosure route, transport or UI schema change. */
export async function consumeRetainedField(
actorInput: unknown,
input: unknown,
withLockedRead?: LockedRead,
): Promise<Result> {
const actor = actorScopeSchema.safeParse(actorInput);
const request = requestSchema.safeParse(input);
if (!actor.success || !request.success) return { state: "invalid" };
if (!withLockedRead) return { state: "unavailable" };
const q = request.data;
try {
return await withLockedRead(
{
...actor.data,
workspaceId: q.workspaceId,
catalogId: q.catalogId,
selections: [...q.selections].sort((a, b) => a.subjectId.localeCompare(b.subjectId)),
},
async tx => {
if (
!z.iso.datetime({ offset: true }).safeParse(tx.asOf).success ||
!Number.isFinite(Date.parse(tx.asOf))
)
return { state: "error" };
const items: Item[] = [];
for (const selection of q.selections) {
const envelope = await tx.readBinding(selection);
if (envelope === null || envelope === undefined) return { state: "unavailable" };
const result = evaluateRetainedStreamField({
operation: "read_field",
recipe: q.method.recipe,
field: q.field,
selection: { ...selection, query: q.query },
asOf: tx.asOf,
boundary: { state: "qualified", envelope },
});
if (result.state !== "ready") return classifyRetainedFieldRefusal(result.reason);
const comparisonState = result.coverage.comparisonState;
if (comparisonState !== "comparable" && comparisonState !== "incomplete")
return { state: "error" };
const { authority, kind } = result.ref;
if (authority !== "context" || kind !== "recording") return { state: "error" };
const amount = result.field.amount;
if (amount !== null && (!Number.isSafeInteger(amount) || amount < 0))
return { state: "unavailable" };
items.push({
ref: { ...result.ref, authority, kind },
binding: selection,
value: amount === null ? { state: "unknown" } : { state: "known", amount },
periods: result.periods,
coverage: {
comparisonState,
absentDateCount: result.coverage.absentDates.length,
nullDateCount: result.coverage.nullDates.length,
missingDateCount: result.coverage.missingDays.length,
},
source: {
provider: result.provenance.provider,
metric: result.provenance.metric,
platform: result.provenance.platform,
territory: result.provenance.territory,
timezone: result.provenance.timezone,
retrievalRange: result.provenance.retrievalRange,
},
});
}
const ready: Ready = {
state: "ready",
contract: "catalog-retained-field-projection/1",
workspaceId: q.workspaceId,
catalogId: q.catalogId,
field: q.field,
method: q.method,
query: q.query,
asOf: tx.asOf,
semantics: "historical_snapshot",
items,
};
return Buffer.byteLength(JSON.stringify(ready), "utf8") > 32768
? { state: "invalid" }
: ready;
},
);
} catch {
return { state: "error" };
}
}
Original file line number Diff line number Diff line change
@@ -0,0 +1,77 @@
{
"contract": "catalog-retained-field-projection/1",
"workspaceId": "10000000-0000-4000-8000-000000000001",
"catalogId": "10000000-0000-4000-8000-000000000003",
"field": "current_streams",
"method": {
"recipe": "catalog-retained-stream-fields/1",
"recipeHash": "65a06a785aa496e1232fe3a9f1606595f19c1a152b710ca30c4c06612f01864a",
"evaluatorSha256": "163563e15dd92d473aa51899c54c96300f20d27072b83e39ffb01c84f3adc64e",
"helperSha256": "91dcca66c8dec0851c1f6e110257a63b730e9b7ae33a60bceb0e9ffcf318dd81"
},
"query": {
"since": "2026-10-24",
"days": 1,
"timezone": "UTC"
},
"asOf": "2026-10-27T00:00:00Z",
"semantics": "historical_snapshot",
"items": [
{
"ref": {
"workspaceId": "10000000-0000-4000-8000-000000000001",
"authority": "context",
"kind": "recording",
"id": "10000000-0000-4000-8000-000000000004",
"sourceVersionIds": [
"10000000-0000-4000-8000-000000000006"
]
},
"binding": {
"ownerId": "10000000-0000-4000-8000-000000000001",
"requestId": "10000000-0000-4000-8000-000000000002",
"subjectId": "10000000-0000-4000-8000-000000000004",
"catalogId": "10000000-0000-4000-8000-000000000003",
"resultId": "10000000-0000-4000-8000-000000000005",
"revision": 7,
"topic": "catalog_retained_streams",
"sourceVersionIds": [
"10000000-0000-4000-8000-000000000006"
]
},
"value": {
"state": "known",
"amount": 100
},
"periods": {
"previous": {
"start": "2026-10-23",
"end_exclusive": "2026-10-24"
},
"current": {
"start": "2026-10-24",
"end_exclusive": "2026-10-25"
},
"days": 1,
"timezone": "UTC"
},
"coverage": {
"comparisonState": "comparable",
"absentDateCount": 0,
"nullDateCount": 0,
"missingDateCount": 0
},
"source": {
"provider": "luminate",
"metric": "daily_streams",
"platform": "all_dsps",
"territory": "worldwide",
"timezone": "UTC",
"retrievalRange": {
"earliest": "2026-10-25T01:00:00Z",
"latest": "2026-10-25T01:00:00Z"
}
}
}
]
}
Original file line number Diff line number Diff line change
@@ -0,0 +1,99 @@
{
"scope": {
"actorId": "10000000-0000-4000-8000-000000000009",
"workspaceId": "10000000-0000-4000-8000-000000000001",
"requestId": "10000000-0000-4000-8000-000000000002",
"catalogId": "10000000-0000-4000-8000-000000000003",
"key": "original"
},
"actor": {
"actorId": "10000000-0000-4000-8000-000000000009",
"sessionId": "session",
"chatId": "chat",
"messageId": "assistant"
},
"receipt": {
"ownerId": "10000000-0000-4000-8000-000000000001",
"requestId": "10000000-0000-4000-8000-000000000002",
"subjectId": "10000000-0000-4000-8000-000000000004",
"catalogId": "10000000-0000-4000-8000-000000000003",
"key": "original",
"commandDigest": "aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa",
"resultId": "10000000-0000-4000-8000-000000000005",
"sourceVersionId": "10000000-0000-4000-8000-000000000006",
"revision": 7
},
"envelope": {
"ownerId": "10000000-0000-4000-8000-000000000001",
"requestId": "10000000-0000-4000-8000-000000000002",
"subjectId": "10000000-0000-4000-8000-000000000004",
"catalogId": "10000000-0000-4000-8000-000000000003",
"resultId": "10000000-0000-4000-8000-000000000005",
"revision": 7,
"topic": "catalog_retained_streams",
"sourceVersionIds": [
"10000000-0000-4000-8000-000000000006"
],
"evidence": {
"schemaVersion": 1,
"workspaceId": "10000000-0000-4000-8000-000000000001",
"catalogId": "10000000-0000-4000-8000-000000000003",
"subjectId": "10000000-0000-4000-8000-000000000004",
"isrc": "USAAA2600001",
"provider": "luminate",
"metric": "daily_streams",
"platform": "all_dsps",
"territory": "worldwide",
"timezone": "UTC",
"since": "2026-10-23",
"until": "2026-10-25",
"periods": {
"previous": {
"start": "2026-10-23",
"end_exclusive": "2026-10-24"
},
"current": {
"start": "2026-10-24",
"end_exclusive": "2026-10-25"
},
"days": 1,
"timezone": "UTC"
},
"providerRecordingId": "provider-original",
"observations": [
{
"catalog_id": "10000000-0000-4000-8000-000000000003",
"isrc": "USAAA2600001",
"date": "2026-10-23",
"run_id": "10000000-0000-4000-8000-000000000007",
"provider_recording_id": "provider-original",
"streams": 50,
"source_hash": "aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa",
"retrieved_at": "2026-10-25T01:00:00Z"
},
{
"catalog_id": "10000000-0000-4000-8000-000000000003",
"isrc": "USAAA2600001",
"date": "2026-10-24",
"run_id": "10000000-0000-4000-8000-000000000007",
"provider_recording_id": "provider-original",
"streams": 100,
"source_hash": "aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa",
"retrieved_at": "2026-10-25T01:00:00Z"
}
],
"absentDates": [],
"nullDates": [],
"retrievalRange": {
"earliest": "2026-10-25T01:00:00Z",
"latest": "2026-10-25T01:00:00Z"
}
}
},
"query": {
"since": "2026-10-24",
"days": 1,
"timezone": "UTC"
},
"asOf": "2026-10-27T00:00:00Z"
}
Original file line number Diff line number Diff line change
@@ -0,0 +1,39 @@
import { commandScopeSchema, receiptSchema, type CommandScope, type Selection } from "./schemas";
/** Proposal only. Reconcile is the existing current-authorized command read, not a client receipt. */
export async function readRetainedFieldSelection(
input: unknown,
reconcile?: (scope: CommandScope) => Promise<unknown>,
): Promise<
{ state: "ready"; selection: Selection } | { state: "invalid" | "unavailable" | "error" }
> {
const scope = commandScopeSchema.safeParse(input);
if (!scope.success) return { state: "invalid" };
if (!reconcile) return { state: "unavailable" };
try {
const parsed = receiptSchema.safeParse(await reconcile(scope.data));
if (!parsed.success) return { state: "unavailable" };
const r = parsed.data;
if (
r.ownerId !== scope.data.workspaceId ||
r.requestId !== scope.data.requestId ||
r.catalogId !== scope.data.catalogId ||
r.key !== scope.data.key
)
return { state: "unavailable" };
return {
state: "ready",
selection: {
ownerId: r.ownerId,
requestId: r.requestId,
subjectId: r.subjectId,
catalogId: r.catalogId,
resultId: r.resultId,
revision: r.revision,
topic: "catalog_retained_streams",
sourceVersionIds: [r.sourceVersionId],
},
};
} catch {
return { state: "error" };
}
}
Loading
Loading