diff --git a/proposals/context-retained-video-2131/CONTRACT.md b/proposals/context-retained-video-2131/CONTRACT.md new file mode 100644 index 0000000..1d81122 --- /dev/null +++ b/proposals/context-retained-video-2131/CONTRACT.md @@ -0,0 +1,144 @@ +# Retained video projection — inactive local-source proposal + +This implements the accepted CE15 source plan using three proposal-schema functions and existing Context tables. The fixture producer is synthetic. No production producer, runtime registration, migration or spending authority is supplied. + +## Local qualification + +Run from this Database checkout with an explicit local PostgreSQL binary directory: + +```sh +PG_BIN=/opt/homebrew/opt/postgresql@15/bin python3 proposals/context-retained-video-2131/test_retained_video.py +PG_BIN=/opt/homebrew/opt/postgresql@17/bin python3 proposals/context-retained-video-2131/test_retained_video.py +``` + +The runner requires the pinned #107 Git object locally; it will fail closed without fetching if absent or mismatched. It verifies pinned Git source hashes and all three local SQL input hashes before PostgreSQL startup, freezes the verified bytes and manifest in its owned directory, then applies those copies and real prerequisite SQL in two separate owned databases (normal and final-main-overrides compatibility). The copied #138 runner machinery is a source reference, not a dependency on Application admission or an inherited qualification pass. No database URL is accepted. All fixture-only grants and synthetic setup are outside retained-video.sql. Fixture seeding is revoked from PUBLIC/browser/service roles and refuses database names outside the two owned qualification profiles before any TRUNCATE. It also requires a Unix socket, disabled TCP listening, and server data/socket directories matching expected settings supplied after the runner verifies its owned-process lease. Missing settings fail closed. Wrong-database and unqualified-profile negative cases preserve existing rows. These checks prevent accidental targeting; names and owner-settable settings are not authentication. The trusted boundary remains the local verified lease and revoked client-role access. + +Use `VIDEO_QUALIFICATION_FAULT=source-export|signal|timeout|worker-exit` with the same command to exercise failure cleanup. Each is expected to exit nonzero while emitting owned cleanup/removal receipts. The worker-exit fault deliberately leaves a child for the supervisor to quiesce. Uncatchable SIGKILL/host loss is not claimed automatically recoverable. Ambiguous identity preserves the owned directory and reports it. A macOS signal permission error is accepted only after bounded independent group-absence checks; a still-present or unverifiable group remains a failure. Postmaster and socket cleanup are checked separately. + +Every negative SQL assertion requires its exact parsed verbose server SQLSTATE; injected trigger faults also require their exact message. Only psql ON_ERROR_STOP script-error exit 3 with one ERROR diagnostic becomes SQLFailure. Executor, connection, signal, timeout and unclassified failures remain separate and cannot satisfy a denial. Negative controls exercise actual missing socket/executable failures and unexpected server states. + +PostgreSQL tests exercise actual service-role calls, transaction rollback, exact replay, CAS, source withdrawal/removal, strict envelopes, 32-source and exact 4 MiB boundaries, both locator rejection/deletion lock orderings with observed blocking PIDs, related mutation locks, and browser/default-service denial. The original #107 SQL fixture runs unchanged and its rollback is checked afterward. Generic enrichment function definitions are compared before/after proposal installation, and video_narrative remains rejected by the real claim. No paid execution/accounting tables are created or changed by this proposal. + +## Decision and exact source boundary + +This implements an **inactive proposal**, outside `supabase/migrations`, which saves and reads a deterministic video-only projection of already retained evidence. It links original `context_source_versions.id` values directly through `context_result_sources`; it never inserts or updates source versions. Source retention/collection itself remains a separate dependency. Synthetic fixtures supply actual retained rows in a disposable database; they do not prove that such rows exist in production. + +Base: Database `7708e337feca212563bdfa48934a72227bcf1b63`. Compose the exact locator migration from open #107 `23262f99d2a6c010f00c459a4b1814d6623c3843` in disposable qualification only. API projection/schema reference is merged `21d9971c9e6152cf11fedbe8de0402e1fc806591` (#1276). All 23 exact file hashes are in `sources.json`, including the six previously omitted chain migrations, #107 SQL fixture and actual #138 runner source. No Database AGENTS.md exists in the base tree or the inspected owning checkout; Mono AGENTS.md governs. + +Open #138 `f507a77696f20429f977df9885d6c8c11a5f304b` and #139 `acea75c0e118b6ca12bb9a22470a8bd7b383f925` own the Application 2104 proposal directories. They are not video admission infrastructure. Do not change their files, branches, worktrees, registration policies, qualification runs or claims. A copied runner safety pattern is not a dependency on their application contract or an inherited test pass. + +Existing API #1019 `ce28c4481ac00bd0a28bd686f3224c6ee8375a33` remains the only proposed video intake. Do not duplicate it. #107 is still open. The associated documentation #366 belongs to archived `recoupable/docs-legacy`; a future release must reconcile current Docs separately. + +## Files and dependency order + +All new implementation files belong under `proposals/context-retained-video-2131/` in Database: + +1. `CONTRACT.md`: this reviewed contract, limitations, exact commands and rollout gates. +2. `retained-video.sql`: three proposal-schema functions below; no new tables, triggers, public RPCs or production grants. +3. `qualification-bootstrap.sql`: minimal synthetic legacy tables/roles needed by real prerequisite SQL, explicitly not a full historical migration replay. +4. `fixtures.sql`: actual transactional positive/negative save/read assertions and synthetic retained evidence. +5. `test_retained_video.py`: owned disposable PostgreSQL runner, source composition verification, concurrency and process-cleanup assertions. +6. `sources.json`: exact commits, paths, blob SHA-256, ordered qualification dependencies and runner-copy provenance. + +No migration path is included in this contribution. A future migration name and public grants require a separate release review; moving this file into the auto-applied migration directory is not a routine follow-up. + +Qualification order is bootstrap, then the real main migrations listed below, then #107's exact locator migration, then `retained-video.sql`, then fixtures. Resolve duplicate prerequisite filenames once, never edit their contents: + +``` +20260920010000_context_foundation.sql +20260920020000_context_spotify_pipeline.sql +20260920030000_context_enrichment_pilot.sql +20260922070000_context_catalog_entry.sql +20260924040000_context_release_entry.sql +20260925100000_context_songwriter_name_entry.sql +20260925110000_context_company_name_entry.sql +20260925120000_context_campaign_brief_entry.sql +20260925130000_context_supporting_text_entry.sql +20261008030000_onboarding_membership_lock_privilege.sql +20261008160000_context_release_cases.sql +20261010213100_context_video_entry.sql # exact #107 blob, not main +``` + +This is the existing locator qualification chain plus actual attempt-input and membership authorization prerequisites. PL/pgSQL bodies not exercised by this slice are not qualified merely by loading their definitions. The separate compatibility profile applies the same ordered prerequisite chain above, then `20260923070000_context_enrichment_scope.sql`, then `20260927090000_context_audio_source.sql` from pinned main, then the proposal SQL and fixtures. These final overrides intentionally run after the base chain in this isolated compatibility profile so the final definitions are explicit. Capture function definitions after overrides and compare before/after the proposal, and execute the real final claim with a valid in-scope video subject/module to prove `video_narrative` still rejects without creating an attempt. The normal profile applies the base chain then proposal/fixtures. Neither profile rewrites migration text. `sources.json` → `qualification` records both exact ordered lists and their source-index references. Any necessary bootstrap adjustment must be recorded, not silently patched into migration text. Extract #107's blob by exact commit from the owning repository and verify its recorded digest before SQL; no ambient branch fallback or automatic network fetch in the runner. + +## Functions and input contract + +Proposal namespace: `context_retained_video_2131`. All functions SECURITY INVOKER, `SET search_path=''`, fully qualified relations. Revoke PUBLIC/anon/authenticated/service_role execution and schema access in proposal SQL. Only the disposable fixture explicitly grants the existing synthetic service role schema usage/function execution. Nothing in this contribution exposes a production entry point. + +```sql +context_retained_video_2131.load_retained_evidence( + p_actor uuid, p_owner uuid, p_request uuid, p_snapshot_version uuid +) returns jsonb; + +context_retained_video_2131.save_projection( + p_actor uuid, p_owner uuid, p_request uuid, p_snapshot_version uuid, + p_key text, p_expected_revision bigint +) returns jsonb; + +context_retained_video_2131.read_projection( + p_actor uuid, p_owner uuid, p_request uuid +) returns jsonb; +``` + +The snapshot version is an **existing retained full-evidence snapshot**, not a newly created object containing only references. Its `content` is the exact API `ContextVideoEvidence` envelope. Its `media_manifest` has a required `contextVideo` object `{contractVersion:"retained-video-evidence-v1", requestId, videoSubjectId, videoId, mediaVersion}`. All referenced versions must already exist, have the same owner, and carry the same binding. The snapshot ID must itself occur exactly once in `content.sources`. A producer can allocate its UUID before retaining the snapshot; this proposal does not implement that producer or mutate a retained snapshot to insert its ID later. + +This is a new **proposed retained-producer format**, not a claim that current production sources already satisfy it. Existing locator snapshots do not satisfy it. Missing content, reference-only content, unrecognized manifests or absent retained rows fail closed. For this proposed snapshot format, require its fingerprint to equal SHA-256 of the UTF-8 PostgreSQL JSONB content serialization; record that format-specific convention explicitly rather than applying it to every legacy source kind. Do not create wrapper source versions to make old data pass. The accepted source-implementation scope explicitly includes this absent producer prerequisite; if the desired outcome includes first-time raw evidence retention, commission that separate source-producing contract rather than mislabel this projection writer as ingestion. + +Loading locks and checks the authenticated actor using existing `public.authorize_context_case_actor(actor,owner)` first. The actor must originate from trusted authentication in any future caller; an actor UUID alone is not authentication. Personal owner access follows that existing metadata policy only. No Application #138 entitlement is substituted. Then lock the request FOR SHARE and require owner, `input.kind=video`, partial/completed status, exact YouTube resource/provider ID and a single output video subject for that resource. Lock the exact resource row and then the video subject row FOR SHARE and recheck their IDs, provider/kind/provider ID and subject-resource relationship. Next select the target `context_resource_links` row for that exact resource/subject and relation `identity` FOR SHARE. Do not filter only on `status=accepted` and assume a previous optimistic observation remains valid: require an actual locked row and explicitly recheck its current ID, resource, subject, relation and `status=accepted` after lock acquisition. Missing, deleted, replaced, rejected, candidate or conflicting bindings fail closed with controlled unavailable/permission error and no evidence payload. Never interpret this locator link as artist or recording identity. + +The uniform row-lock order is membership → request → resource → video subject → target locator link → sources/versions (sorted source-ID/version-ID) → document. Save's operation advisory lock remains after the loader's row locks and before document locking. Preserve all these transaction locks through commit/rollback; never release them between helper validation and save/read/replay. The proposal targets the existing READ COMMITTED behavior. Serialization/deadlock/lock-timeout errors fail the whole call; they are not authorization or permission to bypass a check. The target link uses FOR SHARE, not FOR KEY SHARE: status changes must conflict with the held lock. A concurrent UPDATE or DELETE must wait if the authorized operation acquired its lock first. If rejection/deletion commits first, a waiting operation must inspect the locked current row (or missing row) and refuse. It must not return an optimistic pre-wait status. Resource/subject locks similarly prevent the validated locator tuple from changing during the operation. + +Read the snapshot optimistically without a row lock, validate the complete bounded envelope, enumerate all source IDs, then acquire source/version FOR SHARE locks in stable source-ID/version-ID order and re-read/revalidate the snapshot and exact set. If the snapshot or its source set changed while locks were acquired, fail with a controlled conflict; do not acquire an additional unsorted set. Never lock the snapshot version first out of order. Every ID must resolve; no inner-join omission is acceptable. Reject withdrawn sources, removed versions, absent content, owner/request/subject/video/media mismatches or retrieval timestamps unequal to the retained row's instant. Keep original timestamp spelling/text in returned JSON. Accept API UTC minute/second spellings, reject hour 24/leap-second normalization, and reject nonzero sub-microsecond digits that PostgreSQL would round; persisted retrieval instants must be exactly representable by PostgreSQL. Do not trim, normalize or reconstruct retained prose. UUID comparisons use UUID values while the stored envelope preserves the original text. + +SQL must enforce strict keys, types, limits, source-reference closure, duplicate UUID rejection, range ordering/duration, finite numbers and text validity consistent with API #1276. PostgreSQL cannot store malformed surrogate/NUL JSON in the first place; remaining control/blank-only/Unicode cases need explicit parity fixtures. Do not claim `char_length` alone matches JavaScript UTF-16 bounds; count supplementary code points as two units for the API limits. Bound the total JSON byte size to 4 MiB as an explicit narrower storage limit, tested at the boundary. Avoid silent truncation. + +**Attribution restriction:** accept `identity.state=unresolved` only in this first proposal. Preserve observed performer names and their exact retained references; they confer no account link. Reject `matched` explicitly. The current graph's single accepted `identity` link for the YouTube resource is already the video locator; it cannot also express a recording match. Inventing another relation or trusting caller-supplied recording/artist IDs would exceed this slice. Supporting accepted recording/artist attribution requires a separately reviewed persisted binding contract. The API's broader fixture shape remains unchanged; this storage subset is documented and negative-tested. Do not downgrade matched input silently. + +## Save, replay, read and withdrawal + +`save_projection` obtains live evidence through the helper. Validate key against `^[A-Za-z0-9._:-]{1,128}$` and nonnegative expected revision. Derive the operation identity from a domain-separated JSON tuple of actor/owner/request/key; use its SHA-256 as `context_attempts.module`, attempt=1. An advisory transaction lock on that identity serializes identical requests; hash collisions may serialize unrelated work but never authorize it. Store the complete tuple and input fingerprint in attempt.input and compare them, not only the hash. The fingerprint includes snapshot ID, complete retained envelope and expected revision. + +Check for an existing operation after authorization, locked locator-binding revalidation and lineage validation. Exact replay returns the same attempt/result/revision receipt only if the saved result is still accepted and current for this request, with all original lineage valid. Changed actor scope is a different key scope; changed content/snapshot/revision within the same scope is a controlled conflict. A withdrawn, superseded, cancelled or locator-rejected replay never revives or republishes old content. Store the server-observed locator binding `{linkId,resourceId,subjectId,relation:"identity"}` in attempt.input at the original save. Read and replay must compare that exact recorded binding to the currently locked row; a newly inserted accepted replacement link with another ID cannot silently substitute for the original binding. This is server-owned receipt metadata, never caller-provided evidence or a new source version. It returns a controlled unavailable/conflict error, with no evidence payload. An unexpected pre-existing running/unknown attempt is reconciliation-required, not permission to retry or overwrite. + +For a new save, while retaining membership/request/resource/subject/locator-link/source locks, insert the video_narrative document if absent, lock it FOR UPDATE and compare the expected revision (0 for an absent document). Other requests for the same video subject share that document; a current result from another request must never be returned as this request's evidence. Revision checks prevent late replacement. Create the local projection attempt and accepted interpretation result atomically; derive normalized content exactly from the retained envelope plus `originalSongLyrics:not_established`, `scope:video_only`, `artistBrand:not_established`, with partial coverage and retained source-ID trace. Link exactly the supplied original version IDs in `context_result_sources`, including the full snapshot, and call existing `accept_context_result` with the locked revision. Any validation/CAS/accept failure rolls back every inserted row. + +Attempt metadata is provider `retained-evidence`, model `none`, recipe `video-narrative-fixture-v1`, schema `1`, provider cost NULL/status unknown, no settlement reference. The attempt transitions to succeeded inside this same transaction: there is no externally runnable running state or dispatch claim. Store the resulting document revision in attempt.input as a server-owned receipt field. Return `{state:"saved",attemptId,resultId,documentId,revision,evidence}`. Do not overwrite any existing attempt, source, identity, recording or artist row. + +`read_projection` authorizes and validates the request and locks/revalidates the resource, subject and target locator link in the same order as save. It optimistically locates the current video_narrative document/result and original snapshot ID, loads and locks the complete retained evidence/source set before the document, then locks the document and rechecks the current pointer, request-bound attempt, recorded exact locator binding and projection contract. If the pointer changed while acquiring locks, return a controlled retry/conflict without content; never combine old evidence with a new receipt. Use the already locked retained loader result and compare its full envelope to the saved projection input before returning the receipt/evidence. Do not acquire a newly discovered locator or source set after taking the document lock: a changed optimistic binding/pointer fails with a controlled conflict and no payload. Historical fallback, source substitution, newest-version substitution and provider fallback are unsupported. + +Use existing `withdraw_context_source`: direct links ensure it marks this result withdrawn and clears the current document. Independent version removal must also cause read/replay refusal even if the document pointer remains. Check result status and lineage on every read/replay, not only at save. Source-before-document lock ordering follows existing case readers and withdrawal, with membership/request/resource/subject/locator locks first. Locator rejection need not clear the document or mark the result withdrawn: these functions independently refuse to expose it when the recorded locator binding is no longer accepted. Do not claim this protects unrelated existing raw-table or other readers; public wiring remains disabled. Withdrawal/revocation/locator rejection committed before a call must deny it; a transaction already holding the relevant share locks may finish before withdrawal/revocation commits. This is transaction ordering, not retroactive erasure of already delivered bytes. + +## Admission, claims and accounting + +This proposal is a storage transform of already retained data and creates no provider work. Do not call or broaden `claim_context_enrichment`, `complete_context_enrichment`, execution-node claims, reservation functions or the fixture-only accounting adapter. The generic completion writer creates new source versions and is the wrong persistence path for these exact retained references. + +The local projection attempt exists because `context_results` requires an attempt FK; it is not an admitted paid attempt, a wallet receipt, an execution lease, or a zero-cost assertion. Keep original acquisition cost unknown. No customer debit/reservation/settlement or reuse journal is manufactured. An accepted retained result alone must never be treated as permission to dispatch fresh work. + +Future accounting integration may record authorized retained reuse through its reviewed durable reuse path, with zero new debit, only after this read succeeds. The current API adapter requires injected durable storage and has no production policy defaults; this plan supplies neither. Any provider-produced missing evidence still requires separate authenticated admission, budget/reservation composition, single-dispatch and settlement/reconciliation gates **before** acquisition. Do not add a Boolean `admitted` parameter or accept a fabricated claim to bypass those gates. Production execution remains disabled, including nominally free collection. + +## Required real SQL qualification + +Run the owned Unix-socket-only harness on PostgreSQL 15 and 17. Use the exact #138 source `proposals/application-registration-2104/test_registration.py` at `f507a77696f20429f977df9885d6c8c11a5f304b` as the read-only runner reference; its complete Git-blob hash is now in `sources.json`. Copy/adapt only its standalone supervisor, environment, lease and cleanup machinery into the proposed new runner, not its application registration SQL/import or business tests. Record copied function ranges and any changed behavior in the implementation manifest; do not claim code has already been copied or qualified. The actual #107 SQL fixture is separately pinned, in addition to its Python runner. Explicit binary directory, owned HOME/TMPDIR, allowlisted environment, UTF-8 cluster, no inherited libpq/service/credential/loader variables, isolated worker process group, checked lease before SQL, bounded startup/SQL/lock/worker/cleanup waits, and checked postmaster/socket/process cleanup. Never accept a database URL or reuse a developer/hosted database. Run SQL only through the owned disposable harness below. + +Required tests exercise real functions and tables, not text assertions alone: + +- Save/read exact envelope, prose, UUID references and original snapshot IDs; no source/source-version count change. Correct partial interpretation and NULL/unknown costs; no recording/artist document changes. +- Missing/unavailable analysis and captions retain title/channel/description/performer evidence and honest reasons. Matched attribution, locator-only input and reference-only snapshots are explicitly rejected. +- Same-key same-input replay returns the same IDs with no extra rows. Same-key changed snapshot/envelope/revision conflicts; cancelled request and revoked member replay fail. +- Foreign owner/request/resource/subject/source, missing source, mismatched media version/retrieval time, duplicate/case-variant IDs, dangling references, malformed ranges, Unicode/blank/control cases and size/count boundaries fail before durable inserts. +- Source withdrawal and individual version removal block save, read and replay. Withdrawing any original dependency clears current visibility through existing withdrawal semantics; no wrapper can survive it. +- Concurrent identical saves yield one attempt/result; different keys with the same expected revision produce one save and one revision conflict. Lost save reply retries return the same receipt. Fault after each write proves full rollback. +- Two connections exercise contended source withdrawal, membership revocation, document supersession and request cancellation with deterministic barriers and bounded waits. Test both commit orderings and verify no cross-request receipt leakage. +- Add actual two-connection locator lifecycle tests for each of new save, read and exact replay. In operation-first ordering, connection A invokes the real function in an explicit open transaction, retaining its link lock. Connection B attempts `UPDATE context_resource_links SET status='rejected' WHERE id=`; prove B is blocked by A with backend IDs/`pg_blocking_pids`, not elapsed time alone. Commit A, observe B complete and commit, and prove every subsequent save/read/replay refuses with no payload or new rows. A's prior successful result is allowed because rejection committed later. +- In rejection-first ordering, B updates that target link to rejected and holds the transaction; A invokes the real function and must block on B. Verify the blocker, commit B, then assert A resumes and refuses rather than using an earlier accepted observation. No save attempt/result/document change may survive. Repeat the same orderings for link deletion; test candidate/conflict status as ordinary refusal cases. Use bounded statement/lock timeouts and barriers to avoid hanging fixtures. +- Seed a completed projection, then replace its locator link with a different accepted row ID for the same tuple using fixture setup. Read and same-key replay must reject the changed server-recorded binding rather than treat it as the old link. A brand-new save under a new key may proceed only after all normal current-binding, source and expected-revision checks; no automatic retry or replacement is implied. +- Resource/provider-ID or subject-resource mutation committed before invocation must fail binding checks; a concurrent mutation while the operation holds its resource/subject share locks must block. Verify these row locks in the real service-role path. Do not add a locator rewrite or lifecycle trigger just to pass the tests. +- Actual SET ROLE calls prove anon/authenticated denial, proposal default service denial, and disposable explicitly granted service behavior. Do not mistake owner/superuser tests for runtime privilege proof. +- Compare before/after source IDs, locator document, generic claims/definitions, execution-claim counts and any composed accounting rows; this slice must not create dispatch or spending effects. Run #107's actual fixture and rollback assertion unchanged in the composed cluster. +- Runner failure/timeout/signal/exited-worker-child and synthetic poisoned-environment cases prove cleanup on this harness. A reused runner implementation alone does not inherit #138's results. Check fixture rollback after the fixture actually executes. + +## Delivery and remaining gates + +The accepted plan SHA-256 is recorded in sources.json. This contribution stays a six-file inactive proposal; no public RPC, production grant, migration, provider, accounting or API wiring is included. Independent implementation review and PostgreSQL qualification are required before the draft PR is ready for Root's merge decision. + +Future work must separately reconcile locator #107/#1019, establish the real retained-source producer and its authenticated provenance/admission, review the thin API storage reader, and review production schema/grants/release. The full-snapshot manifest and hash do not authenticate its producer. The unresolved-only subset does not establish recording, artist, rights or ownership identity. Source tests never prove hosted access, provider execution, full ingestion or API/MCP/UI parity. diff --git a/proposals/context-retained-video-2131/fixtures.sql b/proposals/context-retained-video-2131/fixtures.sql new file mode 100644 index 0000000..514aea3 --- /dev/null +++ b/proposals/context-retained-video-2131/fixtures.sql @@ -0,0 +1,46 @@ +-- Test-only setup. Never part of retained-video.sql or a hosted migration. +CREATE SCHEMA video_fixture; +CREATE TABLE video_fixture.state(data jsonb); +GRANT USAGE ON SCHEMA video_fixture TO service_role; +GRANT SELECT ON video_fixture.state TO service_role; +CREATE FUNCTION video_fixture.seed() RETURNS jsonb LANGUAGE plpgsql AS $$ +DECLARE owner_id uuid:='10000000-0000-4000-8000-000000000001'; actor uuid:='10000000-0000-4000-8000-000000000002'; + outsider uuid:='10000000-0000-4000-8000-000000000003'; req jsonb; subject uuid; resource uuid; link uuid; + source uuid:=gen_random_uuid(); dependency uuid:=gen_random_uuid(); snapshot uuid:=gen_random_uuid(); version uuid:=gen_random_uuid(); + e jsonb; manifest jsonb; state jsonb; +BEGIN + IF current_database() NOT IN ('video_normal','video_compat') + OR inet_server_addr() IS NOT NULL + OR current_setting('listen_addresses') <> '' + OR current_setting('video_fixture.expected_data_directory',true) IS DISTINCT FROM current_setting('data_directory') + OR current_setting('video_fixture.expected_socket_directory',true) IS DISTINCT FROM current_setting('unix_socket_directories') THEN + RAISE EXCEPTION 'Video fixture seeding is test-only' USING ERRCODE='42501'; + END IF; + TRUNCATE public.accounts,public.context_resources,public.context_sources CASCADE; + TRUNCATE video_fixture.state,public.account_organization_ids; + INSERT INTO public.accounts(id,name) VALUES(owner_id,'Fixture owner'),(actor,'Fixture member'),(outsider,'Fixture outsider'); + INSERT INTO public.account_organization_ids(account_id,organization_id) VALUES(actor,owner_id); + req:=public.create_context_video_request(owner_id,actor,'AbCdEfGhI_1','locator'); + SELECT resource_id,(output->'subjectIds'->>0)::uuid INTO resource,subject FROM public.context_requests WHERE id=(req->>'id')::uuid; + SELECT id INTO link FROM public.context_resource_links WHERE resource_id=resource AND subject_id=subject AND relation='identity'; + e:=jsonb_build_object('requestId',req->>'id','ownerId',owner_id,'videoSubjectId',subject,'videoId','AbCdEfGhI_1', + 'mediaVersion','upload-1','durationSeconds',120,'variant','unknown','title',E' Café 🎵\nTitle ','description',E'Original\n description ', + 'channel',jsonb_build_object('id','channel-1','name','Uploader'), + 'performers',jsonb_build_array(jsonb_build_object('name','Observed performer','sourceVersionIds',jsonb_build_array(version))), + 'identity',jsonb_build_object('state','unresolved','reason','No recording identity established'), + 'sources',jsonb_build_array(jsonb_build_object('versionId',snapshot,'videoId','AbCdEfGhI_1','mediaVersion','upload-1','retrievedAt','2026-11-18T00:00:00Z'), + jsonb_build_object('versionId',version,'videoId','AbCdEfGhI_1','mediaVersion','upload-1','retrievedAt','2026-11-18T00:00:00Z')), + 'captions',jsonb_build_object('state','available','sourceVersionIds',jsonb_build_array(version),'segments',jsonb_build_array(jsonb_build_object('startSeconds',0,'endSeconds',10,'language','en','text','Retained words'))), + 'analysis',jsonb_build_object('state','unavailable','reason','Analysis not retained')); + manifest:=jsonb_build_object('contextVideo',jsonb_build_object('contractVersion','retained-video-evidence-v1','requestId',req->>'id','videoSubjectId',subject,'videoId','AbCdEfGhI_1','mediaVersion','upload-1')); + INSERT INTO public.context_sources(id,owner_id,source_url,kind) VALUES(source,owner_id,'fixture:snapshot','provider_metadata'),(dependency,owner_id,'fixture:captions','provider_metadata'); + INSERT INTO public.context_source_versions(id,owner_id,source_id,fingerprint,content,media_manifest,retrieved_at) + VALUES(snapshot,owner_id,source,encode(sha256(convert_to(e::text,'UTF8')),'hex'),e,manifest,'2026-11-18T00:00:00Z'), + (version,owner_id,dependency,repeat('a',64),jsonb_build_object('captions','Retained words'),manifest,'2026-11-18T00:00:00Z'); + state:=jsonb_build_object('owner',owner_id,'actor',actor,'outsider',outsider,'request',req->>'id','subject',subject,'resource',resource,'link',link,'source',source,'dependency',dependency,'snapshot',snapshot,'version',version,'evidence',e); + INSERT INTO video_fixture.state VALUES(state); + RETURN state; +END $$; +REVOKE ALL ON FUNCTION video_fixture.seed() FROM PUBLIC,anon,authenticated,service_role; +CREATE FUNCTION video_fixture.raise_fault() RETURNS trigger LANGUAGE plpgsql AS $$ +BEGIN RAISE EXCEPTION 'fixture write fault'; END $$; diff --git a/proposals/context-retained-video-2131/qualification-bootstrap.sql b/proposals/context-retained-video-2131/qualification-bootstrap.sql new file mode 100644 index 0000000..0a9b2d8 --- /dev/null +++ b/proposals/context-retained-video-2131/qualification-bootstrap.sql @@ -0,0 +1,16 @@ +-- Synthetic legacy prerequisites only, inside the owned disposable cluster. +CREATE ROLE service_role BYPASSRLS; +CREATE ROLE anon; +CREATE ROLE authenticated; +CREATE SCHEMA storage; +CREATE TABLE storage.buckets(id text PRIMARY KEY,name text,public boolean,file_size_limit bigint); +CREATE TABLE storage.objects(id uuid PRIMARY KEY,bucket_id text); +CREATE TABLE public.accounts(id uuid PRIMARY KEY DEFAULT gen_random_uuid(),name text); +CREATE TABLE public.songs(isrc text PRIMARY KEY,name text,album text); +CREATE TABLE public.song_identifiers(song text,platform text,identifier_type text,value text); +CREATE TABLE public.account_artist_ids(account_id uuid,artist_id uuid); +CREATE TABLE public.catalogs(id uuid PRIMARY KEY,name text,updated_at timestamptz DEFAULT now()); +CREATE TABLE public.account_catalogs(account uuid,catalog uuid); +CREATE TABLE public.account_organization_ids(account_id uuid,organization_id uuid,updated_at timestamptz DEFAULT now(),PRIMARY KEY(account_id,organization_id)); +GRANT USAGE ON SCHEMA public TO service_role; +GRANT SELECT ON public.account_organization_ids TO service_role; diff --git a/proposals/context-retained-video-2131/retained-video.sql b/proposals/context-retained-video-2131/retained-video.sql new file mode 100644 index 0000000..5fcce2e --- /dev/null +++ b/proposals/context-retained-video-2131/retained-video.sql @@ -0,0 +1,235 @@ +-- INACTIVE PROPOSAL. Not a migration; no public RPC or runtime grant. +BEGIN; +CREATE SCHEMA context_retained_video_2131; +REVOKE ALL ON SCHEMA context_retained_video_2131 FROM PUBLIC,anon,authenticated,service_role; + +CREATE FUNCTION context_retained_video_2131.load_retained_evidence( + p_actor uuid,p_owner uuid,p_request uuid,p_snapshot_version uuid +) RETURNS jsonb LANGUAGE plpgsql SECURITY INVOKER SET search_path='' AS $$ +DECLARE + req public.context_requests; resource public.context_resources; subject public.context_subjects; + link public.context_resource_links; snapshot public.context_source_versions; + evidence jsonb; item jsonb; part jsonb; fragment jsonb; ref jsonb; + ids uuid[]; locked_ids uuid[]:='{}'; locked_row record; seen uuid[]; bound jsonb; field text; + texts jsonb:='[]'; txt jsonb; text_value text; units bigint; duration numeric; start_at numeric; end_at numeric; +BEGIN + PERFORM public.authorize_context_case_actor(p_actor,p_owner); + SELECT * INTO req FROM public.context_requests WHERE id=p_request AND owner_id=p_owner FOR SHARE; + IF req.id IS NULL OR req.status NOT IN ('partial','completed') OR req.input->>'kind' IS DISTINCT FROM 'video' + OR jsonb_typeof(req.output->'subjectIds') IS DISTINCT FROM 'array' OR jsonb_array_length(req.output->'subjectIds')<>1 + THEN RAISE EXCEPTION 'Video evidence unavailable' USING ERRCODE='42501'; END IF; + SELECT * INTO resource FROM public.context_resources WHERE id=req.resource_id FOR SHARE; + SELECT * INTO subject FROM public.context_subjects WHERE id=(req.output->'subjectIds'->>0)::uuid FOR SHARE; + SELECT * INTO link FROM public.context_resource_links + WHERE resource_id=req.resource_id AND subject_id=subject.id AND relation='identity' FOR SHARE; + IF resource.id IS NULL OR resource.provider<>'youtube' OR resource.resource_kind<>'video' + OR resource.provider_id !~ '^[A-Za-z0-9_-]{11}$' OR subject.id IS NULL OR subject.kind<>'video' + OR subject.resource_id IS DISTINCT FROM resource.id OR link.id IS NULL OR link.status<>'accepted' + OR link.resource_id IS DISTINCT FROM resource.id OR link.subject_id IS DISTINCT FROM subject.id OR link.relation<>'identity' + OR req.input->>'videoId' IS DISTINCT FROM resource.provider_id + THEN RAISE EXCEPTION 'Video binding unavailable' USING ERRCODE='42501'; END IF; + -- Optimistic enumeration only: do not lock snapshot ahead of the sorted set. + SELECT * INTO snapshot FROM public.context_source_versions WHERE id=p_snapshot_version AND owner_id=p_owner; + evidence:=snapshot.content; + IF evidence IS NULL OR jsonb_typeof(evidence)<>'object' OR octet_length(evidence::text)>4194304 + OR evidence-ARRAY['requestId','ownerId','videoSubjectId','videoId','mediaVersion','durationSeconds','variant','title','description','channel','performers','identity','sources','captions','analysis']<>'{}'::jsonb + OR (SELECT count(*) FROM jsonb_object_keys(evidence))<>15 + THEN RAISE EXCEPTION 'Invalid retained envelope' USING ERRCODE='22023'; END IF; + FOREACH field IN ARRAY ARRAY['requestId','ownerId','videoSubjectId'] LOOP + IF jsonb_typeof(evidence->field) IS DISTINCT FROM 'string' OR evidence->>field !~* '^([0-9a-f]{8}-[0-9a-f]{4}-[1-8][0-9a-f]{3}-[89ab][0-9a-f]{3}-[0-9a-f]{12}|00000000-0000-0000-0000-000000000000|ffffffff-ffff-ffff-ffff-ffffffffffff)$' THEN RAISE EXCEPTION 'Invalid UUID spelling' USING ERRCODE='22023'; END IF; + END LOOP; + IF (evidence->>'requestId')::uuid IS DISTINCT FROM p_request OR (evidence->>'ownerId')::uuid IS DISTINCT FROM p_owner + OR (evidence->>'videoSubjectId')::uuid IS DISTINCT FROM subject.id OR evidence->>'videoId' IS DISTINCT FROM resource.provider_id + OR jsonb_typeof(evidence->'videoId') IS DISTINCT FROM 'string' + OR evidence->>'variant' NOT IN ('original','live','remix','dubbed','unknown') OR jsonb_typeof(evidence->'variant')<>'string' + OR jsonb_typeof(evidence->'sources') IS DISTINCT FROM 'array' OR jsonb_array_length(evidence->'sources') NOT BETWEEN 1 AND 32 + THEN RAISE EXCEPTION 'Invalid retained binding' USING ERRCODE='22023'; END IF; + IF evidence->'durationSeconds'<>'null'::jsonb THEN + IF jsonb_typeof(evidence->'durationSeconds')<>'number' THEN RAISE EXCEPTION 'Invalid duration' USING ERRCODE='22023'; END IF; + duration:=(evidence->>'durationSeconds')::numeric; + IF duration<=0 OR duration::double precision='Infinity'::double precision THEN RAISE EXCEPTION 'Invalid duration' USING ERRCODE='22023'; END IF; + END IF; + IF jsonb_typeof(evidence->'channel') IS DISTINCT FROM 'object' OR (evidence->'channel')-ARRAY['id','name']<>'{}'::jsonb + OR NOT (evidence->'channel' ?& ARRAY['id','name']) + OR jsonb_typeof(evidence->'identity') IS DISTINCT FROM 'object' OR (evidence->'identity')-ARRAY['state','reason']<>'{}'::jsonb + OR evidence->'identity'->>'state' IS DISTINCT FROM 'unresolved' OR NOT(evidence->'identity' ? 'reason') + THEN RAISE EXCEPTION 'Unsupported identity or channel' USING ERRCODE='22023'; END IF; + texts:=jsonb_build_array(jsonb_build_array(evidence->'mediaVersion',4000,true),jsonb_build_array(evidence->'title',4000,true), + jsonb_build_array(evidence->'description',20000,false),jsonb_build_array(evidence->'channel'->'id',4000,true), + jsonb_build_array(evidence->'channel'->'name',4000,true),jsonb_build_array(evidence->'identity'->'reason',4000,true)); + ids:='{}'; + FOR item IN SELECT value FROM jsonb_array_elements(evidence->'sources') LOOP + IF jsonb_typeof(item)<>'object' OR item-ARRAY['versionId','videoId','mediaVersion','retrievedAt']<>'{}'::jsonb + OR NOT(item ?& ARRAY['versionId','videoId','mediaVersion','retrievedAt']) + OR jsonb_typeof(item->'versionId')<>'string' OR item->>'versionId' !~* '^([0-9a-f]{8}-[0-9a-f]{4}-[1-8][0-9a-f]{3}-[89ab][0-9a-f]{3}-[0-9a-f]{12}|00000000-0000-0000-0000-000000000000|ffffffff-ffff-ffff-ffff-ffffffffffff)$' OR jsonb_typeof(item->'retrievedAt')<>'string' + OR item->>'retrievedAt' !~ '^\d{4}-\d{2}-\d{2}T([01]\d|2[0-3]):[0-5]\d(:[0-5]\d(\.\d+)?)?Z$' + OR item->>'retrievedAt' ~ '\.[0-9]{6}[0-9]*[1-9][0-9]*Z$' + OR item->'videoId' IS DISTINCT FROM evidence->'videoId' OR item->'mediaVersion' IS DISTINCT FROM evidence->'mediaVersion' + OR (item->>'versionId')::uuid=ANY(ids) + THEN RAISE EXCEPTION 'Invalid retained source list' USING ERRCODE='22023'; END IF; + ids:=array_append(ids,(item->>'versionId')::uuid); + END LOOP; + IF NOT(p_snapshot_version=ANY(ids)) THEN RAISE EXCEPTION 'Snapshot not in lineage' USING ERRCODE='22023'; END IF; + IF jsonb_typeof(evidence->'performers') IS DISTINCT FROM 'array' OR jsonb_array_length(evidence->'performers')>50 + THEN RAISE EXCEPTION 'Invalid performers' USING ERRCODE='22023'; END IF; + FOR item IN SELECT value FROM jsonb_array_elements(evidence->'performers') LOOP + IF jsonb_typeof(item)<>'object' OR item-ARRAY['name','sourceVersionIds']<>'{}'::jsonb OR NOT(item ?& ARRAY['name','sourceVersionIds']) + THEN RAISE EXCEPTION 'Invalid performer' USING ERRCODE='22023'; END IF; + texts:=texts||jsonb_build_array(jsonb_build_array(item->'name',4000,true)); + END LOOP; + FOREACH field IN ARRAY ARRAY['captions','analysis'] LOOP + part:=evidence->field; + IF jsonb_typeof(part) IS DISTINCT FROM 'object' THEN RAISE EXCEPTION 'Invalid evidence section' USING ERRCODE='22023'; END IF; + IF part->>'state' IN ('missing','unavailable') THEN + IF part-ARRAY['state','reason']<>'{}'::jsonb OR NOT(part ? 'reason') THEN RAISE EXCEPTION 'Invalid unavailable evidence' USING ERRCODE='22023'; END IF; + texts:=texts||jsonb_build_array(jsonb_build_array(part->'reason',4000,true)); + ELSIF part->>'state'='available' THEN + IF part-ARRAY['state','sourceVersionIds',CASE field WHEN 'captions' THEN 'segments' ELSE 'ranges' END]<>'{}'::jsonb + OR NOT(part ? 'sourceVersionIds') THEN RAISE EXCEPTION 'Invalid available evidence' USING ERRCODE='22023'; END IF; + fragment:=part->(CASE field WHEN 'captions' THEN 'segments' ELSE 'ranges' END); + IF jsonb_typeof(fragment) IS DISTINCT FROM 'array' OR jsonb_array_length(fragment) NOT BETWEEN 1 AND (CASE field WHEN 'captions' THEN 500 ELSE 100 END) + THEN RAISE EXCEPTION 'Invalid evidence ranges' USING ERRCODE='22023'; END IF; + FOR item IN SELECT value FROM jsonb_array_elements(fragment) LOOP + IF jsonb_typeof(item)<>'object' OR NOT(item ?& ARRAY['startSeconds','endSeconds']) + OR jsonb_typeof(item->'startSeconds') IS DISTINCT FROM 'number' OR jsonb_typeof(item->'endSeconds') IS DISTINCT FROM 'number' + THEN RAISE EXCEPTION 'Invalid evidence range' USING ERRCODE='22023'; END IF; + start_at:=(item->>'startSeconds')::numeric;end_at:=(item->>'endSeconds')::numeric; + IF start_at<0 OR end_at<=start_at OR (duration IS NOT NULL AND end_at>duration) + OR start_at::double precision='Infinity'::double precision OR end_at::double precision='Infinity'::double precision + THEN RAISE EXCEPTION 'Invalid evidence range' USING ERRCODE='22023'; END IF; + IF field='captions' THEN + IF item-ARRAY['startSeconds','endSeconds','language','text']<>'{}'::jsonb OR NOT(item ?& ARRAY['language','text']) + THEN RAISE EXCEPTION 'Invalid caption' USING ERRCODE='22023'; END IF; + IF jsonb_typeof(item->'language') IS DISTINCT FROM 'string' OR length(item->>'language')=0 THEN RAISE EXCEPTION 'Invalid language' USING ERRCODE='22023'; END IF; + texts:=texts||jsonb_build_array(jsonb_build_array(item->'language',100,false),jsonb_build_array(item->'text',4000,true)); + ELSE + IF item-ARRAY['startSeconds','endSeconds','observations','interpretation']<>'{}'::jsonb OR NOT(item ? 'interpretation') + OR jsonb_typeof(item->'observations') IS DISTINCT FROM 'array' OR jsonb_array_length(item->'observations') NOT BETWEEN 1 AND 50 + THEN RAISE EXCEPTION 'Invalid analysis' USING ERRCODE='22023'; END IF; + texts:=texts||jsonb_build_array(jsonb_build_array(item->'interpretation',4000,true)); + FOR ref IN SELECT value FROM jsonb_array_elements(item->'observations') LOOP texts:=texts||jsonb_build_array(jsonb_build_array(ref,4000,true)); END LOOP; + END IF; + END LOOP; + ELSE RAISE EXCEPTION 'Invalid evidence state' USING ERRCODE='22023'; END IF; + END LOOP; + -- Every reference set is nonempty, unique under UUID comparison and closed over sources. + FOR item IN SELECT value FROM jsonb_array_elements(evidence->'performers') + UNION ALL SELECT evidence->'captions' WHERE evidence->'captions'->>'state'='available' + UNION ALL SELECT evidence->'analysis' WHERE evidence->'analysis'->>'state'='available' + LOOP + IF jsonb_typeof(item->'sourceVersionIds') IS DISTINCT FROM 'array' OR jsonb_array_length(item->'sourceVersionIds') NOT BETWEEN 1 AND 32 + THEN RAISE EXCEPTION 'Invalid evidence references' USING ERRCODE='22023'; END IF; + seen:='{}'; + FOR ref IN SELECT value FROM jsonb_array_elements(item->'sourceVersionIds') LOOP + IF jsonb_typeof(ref)<>'string' OR (ref#>>'{}') !~* '^([0-9a-f]{8}-[0-9a-f]{4}-[1-8][0-9a-f]{3}-[89ab][0-9a-f]{3}-[0-9a-f]{12}|00000000-0000-0000-0000-000000000000|ffffffff-ffff-ffff-ffff-ffffffffffff)$' OR (ref#>>'{}')::uuid=ANY(seen) OR NOT((ref#>>'{}')::uuid=ANY(ids)) + THEN RAISE EXCEPTION 'Unbound evidence reference' USING ERRCODE='22023'; END IF; + seen:=array_append(seen,(ref#>>'{}')::uuid); + END LOOP; + END LOOP; +-- Blankness uses Node Unicode 17.0 \s/Cf/Default_Ignorable_Code_Point union, matching API predicates. + FOR txt IN SELECT value FROM jsonb_array_elements(texts) LOOP + IF jsonb_typeof(txt->0) IS DISTINCT FROM 'string' THEN RAISE EXCEPTION 'Invalid retained text' USING ERRCODE='22023'; END IF; + text_value:=txt->>0; + SELECT length(text_value)+count(*) INTO units FROM regexp_matches(text_value,U&'[\+010000-\+10FFFF]','g'); + IF units>(txt->>1)::integer OR text_value ~ ('['||chr(1)||'-'||chr(8)||chr(11)||chr(12)||chr(14)||'-'||chr(31)||chr(127)||']') + OR ((txt->>2)::boolean AND regexp_replace(text_value,U&'[\0009-\000D\0020\00A0\00AD\034F\0600-\0605\061C\06DD\070F\0890-\0891\08E2\115F-\1160\1680\17B4-\17B5\180B-\180F\2000-\200F\2028-\202F\205F-\206F\3000\3164\FE00-\FE0F\FEFF\FFA0\FFF0-\FFFB\+0110BD\+0110CD\+013430-\+01343F\+01BCA0-\+01BCA3\+01D173-\+01D17A\+0E0000-\+0E0FFF]','','g')='') + THEN RAISE EXCEPTION 'Invalid retained text' USING ERRCODE='22023'; END IF; + END LOOP; + -- Locks remain held through caller completion. Recheck after any wait. + FOR locked_row IN SELECT v.id,v.source_id FROM public.context_sources s JOIN public.context_source_versions v ON v.source_id=s.id AND v.owner_id=s.owner_id + WHERE v.owner_id=p_owner AND v.id=ANY(ids) ORDER BY s.id,v.id FOR SHARE OF s,v LOOP + locked_ids:=array_append(locked_ids,locked_row.id); + END LOOP; + IF cardinality(locked_ids)<>cardinality(ids) OR NOT(locked_ids @> ids) THEN + RAISE EXCEPTION 'Incomplete retained lock set' USING ERRCODE='42501'; + END IF; + SELECT * INTO snapshot FROM public.context_source_versions WHERE id=p_snapshot_version AND owner_id=p_owner; + IF snapshot.content IS DISTINCT FROM evidence OR snapshot.fingerprint IS DISTINCT FROM encode(sha256(convert_to(evidence::text,'UTF8')),'hex') + THEN RAISE EXCEPTION 'Retained snapshot changed' USING ERRCODE='40001'; END IF; + bound:=jsonb_build_object('contractVersion','retained-video-evidence-v1','requestId',p_request,'videoSubjectId',subject.id,'videoId',resource.provider_id,'mediaVersion',evidence->>'mediaVersion'); + FOR item IN SELECT value FROM jsonb_array_elements(evidence->'sources') LOOP + IF NOT EXISTS(SELECT 1 FROM public.context_source_versions v JOIN public.context_sources s ON s.id=v.source_id AND s.owner_id=v.owner_id + WHERE v.id=(item->>'versionId')::uuid AND v.owner_id=p_owner AND v.removed_at IS NULL AND s.withdrawn_at IS NULL + AND v.content IS NOT NULL AND v.content<>'null'::jsonb AND v.media_manifest->'contextVideo'=bound + AND v.retrieved_at=(item->>'retrievedAt')::timestamptz) + THEN RAISE EXCEPTION 'Retained source unavailable' USING ERRCODE='42501'; END IF; + END LOOP; + RETURN evidence; +END $$; + +CREATE FUNCTION context_retained_video_2131.save_projection( + p_actor uuid,p_owner uuid,p_request uuid,p_snapshot_version uuid,p_key text,p_expected_revision bigint +) RETURNS jsonb LANGUAGE plpgsql SECURITY INVOKER SET search_path='' AS $$ +DECLARE e jsonb; identity_key jsonb; operation text; fingerprint text; binding jsonb; a public.context_attempts; + d public.context_documents; result_id uuid; attempt_id uuid; receipt jsonb; item jsonb; content jsonb; +BEGIN + e:=context_retained_video_2131.load_retained_evidence(p_actor,p_owner,p_request,p_snapshot_version); + IF p_key IS NULL OR p_key !~ '^[A-Za-z0-9._:-]{1,128}$' OR p_expected_revision IS NULL OR p_expected_revision<0 + THEN RAISE EXCEPTION 'Invalid projection key/revision' USING ERRCODE='22023'; END IF; + SELECT jsonb_build_object('linkId',l.id,'resourceId',l.resource_id,'subjectId',l.subject_id,'relation',l.relation) INTO STRICT binding + FROM public.context_resource_links l JOIN public.context_requests q ON q.resource_id=l.resource_id + WHERE q.id=p_request AND q.owner_id=p_owner AND l.subject_id=(e->>'videoSubjectId')::uuid AND l.relation='identity' AND l.status='accepted'; + identity_key:=jsonb_build_array('retained-video-projection-v1',p_actor,p_owner,p_request,p_key); + operation:=encode(sha256(convert_to(identity_key::text,'UTF8')),'hex'); + fingerprint:=encode(sha256(convert_to(jsonb_build_array(p_snapshot_version,e,p_expected_revision)::text,'UTF8')),'hex'); + PERFORM pg_advisory_xact_lock(hashtextextended(operation,0)); + SELECT * INTO a FROM public.context_attempts WHERE owner_id=p_owner AND request_id=p_request AND module=operation AND attempt=1; + IF a.id IS NOT NULL THEN + IF a.input->'identity' IS DISTINCT FROM identity_key OR a.input->>'fingerprint' IS DISTINCT FROM fingerprint + OR a.input->'locatorBinding' IS DISTINCT FROM binding THEN RAISE EXCEPTION 'Projection replay conflict' USING ERRCODE='22023'; END IF; + IF a.status<>'succeeded' THEN RAISE EXCEPTION 'Projection requires reconciliation' USING ERRCODE='40001'; END IF; + SELECT * INTO d FROM public.context_documents WHERE owner_id=p_owner AND subject_id=(e->>'videoSubjectId')::uuid AND topic='video_narrative' FOR SHARE; + IF d.current_result_id IS DISTINCT FROM (a.input->'receipt'->>'resultId')::uuid OR NOT EXISTS( + SELECT 1 FROM public.context_results r WHERE r.id=d.current_result_id AND r.owner_id=p_owner AND r.attempt_id=a.id AND r.status='accepted') + THEN RAISE EXCEPTION 'Projection unavailable' USING ERRCODE='42501'; END IF; + RETURN context_retained_video_2131.read_projection(p_actor,p_owner,p_request); + END IF; + INSERT INTO public.context_documents(owner_id,subject_id,topic) VALUES(p_owner,(e->>'videoSubjectId')::uuid,'video_narrative') ON CONFLICT DO NOTHING; + SELECT * INTO STRICT d FROM public.context_documents WHERE owner_id=p_owner AND subject_id=(e->>'videoSubjectId')::uuid AND topic='video_narrative' FOR UPDATE; + IF d.revision<>p_expected_revision THEN RAISE EXCEPTION 'Projection revision conflict' USING ERRCODE='40001'; END IF; + INSERT INTO public.context_attempts(owner_id,request_id,module,attempt,status,provider,model,recipe_version,schema_version,input,started_at,finished_at) + VALUES(p_owner,p_request,operation,1,'succeeded','retained-evidence','none','video-narrative-fixture-v1','1',jsonb_build_object('identity',identity_key,'fingerprint',fingerprint,'evidence',e,'snapshotVersionId',p_snapshot_version,'locatorBinding',binding),now(),now()) RETURNING id INTO attempt_id; + content:=e||jsonb_build_object('originalSongLyrics','not_established','scope','video_only','artistBrand','not_established'); + INSERT INTO public.context_results(owner_id,attempt_id,subject_id,topic,reuse_key,status,evidence_kind,normalized_response,raw_response,coverage) + VALUES(p_owner,attempt_id,d.subject_id,'video_narrative',fingerprint,'accepted','interpretation',content, + jsonb_build_object('method','retained_video_projection','sourceVersionIds',(SELECT jsonb_agg(value->'versionId') FROM jsonb_array_elements(e->'sources'))),jsonb_build_object('extent','partial','recipe','video-narrative-fixture-v1')) RETURNING id INTO result_id; + FOR item IN SELECT value FROM jsonb_array_elements(e->'sources') LOOP + INSERT INTO public.context_result_sources(owner_id,result_id,source_version_id) VALUES(p_owner,result_id,(item->>'versionId')::uuid); + END LOOP; + IF NOT public.accept_context_result(p_owner,d.id,result_id,d.revision) THEN RAISE EXCEPTION 'Projection acceptance failed' USING ERRCODE='40001'; END IF; + receipt:=jsonb_build_object('state','saved','attemptId',attempt_id,'resultId',result_id,'documentId',d.id,'revision',d.revision+1,'evidence',e); + UPDATE public.context_attempts SET input=input||jsonb_build_object('receipt',receipt) WHERE id=attempt_id AND owner_id=p_owner; + RETURN receipt; +END $$; + +CREATE FUNCTION context_retained_video_2131.read_projection(p_actor uuid,p_owner uuid,p_request uuid) +RETURNS jsonb LANGUAGE plpgsql SECURITY INVOKER SET search_path='' AS $$ +DECLARE a public.context_attempts; d public.context_documents; r public.context_results; e jsonb; binding jsonb; actual uuid[]; expected uuid[]; +BEGIN + -- Optimistic lookup selects no payload for return; helper performs authority and sorted locks. + PERFORM public.authorize_context_case_actor(p_actor,p_owner); + SELECT a0.* INTO a FROM public.context_requests q JOIN public.context_documents d0 ON d0.owner_id=q.owner_id + AND d0.subject_id::text=q.output->'subjectIds'->>0 AND d0.topic='video_narrative' + JOIN public.context_results r0 ON r0.id=d0.current_result_id AND r0.owner_id=d0.owner_id + JOIN public.context_attempts a0 ON a0.id=r0.attempt_id AND a0.owner_id=r0.owner_id AND a0.request_id=q.id + WHERE q.id=p_request AND q.owner_id=p_owner; + IF a.id IS NULL THEN RAISE EXCEPTION 'Projection unavailable' USING ERRCODE='42501'; END IF; + e:=context_retained_video_2131.load_retained_evidence(p_actor,p_owner,p_request,(a.input->>'snapshotVersionId')::uuid); + SELECT jsonb_build_object('linkId',l.id,'resourceId',l.resource_id,'subjectId',l.subject_id,'relation',l.relation) INTO STRICT binding + FROM public.context_resource_links l JOIN public.context_requests q ON q.resource_id=l.resource_id + WHERE q.id=p_request AND q.owner_id=p_owner AND l.subject_id=(e->>'videoSubjectId')::uuid AND l.relation='identity' AND l.status='accepted'; + SELECT * INTO d FROM public.context_documents WHERE owner_id=p_owner AND subject_id=(e->>'videoSubjectId')::uuid AND topic='video_narrative' FOR SHARE; + SELECT * INTO r FROM public.context_results WHERE id=d.current_result_id AND owner_id=p_owner; + SELECT array_agg(source_version_id ORDER BY source_version_id) INTO actual FROM public.context_result_sources WHERE result_id=r.id AND owner_id=p_owner; + SELECT array_agg((value->>'versionId')::uuid ORDER BY (value->>'versionId')::uuid) INTO expected FROM jsonb_array_elements(e->'sources'); + IF a.status<>'succeeded' OR a.recipe_version<>'video-narrative-fixture-v1' OR a.provider<>'retained-evidence' OR a.model<>'none' + OR a.input->'evidence' IS DISTINCT FROM e OR a.input->'locatorBinding' IS DISTINCT FROM binding + OR r.id IS DISTINCT FROM (a.input->'receipt'->>'resultId')::uuid OR r.attempt_id IS DISTINCT FROM a.id OR r.status IS DISTINCT FROM 'accepted' + OR r.topic IS DISTINCT FROM 'video_narrative' OR r.subject_id IS DISTINCT FROM (e->>'videoSubjectId')::uuid OR actual IS DISTINCT FROM expected + OR a.input->'receipt'->'evidence' IS DISTINCT FROM e OR d.revision IS DISTINCT FROM (a.input->'receipt'->>'revision')::bigint + OR r.normalized_response IS DISTINCT FROM e||jsonb_build_object('originalSongLyrics','not_established','scope','video_only','artistBrand','not_established') + THEN RAISE EXCEPTION 'Projection unavailable' USING ERRCODE='42501'; END IF; + RETURN a.input->'receipt'; +END $$; +REVOKE ALL ON ALL FUNCTIONS IN SCHEMA context_retained_video_2131 FROM PUBLIC,anon,authenticated,service_role; +COMMIT; diff --git a/proposals/context-retained-video-2131/sources.json b/proposals/context-retained-video-2131/sources.json new file mode 100644 index 0000000..b8f54ea --- /dev/null +++ b/proposals/context-retained-video-2131/sources.json @@ -0,0 +1,342 @@ +{ + "sources": [ + { + "repository": "database", + "commit": "7708e337feca212563bdfa48934a72227bcf1b63", + "path": "supabase/migrations/20260920010000_context_foundation.sql", + "sha256": "e298767479bf60f460dc78f12fe7cade571582bcc808f8e305e61fde72803fe9" + }, + { + "repository": "database", + "commit": "7708e337feca212563bdfa48934a72227bcf1b63", + "path": "supabase/migrations/20260920020000_context_spotify_pipeline.sql", + "sha256": "85975e5bbcf31356ca784aadca6fb24fe577c4a4cf0e534833d192951504f227" + }, + { + "repository": "database", + "commit": "7708e337feca212563bdfa48934a72227bcf1b63", + "path": "supabase/migrations/20260920030000_context_enrichment_pilot.sql", + "sha256": "888eed1c508d5d129880bbd1f974af4af2ebbbee8327e8e27d876af48df2281c" + }, + { + "repository": "database", + "commit": "7708e337feca212563bdfa48934a72227bcf1b63", + "path": "supabase/migrations/20260923070000_context_enrichment_scope.sql", + "sha256": "4f591cac6073b7068ffc5b4766aa88022958d19d7274adc9458eef6633ade1dd" + }, + { + "repository": "database", + "commit": "7708e337feca212563bdfa48934a72227bcf1b63", + "path": "supabase/migrations/20260927090000_context_audio_source.sql", + "sha256": "7bd9fbf2e12a4d3f656fa943c5ad9339e24226605ddcdf0502d898afb654d57e" + }, + { + "repository": "database", + "commit": "7708e337feca212563bdfa48934a72227bcf1b63", + "path": "supabase/migrations/20261008160000_context_release_cases.sql", + "sha256": "095692a2d01d079993114a8b9c2acd0433ccc31efe9b8280ab18908631ac1884" + }, + { + "repository": "database", + "commit": "7708e337feca212563bdfa48934a72227bcf1b63", + "path": "supabase/migrations/20261008030000_onboarding_membership_lock_privilege.sql", + "sha256": "6d6ff0bd683516b7fc5a15c26fd9998f48d44a7a1bddf1ebaf5f27136512f7d3" + }, + { + "repository": "database", + "commit": "23262f99d2a6c010f00c459a4b1814d6623c3843", + "path": "supabase/migrations/20261010213100_context_video_entry.sql", + "sha256": "7812f0020f8c3690ed391012f97ea105984157f3c14275939da3472bd1914d20" + }, + { + "repository": "database", + "commit": "23262f99d2a6c010f00c459a4b1814d6623c3843", + "path": "tests/test_context_video_entry.py", + "sha256": "7663dcc360ca1d44c7226ac09df2b85485112cf2cbe6a1dc6c4de7216df78701" + }, + { + "repository": "api", + "commit": "21d9971c9e6152cf11fedbe8de0402e1fc806591", + "path": "lib/context/video/contextVideoEvidenceSchema.ts", + "sha256": "e768b61775cd2a96124e65bc2d0536f52805ea6b22aa7210043a7256e00c19cd" + }, + { + "repository": "api", + "commit": "21d9971c9e6152cf11fedbe8de0402e1fc806591", + "path": "lib/context/video/contextVideoLimits.ts", + "sha256": "fd6437cd6dca9068a9db9df1e6e7b1d6b7c8e484de6f3319224fb87a27a00a2a" + }, + { + "repository": "api", + "commit": "21d9971c9e6152cf11fedbe8de0402e1fc806591", + "path": "lib/context/video/projectContextVideoEvidence.ts", + "sha256": "eda595f8d82f522e001f43170ea02412f2d8a1ef7efc3395607e79a02f1d63ff" + }, + { + "repository": "api", + "commit": "21d9971c9e6152cf11fedbe8de0402e1fc806591", + "path": "lib/context/accounting/README.md", + "sha256": "680ede7fb7e3f5361dbee17a88333b631cbdcc3bd884608823bec0a658965f31" + }, + { + "repository": "api", + "commit": "21d9971c9e6152cf11fedbe8de0402e1fc806591", + "path": "lib/context/enrichment/runContextEnrichment.ts", + "sha256": "4bb6380d20e89a566ad73b233fc30113bee06aea314f152e196a3bdf17acea5d" + }, + { + "repository": "api", + "commit": "21d9971c9e6152cf11fedbe8de0402e1fc806591", + "path": "lib/context/planning/planStoredContextModules.ts", + "sha256": "4b227dbde6421e28f281be312fd8fb26046cf1d7b05e19e0afe35d44cc6acc2d" + }, + { + "repository": "database", + "commit": "7708e337feca212563bdfa48934a72227bcf1b63", + "path": "supabase/migrations/20260922070000_context_catalog_entry.sql", + "sha256": "1e37dcabd4a594a1d14920ee76410efd9fecbd622054c343cad113186b93b828" + }, + { + "repository": "database", + "commit": "7708e337feca212563bdfa48934a72227bcf1b63", + "path": "supabase/migrations/20260924040000_context_release_entry.sql", + "sha256": "7925253d7eb10ffe417e35cca609344fffbb501ec4a7cdb44a9d9ac13891ab4b" + }, + { + "repository": "database", + "commit": "7708e337feca212563bdfa48934a72227bcf1b63", + "path": "supabase/migrations/20260925100000_context_songwriter_name_entry.sql", + "sha256": "118c08ccb5be7f78759d587740d8db029f5cf83ba8325e7abd7df75528e0eb68" + }, + { + "repository": "database", + "commit": "7708e337feca212563bdfa48934a72227bcf1b63", + "path": "supabase/migrations/20260925110000_context_company_name_entry.sql", + "sha256": "cbab76d87271f38d8e6f2af88fce756095c3b580590e05b619cd50b5b7f8cf5e" + }, + { + "repository": "database", + "commit": "7708e337feca212563bdfa48934a72227bcf1b63", + "path": "supabase/migrations/20260925120000_context_campaign_brief_entry.sql", + "sha256": "42471a3b3fc97bb98fdacbb8fd76df91b3b9a89f9e29164017246d156defffb8" + }, + { + "repository": "database", + "commit": "7708e337feca212563bdfa48934a72227bcf1b63", + "path": "supabase/migrations/20260925130000_context_supporting_text_entry.sql", + "sha256": "12d2d5749be9d7d793bee97b50fdc46d6329cb7586a1926c838a774ba2d1dfa6" + }, + { + "repository": "database", + "commit": "23262f99d2a6c010f00c459a4b1814d6623c3843", + "path": "supabase/tests/context_video_entry.sql", + "sha256": "a1259858c736e2d0487374ad4492503c6cd1743c45f8edf1f21c20dc063726eb" + }, + { + "repository": "database", + "commit": "f507a77696f20429f977df9885d6c8c11a5f304b", + "path": "proposals/application-registration-2104/test_registration.py", + "sha256": "4051b2581646cea5ac2d36a23d5d5e6271dd02cbf0997e0792623289914d230c" + } + ], + "qualification": { + "status": "owned_local_qualification_recipe", + "prefix": [ + { + "path": "proposals/context-retained-video-2131/qualification-bootstrap.sql", + "sha256": "d2c3be8318d3b5b975ed03098c83a082f29aefe8a84b649e505a9f1784888b58" + } + ], + "normal_prerequisites": [ + { + "repository": "database", + "commit": "7708e337feca212563bdfa48934a72227bcf1b63", + "path": "supabase/migrations/20260920010000_context_foundation.sql", + "sha256": "e298767479bf60f460dc78f12fe7cade571582bcc808f8e305e61fde72803fe9" + }, + { + "repository": "database", + "commit": "7708e337feca212563bdfa48934a72227bcf1b63", + "path": "supabase/migrations/20260920020000_context_spotify_pipeline.sql", + "sha256": "85975e5bbcf31356ca784aadca6fb24fe577c4a4cf0e534833d192951504f227" + }, + { + "repository": "database", + "commit": "7708e337feca212563bdfa48934a72227bcf1b63", + "path": "supabase/migrations/20260920030000_context_enrichment_pilot.sql", + "sha256": "888eed1c508d5d129880bbd1f974af4af2ebbbee8327e8e27d876af48df2281c" + }, + { + "repository": "database", + "commit": "7708e337feca212563bdfa48934a72227bcf1b63", + "path": "supabase/migrations/20260922070000_context_catalog_entry.sql", + "sha256": "1e37dcabd4a594a1d14920ee76410efd9fecbd622054c343cad113186b93b828" + }, + { + "repository": "database", + "commit": "7708e337feca212563bdfa48934a72227bcf1b63", + "path": "supabase/migrations/20260924040000_context_release_entry.sql", + "sha256": "7925253d7eb10ffe417e35cca609344fffbb501ec4a7cdb44a9d9ac13891ab4b" + }, + { + "repository": "database", + "commit": "7708e337feca212563bdfa48934a72227bcf1b63", + "path": "supabase/migrations/20260925100000_context_songwriter_name_entry.sql", + "sha256": "118c08ccb5be7f78759d587740d8db029f5cf83ba8325e7abd7df75528e0eb68" + }, + { + "repository": "database", + "commit": "7708e337feca212563bdfa48934a72227bcf1b63", + "path": "supabase/migrations/20260925110000_context_company_name_entry.sql", + "sha256": "cbab76d87271f38d8e6f2af88fce756095c3b580590e05b619cd50b5b7f8cf5e" + }, + { + "repository": "database", + "commit": "7708e337feca212563bdfa48934a72227bcf1b63", + "path": "supabase/migrations/20260925120000_context_campaign_brief_entry.sql", + "sha256": "42471a3b3fc97bb98fdacbb8fd76df91b3b9a89f9e29164017246d156defffb8" + }, + { + "repository": "database", + "commit": "7708e337feca212563bdfa48934a72227bcf1b63", + "path": "supabase/migrations/20260925130000_context_supporting_text_entry.sql", + "sha256": "12d2d5749be9d7d793bee97b50fdc46d6329cb7586a1926c838a774ba2d1dfa6" + }, + { + "repository": "database", + "commit": "7708e337feca212563bdfa48934a72227bcf1b63", + "path": "supabase/migrations/20261008030000_onboarding_membership_lock_privilege.sql", + "sha256": "6d6ff0bd683516b7fc5a15c26fd9998f48d44a7a1bddf1ebaf5f27136512f7d3" + }, + { + "repository": "database", + "commit": "7708e337feca212563bdfa48934a72227bcf1b63", + "path": "supabase/migrations/20261008160000_context_release_cases.sql", + "sha256": "095692a2d01d079993114a8b9c2acd0433ccc31efe9b8280ab18908631ac1884" + }, + { + "repository": "database", + "commit": "23262f99d2a6c010f00c459a4b1814d6623c3843", + "path": "supabase/migrations/20261010213100_context_video_entry.sql", + "sha256": "7812f0020f8c3690ed391012f97ea105984157f3c14275939da3472bd1914d20" + } + ], + "compatibility_prerequisites": [ + { + "repository": "database", + "commit": "7708e337feca212563bdfa48934a72227bcf1b63", + "path": "supabase/migrations/20260920010000_context_foundation.sql", + "sha256": "e298767479bf60f460dc78f12fe7cade571582bcc808f8e305e61fde72803fe9" + }, + { + "repository": "database", + "commit": "7708e337feca212563bdfa48934a72227bcf1b63", + "path": "supabase/migrations/20260920020000_context_spotify_pipeline.sql", + "sha256": "85975e5bbcf31356ca784aadca6fb24fe577c4a4cf0e534833d192951504f227" + }, + { + "repository": "database", + "commit": "7708e337feca212563bdfa48934a72227bcf1b63", + "path": "supabase/migrations/20260920030000_context_enrichment_pilot.sql", + "sha256": "888eed1c508d5d129880bbd1f974af4af2ebbbee8327e8e27d876af48df2281c" + }, + { + "repository": "database", + "commit": "7708e337feca212563bdfa48934a72227bcf1b63", + "path": "supabase/migrations/20260922070000_context_catalog_entry.sql", + "sha256": "1e37dcabd4a594a1d14920ee76410efd9fecbd622054c343cad113186b93b828" + }, + { + "repository": "database", + "commit": "7708e337feca212563bdfa48934a72227bcf1b63", + "path": "supabase/migrations/20260924040000_context_release_entry.sql", + "sha256": "7925253d7eb10ffe417e35cca609344fffbb501ec4a7cdb44a9d9ac13891ab4b" + }, + { + "repository": "database", + "commit": "7708e337feca212563bdfa48934a72227bcf1b63", + "path": "supabase/migrations/20260925100000_context_songwriter_name_entry.sql", + "sha256": "118c08ccb5be7f78759d587740d8db029f5cf83ba8325e7abd7df75528e0eb68" + }, + { + "repository": "database", + "commit": "7708e337feca212563bdfa48934a72227bcf1b63", + "path": "supabase/migrations/20260925110000_context_company_name_entry.sql", + "sha256": "cbab76d87271f38d8e6f2af88fce756095c3b580590e05b619cd50b5b7f8cf5e" + }, + { + "repository": "database", + "commit": "7708e337feca212563bdfa48934a72227bcf1b63", + "path": "supabase/migrations/20260925120000_context_campaign_brief_entry.sql", + "sha256": "42471a3b3fc97bb98fdacbb8fd76df91b3b9a89f9e29164017246d156defffb8" + }, + { + "repository": "database", + "commit": "7708e337feca212563bdfa48934a72227bcf1b63", + "path": "supabase/migrations/20260925130000_context_supporting_text_entry.sql", + "sha256": "12d2d5749be9d7d793bee97b50fdc46d6329cb7586a1926c838a774ba2d1dfa6" + }, + { + "repository": "database", + "commit": "7708e337feca212563bdfa48934a72227bcf1b63", + "path": "supabase/migrations/20261008030000_onboarding_membership_lock_privilege.sql", + "sha256": "6d6ff0bd683516b7fc5a15c26fd9998f48d44a7a1bddf1ebaf5f27136512f7d3" + }, + { + "repository": "database", + "commit": "7708e337feca212563bdfa48934a72227bcf1b63", + "path": "supabase/migrations/20261008160000_context_release_cases.sql", + "sha256": "095692a2d01d079993114a8b9c2acd0433ccc31efe9b8280ab18908631ac1884" + }, + { + "repository": "database", + "commit": "23262f99d2a6c010f00c459a4b1814d6623c3843", + "path": "supabase/migrations/20261010213100_context_video_entry.sql", + "sha256": "7812f0020f8c3690ed391012f97ea105984157f3c14275939da3472bd1914d20" + }, + { + "repository": "database", + "commit": "7708e337feca212563bdfa48934a72227bcf1b63", + "path": "supabase/migrations/20260923070000_context_enrichment_scope.sql", + "sha256": "4f591cac6073b7068ffc5b4766aa88022958d19d7274adc9458eef6633ade1dd" + }, + { + "repository": "database", + "commit": "7708e337feca212563bdfa48934a72227bcf1b63", + "path": "supabase/migrations/20260927090000_context_audio_source.sql", + "sha256": "7bd9fbf2e12a4d3f656fa943c5ad9339e24226605ddcdf0502d898afb654d57e" + } + ], + "suffix": [ + { + "path": "proposals/context-retained-video-2131/retained-video.sql", + "sha256": "207296c823133abd2532240f608faa6308770baccdc370e0c00c93bea780c3fd" + }, + { + "path": "proposals/context-retained-video-2131/fixtures.sql", + "sha256": "f02f65fb09c27439105ea9f129cc51111ddf87e4ede6a9da023eb6ca08377eba" + } + ], + "unchanged_locator_regression_fixture": { + "repository": "database", + "commit": "23262f99d2a6c010f00c459a4b1814d6623c3843", + "path": "supabase/tests/context_video_entry.sql", + "sha256": "a1259858c736e2d0487374ad4492503c6cd1743c45f8edf1f21c20dc063726eb" + }, + "runner_reference": { + "repository": "database", + "commit": "f507a77696f20429f977df9885d6c8c11a5f304b", + "path": "proposals/application-registration-2104/test_registration.py", + "sha256": "4051b2581646cea5ac2d36a23d5d5e6271dd02cbf0997e0792623289914d230c" + } + }, + "plan_sha256": "15b538c55a23e407c734afe796f55920f4af658173db9ddfbb5c5bb3a2909c42", + "runner_adaptations": [ + "Safety functions adapted from pinned #138 test_registration.py, fixture_environment through quiesce_worker; application tests/SQL excluded.", + "REGISTRATION_QUALIFICATION controls renamed VIDEO_QUALIFICATION; root prefix video-qual; worker deadline 240 seconds for two profiles.", + "Source export occurs inside supervised try after environment scrubbing; /usr/bin/git with allowlisted environment and disabled global/system config reads exact local objects only.", + "Added source-export hash-failure fault before initdb and explicit owned-root removal receipt; cluster_created reflects actual cluster directory existence.", + "Worker test suite supplies source composition, real SQL and blocking-PID assertions; no inherited application test pass.", + "Actual macOS exited-worker SIGKILL PermissionError reproduced: bounded checked group absence required after signal denial; a still-present/unverifiable group fails closed. Added unit fault tests and worker-group PID in owned lease for diagnosis; postmaster/socket checks unchanged." + ] +} diff --git a/proposals/context-retained-video-2131/test_retained_video.py b/proposals/context-retained-video-2131/test_retained_video.py new file mode 100644 index 0000000..2e49670 --- /dev/null +++ b/proposals/context-retained-video-2131/test_retained_video.py @@ -0,0 +1,730 @@ +"""Owned disposable SQL qualification only; no DSNs, credentials or hosted targets.""" +import hashlib +import json +import os +import re +import pathlib +import shlex +from pathlib import Path +import shutil +import signal +import subprocess +import sys +import tempfile +import time +import unittest +from concurrent.futures import ThreadPoolExecutor +HERE = Path(__file__).resolve().parent +REPO = HERE.parents[1] + + +def export_sources(root, env): + """Resolve only pinned local Git blobs before PostgreSQL starts.""" + manifest_text = (HERE/'sources.json').read_text() + manifest = json.loads(manifest_text) + local_inputs = manifest['qualification']['prefix'] + manifest['qualification']['suffix'] + expected_paths = {'proposals/context-retained-video-2131/'+name for name in + ['qualification-bootstrap.sql','retained-video.sql','fixtures.sql']} + if len(local_inputs) != 3 or {entry['path'] for entry in local_inputs} != expected_paths: + raise RuntimeError('Invalid local qualification source set') + verified_local = {} + for entry in local_inputs: + blob = (REPO/entry['path']).read_bytes() + if hashlib.sha256(blob).hexdigest() != entry['sha256']: + raise RuntimeError('Local qualification source hash mismatch: '+entry['path']) + verified_local[Path(entry['path']).name] = blob + needed = manifest['qualification']['compatibility_prerequisites'] + [manifest['qualification']['unchanged_locator_regression_fixture']] + target = root/'sources' + target.mkdir(mode=0o700) + (target/'manifest.json').write_text(manifest_text) + for name, blob in verified_local.items(): + (target/name).write_bytes(blob) + for entry in needed: + blob = subprocess.run(['/usr/bin/git', '-C', str(REPO), 'show', entry['commit']+':'+entry['path']], env={**env, 'GIT_CONFIG_NOSYSTEM':'1', 'GIT_CONFIG_GLOBAL':'/dev/null'}, check=True, capture_output=True, timeout=10).stdout + if env.get('VIDEO_QUALIFICATION_FAULT') == 'source-export' or hashlib.sha256(blob).hexdigest() != entry['sha256']: + raise RuntimeError('Pinned source hash mismatch: '+entry['path']) + (target/Path(entry['path']).name).write_bytes(blob) + + +def fixture_environment(root, pg_bin, fault=None): + """Allowlist only owned paths and fixed controls; never copy ambient values.""" + metadata = json.loads((root/'fixture-owner.json').read_text()) + socket = Path(metadata['socket']) + if not root.is_absolute() or root.resolve() != root or socket.parent != root: + raise RuntimeError('Owned absolute fixture root required') + for directory in [root/'home', root/'tmp']: + if directory.is_symlink() or not directory.is_dir() or directory.stat().st_uid != os.getuid(): + raise RuntimeError('Owned fixture home/tmp directory required') + if fault not in (None, 'signal', 'timeout', 'worker-exit', 'source-export'): + raise RuntimeError('Unknown qualification fault') + env = {'HOME': str(root/'home'), 'TMPDIR': str(root/'tmp'), + 'PATH': os.pathsep.join([str(pg_bin), str(Path(sys.executable).resolve().parent), '/usr/bin', '/bin']), + 'LANG': 'C', 'LC_ALL': 'C', 'TZ': 'UTC', + 'VIDEO_QUALIFICATION_BIN': str(pg_bin), + 'VIDEO_QUALIFICATION_SOCKET': str(socket)} + if fault is not None: env['VIDEO_QUALIFICATION_FAULT'] = fault + return env + + +def prepare_worker_environment(): + """Scrub before even lease-check subprocesses; then validate the registered owner.""" + socket = Path(os.environ['VIDEO_QUALIFICATION_SOCKET']) + if not socket.is_absolute() or socket.is_symlink() or socket.resolve() != socket: + raise RuntimeError('Owned absolute Unix socket directory required') + pg_bin = Path(os.environ['VIDEO_QUALIFICATION_BIN']) + fault = os.environ.get('VIDEO_QUALIFICATION_FAULT') + env = fixture_environment(socket.parent, pg_bin, fault) + os.environ.clear() + os.environ.update(env) + return verify_worker_lease() + + +def verify_worker_lease(): + """Reject direct/remote worker targets before any SQL or environment forwarding.""" + import stat + socket = Path(os.environ['VIDEO_QUALIFICATION_SOCKET']) + if not socket.is_absolute() or socket.is_symlink() or socket.resolve() != socket: + raise RuntimeError('Owned absolute Unix socket directory required') + root = socket.parent + metadata = json.loads((root/'fixture-owner.json').read_text()) + info = socket.stat() + cluster = socket/'pg' + if (metadata['socket'], metadata['cluster']) != (str(socket), str(cluster)) or cluster.is_symlink(): + raise RuntimeError('Worker lease path mismatch') + if metadata['worker_pid'] != os.getppid() or (metadata['uid'], metadata['device'], metadata['inode']) != (info.st_uid, info.st_dev, info.st_ino) or info.st_uid != os.getuid(): + raise RuntimeError('Worker lease owner mismatch') + pg_bin = Path(metadata['pg_bin']) + if str(pg_bin) != os.environ['VIDEO_QUALIFICATION_BIN'] or pg_bin.resolve() != pg_bin: + raise RuntimeError('Worker executable binding mismatch') + if not stat.S_ISSOCK((socket/'.s.PGSQL.55473').stat().st_mode): + raise RuntimeError('Owned Unix socket absent') + lines = (cluster/'postmaster.pid').read_text().splitlines() + if Path(lines[1]).resolve() != cluster or _owned_postmasters(cluster) != [int(lines[0])]: + raise RuntimeError('Worker postmaster identity mismatch') + return str(socket), pg_bin + + +def interrupted(_signum, _frame): + raise InterruptedError('Qualification supervisor interrupted') + + +def supervise(): + """Own one random cluster; stop only after exact directory/process identity checks.""" + pg_bin = Path(os.environ.get('PG_BIN', '/opt/homebrew/opt/postgresql@15/bin')).resolve() + fault = os.environ.get('VIDEO_QUALIFICATION_FAULT') + root = Path(tempfile.mkdtemp(prefix='video-qual-', dir='/private/tmp' if Path('/private/tmp').exists() else '/tmp')).resolve() + socket = Path(tempfile.mkdtemp(prefix='recoup-cases-', dir=root)).resolve() + cluster = socket / 'pg'; info = socket.stat() + (root/'fixture-owner.json').write_text(json.dumps({'socket': str(socket), 'cluster': str(cluster), + 'uid': info.st_uid, 'device': info.st_dev, 'inode': info.st_ino, 'worker_pid': os.getpid(), 'pg_bin': str(pg_bin)})) + (root/'home').mkdir(mode=0o700) + (root/'tmp').mkdir(mode=0o700) + env = fixture_environment(root, pg_bin, fault) + os.environ.clear() + os.environ.update(env) + worker = None + old_handlers = {sig: signal.signal(sig, interrupted) for sig in (signal.SIGTERM, signal.SIGINT)} + try: + export_sources(root, env) + subprocess.run([str(pg_bin/'initdb'), '-D', str(cluster), '-A', 'trust', '-E', 'UTF8'], env=env, check=True, capture_output=True, timeout=30) + subprocess.run([str(pg_bin/'pg_ctl'), '-D', str(cluster), '-l', str(root/'postgres.log'), '-w', '-t', '10', + '-o', f"-k {socket} -p 55473 -c listen_addresses=''", 'start'], env=env, check=True, capture_output=True, timeout=15) + worker = subprocess.Popen([str(Path(sys.executable).resolve()), '-I', str(Path(__file__).resolve()), '--worker'], env=env, start_new_session=True) + metadata_path=root/'fixture-owner.json' + metadata=json.loads(metadata_path.read_text()) + metadata['worker_group_pid']=worker.pid + replacement=root/'fixture-owner.next.json' + replacement.write_text(json.dumps(metadata)) + os.replace(replacement,metadata_path) + if env.get('VIDEO_QUALIFICATION_FAULT') == 'signal': + os.kill(os.getpid(), signal.SIGTERM) + return worker.wait(timeout=.3 if env.get('VIDEO_QUALIFICATION_FAULT') == 'timeout' else 240) + finally: + # Ignore repeated termination while bounded cleanup owns the lease. + for sig in old_handlers: signal.signal(sig, signal.SIG_IGN) + try: + if worker: + print('OWNED_WORKER_CLEANUP '+json.dumps(quiesce_worker(worker)), flush=True) + outcome = stop_owned_cluster(root, pg_bin, env, os.getpid()) + print('OWNED_CLUSTER_CLEANUP '+json.dumps(outcome), flush=True) + except BaseException: + print('PRESERVED_AMBIGUOUS_CLUSTER '+str(root), file=sys.stderr) + raise + else: + shutil.rmtree(root) + print('OWNED_ROOT_REMOVED '+str(root), flush=True) + finally: + for sig, handler in old_handlers.items(): signal.signal(sig, handler) + +# Directory/PID-bound cleanup is copied from the pinned source with one receipt correction: +# cluster_created reports actual directory existence for pre-initdb source-export failure. +"""Verify and stop PostgreSQL bound to one registered directory lease.""" + + +def _process_command(pid): + result = subprocess.run(['/bin/ps', '-ww', '-p', str(pid), '-o', 'command='], capture_output=True, text=True, timeout=5) + if result.returncode == 1 and not result.stdout.strip(): + return None + if result.returncode: + raise RuntimeError('Cannot verify fixture process identity') + return result.stdout.strip() or None + + +def _owned_postmasters(cluster): + # Read process metadata locally; expose only PIDs exactly bound to this lease. + result = subprocess.run(['/bin/ps', '-ww', '-axo', 'pid=,command='], capture_output=True, text=True, timeout=5) + if result.returncode: + raise RuntimeError('Cannot enumerate owned startup processes; preserve root') + found = [] + for line in result.stdout.splitlines(): + fields = line.strip().split(None, 1) + if len(fields) != 2 or str(cluster) not in fields[1]: + continue + try: + argv = shlex.split(fields[1]) + except ValueError: + raise RuntimeError('Ambiguous owned startup process; preserve root') + if not argv or pathlib.Path(argv[0]).name != 'postgres' or '-D' not in argv or argv[argv.index('-D') + 1] != str(cluster): + raise RuntimeError('Unquiesced process references owned cluster; preserve root') + found.append(int(fields[0])) + return found + + +def stop_owned_cluster(root, pg_bin, env, worker_pid): + metadata_path = root / 'fixture-owner.json' + if not metadata_path.exists(): + if list(root.glob('recoup-cases-*')): + raise RuntimeError('Unregistered fixture directory; preserve root') + return {'cluster_created': False, 'socket_absent': True, 'postmaster_absent': True} + metadata = json.loads(metadata_path.read_text()) + socket = pathlib.Path(metadata['socket']) + cluster = pathlib.Path(metadata['cluster']) + if socket.parent != root or socket.resolve() != socket or socket.is_symlink() or cluster != socket / 'pg' or cluster.is_symlink(): + raise RuntimeError('Fixture path ownership mismatch; preserve root') + info = socket.stat() + if (info.st_uid, info.st_dev, info.st_ino) != (metadata['uid'], metadata['device'], metadata['inode']) or info.st_uid != os.getuid() or metadata['worker_pid'] != worker_pid: + raise RuntimeError('Fixture directory identity mismatch; preserve root') + pidfile = cluster / 'postmaster.pid' + discovered = _owned_postmasters(cluster) + if len(discovered) > 1: + raise RuntimeError('Multiple postmasters bound to lease; preserve root') + pid = metadata.get('postmaster_pid') + if pidfile.exists(): + lines = pidfile.read_text().splitlines() + pid = int(lines[0]) + if pathlib.Path(lines[1]).resolve() != cluster: + raise RuntimeError('Postmaster data directory mismatch; preserve root') + if discovered and discovered != [pid]: + raise RuntimeError('Postmaster PID identity mismatch; preserve root') + if discovered: + result = subprocess.run([str(pg_bin / 'pg_ctl'), '-D', str(cluster), '-m', 'immediate', '-w', '-t', '5', 'stop'], env=env, capture_output=True, text=True, timeout=8) + if result.returncode: + raise RuntimeError('Parent could not stop owned PostgreSQL: ' + result.stderr) + elif discovered: + # A detached postmaster can precede its PID/socket files. Independently + # bind its command to the exact owned -D path before targeted immediate stop. + pid = discovered[0] + if _owned_postmasters(cluster) != [pid]: + raise RuntimeError('Startup process identity changed; preserve root') + os.kill(pid, signal.SIGQUIT) + deadline = time.monotonic() + 8 + while _owned_postmasters(cluster) and time.monotonic() < deadline: + time.sleep(0.05) + if _owned_postmasters(cluster) or (pid and _process_command(pid)): + raise RuntimeError('Recorded postmaster still present; preserve root') + socket_file = socket / '.s.PGSQL.55473' + if socket_file.exists(): + raise RuntimeError('Owned PostgreSQL socket still present; preserve root') + return {'cluster_created': cluster.exists(), 'registered_socket': str(socket), 'registered_cluster': str(cluster), + 'postmaster_pid': pid, 'discovered_postmasters': discovered, + 'postmaster_absent': True, 'socket_absent': True} + +# Reviewed group cleanup, with a narrow PermissionError enumeration adaptation; provenance in sources.json. +"""Quiesce only the dedicated fixture worker process group.""" + + +def _group_exists(group): + try: + os.killpg(group, 0) + return True + except ProcessLookupError: + return False + except PermissionError: + # An actual exited-worker probe was denied by the host. Require independent + # complete process metadata; never infer absence from a signal denial. + result = subprocess.run(['/bin/ps', '-axo', 'pgid='], capture_output=True, text=True, timeout=5) + if result.returncode: + raise RuntimeError('Cannot enumerate worker groups; preserve owned files') + groups = result.stdout.splitlines() + if not groups or any(not line.strip().isascii() or not line.strip().isdigit() for line in groups): + raise RuntimeError('Invalid worker group enumeration; preserve owned files') + return group in {int(line.strip()) for line in groups} + + +def signal_owned_worker_group(group, sig): + """A signal denial is never absence; require independent bounded absence proof.""" + try: + os.killpg(group, sig) + except ProcessLookupError: + return + except PermissionError: + # macOS can deny signaling a just-exited adopted child group. Never + # ignore a still-present group: wait only for checked enumeration absence. + deadline = time.monotonic() + 5 + while _group_exists(group) and time.monotonic() < deadline: + time.sleep(.05) + if _group_exists(group): + raise RuntimeError('Cannot signal present owned worker group; preserve owned files') + + +def quiesce_worker(worker): + """Signal only the session/group created for this exact Popen child.""" + group = worker.pid # start_new_session=True establishes PID == PGID == SID. + if _group_exists(group): + signal_owned_worker_group(group, signal.SIGTERM) + try: + worker.wait(timeout=1) + except subprocess.TimeoutExpired: + pass + if _group_exists(group): + signal_owned_worker_group(group, signal.SIGKILL) + worker.wait(timeout=5) + deadline = time.monotonic() + 5 + while _group_exists(group) and time.monotonic() < deadline: + time.sleep(0.05) + if _group_exists(group): + raise RuntimeError('Fixture process group not quiescent; preserve owned files') + return {'worker_pid': worker.pid, 'worker_group_absent': True, 'worker_returncode': worker.returncode} + + + +def quote(value): + return "'"+str(value).replace("'", "''")+"'" + + +class PsqlExecutionFailure(RuntimeError): + """Executor, connection, termination, or unclassified diagnostic failure.""" + + +class SQLFailure(RuntimeError): + """A completed ON_ERROR_STOP script error with one verbose server SQLSTATE.""" + def __init__(self, sqlstate, message, result): + super().__init__(f'{sqlstate}: {message}') + self.sqlstate = sqlstate + self.message = message + self.result = result + + +def checked_psql_result(result): + if result.returncode == 0: + return result + # psql exit 3 is ON_ERROR_STOP script failure; 1/2/signals are not SQL denials. + errors = re.findall(r'^ERROR: ([0-9A-Z]{5}): ([^\n]*)$', result.stderr, re.MULTILINE) + if result.returncode != 3 or len(errors) != 1: + raise PsqlExecutionFailure(f'psql execution failed ({result.returncode}): {result.stderr}') + sqlstate, message = errors[0] + raise SQLFailure(sqlstate, message, result) + + +class Video(unittest.TestCase): + @classmethod + def setUpClass(cls): + cls.socket, cls.bin = verify_worker_lease() + cls.env = {**fixture_environment(Path(cls.socket).parent, cls.bin), 'PGHOST':cls.socket, + 'PGPORT':'55473', 'PGDATABASE':'postgres', 'PGOPTIONS':('-c statement_timeout=12000 -c lock_timeout=10000 ' + f'-c video_fixture.expected_data_directory={cls.socket}/pg ' + f'-c video_fixture.expected_socket_directory={cls.socket}')} + # The owned cluster only; profiles use separate databases with identical bootstrap. + cls.sql('CREATE DATABASE video_normal; CREATE DATABASE video_compat;') + manifest = json.loads((Path(cls.socket).parent/'sources'/'manifest.json').read_text()) + for profile in ['normal','compatibility']: + cls.database = 'video_normal' if profile=='normal' else 'video_compat' + cls.sql((Path(cls.socket).parent/'sources'/'qualification-bootstrap.sql').read_text()) if profile=='normal' else cls.sql((Path(cls.socket).parent/'sources'/'qualification-bootstrap.sql').read_text().replace('CREATE ROLE service_role BYPASSRLS;','').replace('CREATE ROLE anon;','').replace('CREATE ROLE authenticated;','')) + for entry in manifest['qualification'][profile+'_prerequisites']: + cls.sql((Path(cls.socket).parent/'sources'/Path(entry['path']).name).read_text()) + cls.before_definitions = cls.sql("SELECT pg_get_functiondef('public.claim_context_enrichment(uuid,uuid,jsonb)'::regprocedure)||pg_get_functiondef('public.complete_context_enrichment(uuid,uuid,uuid,jsonb)'::regprocedure)") + cls.sql((Path(cls.socket).parent/'sources'/'retained-video.sql').read_text()) + after = cls.sql("SELECT pg_get_functiondef('public.claim_context_enrichment(uuid,uuid,jsonb)'::regprocedure)||pg_get_functiondef('public.complete_context_enrichment(uuid,uuid,uuid,jsonb)'::regprocedure)") + if after != cls.before_definitions: raise AssertionError('Generic enrichment definitions changed') + denied = cls.sql("SELECT has_schema_privilege('service_role','context_retained_video_2131','USAGE'),has_function_privilege('service_role','context_retained_video_2131.save_projection(uuid,uuid,uuid,uuid,text,bigint)','EXECUTE')") + if denied != 'f|f': raise AssertionError('Proposal grants runtime access') + # Execute actual role denials before fixture-only grants. + cls.deny("SET ROLE service_role; SELECT context_retained_video_2131.read_projection(NULL,NULL,NULL)", '42501') + cls.sql('GRANT USAGE ON SCHEMA context_retained_video_2131 TO service_role; GRANT EXECUTE ON ALL FUNCTIONS IN SCHEMA context_retained_video_2131 TO service_role;') + cls.sql((Path(cls.socket).parent/'sources'/'fixtures.sql').read_text()) + regression=(Path(cls.socket).parent/'sources'/'context_video_entry.sql').read_text() + result=cls.execute('BEGIN;\n'+regression+'\nROLLBACK;') + if 'PASS: context_video_entry' not in result.stderr: raise AssertionError('Locator fixture did not run') + if cls.sql('SELECT count(*) FROM public.accounts')!='0': raise AssertionError('Locator rollback not verified') + # The fixture's SQL guard also refuses a different owned database before TRUNCATE. + cls.database='postgres' + cls.sql((Path(cls.socket).parent/'sources'/'fixtures.sql').read_text()) + cls.sql("CREATE TABLE public.accounts(id text); CREATE TABLE public.context_resources(id text); CREATE TABLE public.context_sources(id text); INSERT INTO public.accounts VALUES('account sentinel'); INSERT INTO public.context_resources VALUES('resource sentinel'); INSERT INTO public.context_sources VALUES('source sentinel'); INSERT INTO video_fixture.state VALUES('true');") + sentinel_sql="SELECT jsonb_build_array((SELECT jsonb_agg(a) FROM public.accounts a),(SELECT jsonb_agg(r) FROM public.context_resources r),(SELECT jsonb_agg(s) FROM public.context_sources s),(SELECT jsonb_agg(f) FROM video_fixture.state f))" + before=cls.sql(sentinel_sql) + cls.deny('SELECT video_fixture.seed()', '42501', 'Video fixture seeding is test-only') + if cls.sql(sentinel_sql) != before: + raise AssertionError('Rejected fixture seeding changed state') + cls.database='video_normal' + + @classmethod + def args(cls): + return [str(cls.bin/'psql'),'-XqAt','-h',cls.socket,'-p','55473','-d',getattr(cls,'database','postgres'),'-v','ON_ERROR_STOP=1','-v','VERBOSITY=verbose'] + + @classmethod + def execute(cls,sql,name='video-test'): + try: + result=subprocess.run(cls.args(),input=sql,env={**cls.env,'PGAPPNAME':name},capture_output=True,text=True,timeout=18) + except (OSError, subprocess.TimeoutExpired) as error: + raise PsqlExecutionFailure('psql executor failed') from error + return checked_psql_result(result) + + @classmethod + def sql(cls,sql,name='video-test'): + return cls.execute(sql,name).stdout.strip() + + def setUp(self): + self.__class__.database='video_normal' + self.seed() + + def seed(self): + self.f=json.loads(self.sql('SELECT video_fixture.seed()')) + + def save_sql(self,key='save',revision=0,**overrides): + f={**self.f,**overrides} + return 'SET ROLE service_role; SELECT context_retained_video_2131.save_projection('+','.join(quote(f[k]) for k in ['actor','owner','request','snapshot'])+','+quote(key)+','+str(revision)+');' + + def read_sql(self,**overrides): + f={**self.f,**overrides} + return 'SET ROLE service_role; SELECT context_retained_video_2131.read_projection('+','.join(quote(f[k]) for k in ['actor','owner','request'])+');' + + def save(self,**kwargs): return json.loads(self.sql(self.save_sql(**kwargs))) + + @classmethod + def deny(cls,sql,expected_sqlstate,expected_message=None): + try: + cls.execute(sql) + except SQLFailure as error: + if error.sqlstate != expected_sqlstate: + raise AssertionError(f'Expected SQLSTATE {expected_sqlstate}, got {error.sqlstate}: {error.message}') from error + if expected_message is not None and error.message != expected_message: + raise AssertionError(f'Unexpected server error message: {error.message}') from error + if error.result.stdout.strip(): + raise AssertionError('SQL denial returned unexpected payload') + return error.result.stderr + raise AssertionError(f'Expected SQLSTATE {expected_sqlstate}, but SQL succeeded') + + def mutate(self,expression): + self.sql("UPDATE public.context_source_versions SET content="+expression+" WHERE id="+quote(self.f['snapshot'])+"; UPDATE public.context_source_versions SET fingerprint=encode(sha256(convert_to(content::text,'UTF8')),'hex') WHERE id="+quote(self.f['snapshot'])) + + def counts(self): + return self.sql("SELECT jsonb_build_array((SELECT count(*) FROM public.context_attempts),(SELECT count(*) FROM public.context_results),(SELECT count(*) FROM public.context_documents),(SELECT count(*) FROM public.context_result_sources),(SELECT count(*) FROM public.context_sources),(SELECT count(*) FROM public.context_source_versions))") + + def test_roundtrip_replay_lineage_and_both_profiles(self): + for database in ['video_normal','video_compat']: + with self.subTest(database=database): + self.__class__.database=database;self.seed() + source_before=self.sql('SELECT jsonb_agg(id ORDER BY id) FROM public.context_source_versions') + locator_before=self.sql("SELECT to_jsonb(d) FROM public.context_documents d WHERE topic='video_locator'") + saved=self.save(); counts=self.counts() + self.assertEqual(saved['evidence'],self.f['evidence']) + self.assertEqual(saved,json.loads(self.sql(self.read_sql()))) + self.assertEqual(saved,self.save());self.assertEqual(counts,self.counts()) + self.assertEqual(source_before,self.sql('SELECT jsonb_agg(id ORDER BY id) FROM public.context_source_versions')) + self.assertEqual(locator_before,self.sql("SELECT to_jsonb(d) FROM public.context_documents d WHERE topic='video_locator'")) + self.assertEqual('t',self.sql("SELECT provider_cost_micros IS NULL AND provider_cost_status='unknown' AND settlement_reference IS NULL FROM public.context_attempts WHERE id="+quote(saved['attemptId']))) + self.assertEqual(sorted([self.f['snapshot'],self.f['version']]),json.loads(self.sql('SELECT jsonb_agg(source_version_id ORDER BY source_version_id) FROM public.context_result_sources WHERE result_id='+quote(saved['resultId'])))) + self.assertEqual('0',self.sql("SELECT count(*) FROM public.context_documents WHERE topic NOT IN ('video_locator','video_narrative')")) + self.deny(self.save_sql(revision=1), '22023');self.assertEqual(counts,self.counts()) + self.deny(self.save_sql(key='late',revision=0), '40001');self.assertEqual(counts,self.counts()) + module={'fingerprint':'a'*64,'subjectId':self.f['subject'],'topic':'video_narrative','evidenceKind':'interpretation'} + self.deny('SET ROLE service_role; SELECT public.claim_context_enrichment('+quote(self.f['owner'])+','+quote(self.f['request'])+','+quote(json.dumps(module))+'::jsonb)', 'P0001', 'Invalid enrichment module') + self.assertEqual(counts,self.counts()) + + def test_withdrawal_removed_and_identity_lifecycle(self): + changes=["SELECT public.withdraw_context_source({owner},{dependency})", "UPDATE public.context_source_versions SET removed_at=now() WHERE id={version}", + "UPDATE public.context_resource_links SET status='rejected' WHERE id={link}","UPDATE public.context_resource_links SET status='candidate' WHERE id={link}","UPDATE public.context_resource_links SET status='conflict' WHERE id={link}","DELETE FROM public.context_resource_links WHERE id={link}", + "UPDATE public.context_resources SET provider_id='OtherIdAB_2' WHERE id={resource}","UPDATE public.context_requests SET status='cancelled' WHERE id={request}","DELETE FROM public.account_organization_ids WHERE account_id={actor}"] + for change in changes: + with self.subTest(change=change): + self.seed();self.save();self.sql(change.format(**{k:quote(v) for k,v in self.f.items() if k!='evidence'}));counts=self.counts() + self.deny(self.read_sql(), '42501');self.deny(self.save_sql(), '42501');self.deny(self.save_sql(key='new',revision=1), '42501');self.assertEqual(counts,self.counts()) + self.seed();self.save() + self.sql('UPDATE public.context_resource_links SET id=gen_random_uuid() WHERE id='+quote(self.f['link'])) + self.deny(self.read_sql(), '42501');self.deny(self.save_sql(), '22023') + + def test_scope_and_source_rejections(self): + for args,state in [(dict(actor=self.f['outsider']),'42501'),(dict(owner=self.f['outsider']),'42501'),(dict(request='20000000-0000-4000-8000-000000000000'),'42501'),(dict(snapshot=self.f['version']),'22023')]: + before=self.counts();self.deny(self.save_sql(**args), state);self.assertEqual(before,self.counts()) + for change in ["UPDATE public.context_source_versions SET media_manifest='{{}}' WHERE id={version}", + "UPDATE public.context_source_versions SET retrieved_at=retrieved_at+interval '1 second' WHERE id={version}", + "UPDATE public.context_source_versions SET content=NULL WHERE id={version}"]: + self.seed();self.sql(change.format(**{k:quote(v) for k,v in self.f.items() if k!='evidence'}));before=self.counts();self.deny(self.save_sql(), '42501');self.assertEqual(before,self.counts()) + + def test_envelope_negative_matrix(self): + expressions=["content||'{\"extra\":true}'", "content-'title'", "jsonb_set(content,'{identity}','{\"state\":\"matched\"}')", + "jsonb_set(content,'{title}','null')", "jsonb_set(content,'{title}',to_jsonb(repeat(' ',4)))", "jsonb_set(content,'{title}',to_jsonb(chr(8203)))", + "jsonb_set(content,'{title}',to_jsonb(chr(7)))", "jsonb_set(content,'{title}',to_jsonb(repeat('🎵',2001)))", + "jsonb_set(content,'{description}',to_jsonb(repeat('a',20001)))", "jsonb_set(content,'{durationSeconds}','0')", + "jsonb_set(content,'{captions,segments,0,endSeconds}','121')", "jsonb_set(content,'{captions,segments,0,endSeconds}','0')", + "jsonb_set(content,'{captions,segments,0,startSeconds}','-1')", "jsonb_set(content,'{captions,segments,0,text}','\"\"')", + "jsonb_set(content,'{sources}',(content->'sources')||(content->'sources'->0))", "jsonb_set(content,'{sources}','[]')", + "jsonb_set(content,'{sources,1,mediaVersion}','\"wrong\"')", "jsonb_set(content,'{performers,0,sourceVersionIds}','[]')", + "jsonb_set(content,'{performers,0,sourceVersionIds}',jsonb_build_array(gen_random_uuid()))", + "jsonb_set(content,'{ownerId}',to_jsonb(gen_random_uuid()))", "jsonb_set(content,'{videoSubjectId}',to_jsonb(gen_random_uuid()))", + "jsonb_set(content,'{analysis}','{\"state\":\"unavailable\",\"reason\":\"ok\",\"extra\":1}')"] + for expression in expressions: + with self.subTest(expression=expression): + self.seed();self.mutate(expression);before=self.counts();self.deny(self.save_sql(), '22023');self.assertEqual(before,self.counts()) + self.seed();self.mutate("jsonb_set(jsonb_set(content,'{title}',to_jsonb(repeat('🎵',2000))),'{captions}','{\"state\":\"missing\",\"reason\":\"Not retained\"}')") + self.assertEqual('missing',self.save()['evidence']['captions']['state']) + + def holder(self,sql): + import select + p=subprocess.Popen(self.args(),env={**self.env,'PGAPPNAME':'video-holder'},stdin=subprocess.PIPE,stdout=subprocess.PIPE,stderr=subprocess.PIPE,text=True) + self.addCleanup(lambda: p.poll() is None and p.kill()) + p.stdin.write('BEGIN;'+sql+"SELECT 'held';\n");p.stdin.flush() + deadline=time.monotonic()+14;output='' + while time.monotonic()>'ownerId','-','')))", + "jsonb_set(content,'{sources,0,versionId}',to_jsonb('{'||(content->'sources'->0->>'versionId')||'}'))", + "jsonb_set(content,'{title}',to_jsonb(chr(1536)))", + "jsonb_set(content,'{sources,0,retrievedAt}','\"2026-11-17T24:00:00Z\"')", + "jsonb_set(content,'{sources,0,retrievedAt}','\"2026-11-17T23:59:60Z\"')", + "jsonb_set(content,'{sources,0,retrievedAt}','\"2026-11-18T00:00:00.0000001Z\"')"]: + self.seed();self.mutate(expr);self.deny(self.save_sql(), '22023') + self.seed() + analysis={'state':'available','sourceVersionIds':[self.f['version']], 'ranges':[{'startSeconds':0,'endSeconds':10,'observations':['Retained observation'],'interpretation':'Bounded interpretation'}]} + self.mutate("jsonb_set(content,'{analysis}',"+quote(json.dumps(analysis))+"::jsonb)") + saved=self.save();self.assertEqual(analysis,saved['evidence']['analysis']);self.assertEqual(saved,json.loads(self.sql(self.read_sql()))) + for mutation in ["DELETE FROM public.context_result_sources WHERE result_id={result} AND source_version_id={version}", + "UPDATE public.context_results SET normalized_response='{{}}' WHERE id={result}"]: + self.seed();saved=self.save() + self.sql(mutation.format(result=quote(saved['resultId']),version=quote(self.f['version']))) + self.assertIn('42501',self.deny(self.read_sql(), '42501'));self.assertIn('42501',self.deny(self.save_sql(), '42501')) + + def test_missing_foreign_versions_subject_lock_and_supersession(self): + self.sql('DELETE FROM public.context_source_versions WHERE id='+quote(self.f['version'])) + self.assertIn('42501',self.deny(self.save_sql(), '42501')) + self.seed() + # Real foreign-owner source/version rows replace the referenced dependency. + self.sql('DELETE FROM public.context_source_versions WHERE id='+quote(self.f['version'])+'; INSERT INTO public.context_sources(id,owner_id,kind) VALUES(gen_random_uuid(),'+quote(self.f['outsider'])+",'provider_metadata'); INSERT INTO public.context_source_versions(id,owner_id,source_id,fingerprint,content) SELECT "+quote(self.f['version'])+','+quote(self.f['outsider'])+",id,repeat('b',64),'{}' FROM public.context_sources WHERE owner_id="+quote(self.f['outsider'])) + self.assertIn('42501',self.deny(self.save_sql(), '42501')) + for first in ['operation','mutation']: + self.seed() + # A no-op still obtains UPDATE row locks, proving the subject lock exists. + change='UPDATE public.context_subjects SET resource_id=resource_id WHERE id='+quote(self.f['subject'])+';' + if first=='operation':self.race(self.save_sql(),change) + else:self.race(change,self.save_sql()) + self.seed();original=self.save() + replacement=self.save_sql(key='new',revision=1) + self.race(self.read_sql(),replacement) + self.assertNotEqual(original['resultId'],json.loads(self.sql(self.read_sql()))['resultId']) + self.assertIn('42501',self.deny(self.save_sql(), '42501')) + + def test_source_count_and_exact_byte_boundaries(self): + # Add real retained rows, preserving the same owner/request/media binding. + for n in [30,31]: + self.seed() + self.sql("INSERT INTO public.context_source_versions(owner_id,source_id,fingerprint,content,media_manifest,retrieved_at) SELECT owner_id,source_id,encode(sha256(convert_to(n::text,'UTF8')),'hex'),jsonb_build_object('observation',n),media_manifest,retrieved_at FROM public.context_source_versions CROSS JOIN generate_series(1,"+str(n)+") n WHERE id="+quote(self.f['version'])) + self.mutate("jsonb_set(content,'{sources}',(SELECT jsonb_agg(jsonb_build_object('versionId',v.id,'videoId','AbCdEfGhI_1','mediaVersion','upload-1','retrievedAt','2026-11-18T00:00:00Z') ORDER BY v.id) FROM public.context_source_versions v WHERE v.media_manifest ? 'contextVideo'))") + if n==30:self.assertEqual(32,len(self.save()['evidence']['sources'])) + else:self.deny(self.save_sql(), '22023') + self.seed() + analysis={'state':'available','sourceVersionIds':[self.f['version']], 'ranges':[{'startSeconds':0,'endSeconds':10,'observations':['a']*11,'interpretation':'Known'} for _ in range(100)]} + self.mutate("jsonb_set(content,'{analysis}',"+quote(json.dumps(analysis))+"::jsonb)") + base=int(self.sql('SELECT octet_length(content::text) FROM public.context_source_versions WHERE id='+quote(self.f['snapshot']))) + remaining=4194304-base + for row in analysis['ranges']: + for i in range(11): + added=min(3999,remaining);row['observations'][i]='a'*(1+added);remaining-=added + self.assertEqual(remaining,0) + self.mutate("jsonb_set(content,'{analysis}',"+quote(json.dumps(analysis))+"::jsonb)") + self.assertEqual('4194304',self.sql('SELECT octet_length(content::text) FROM public.context_source_versions WHERE id='+quote(self.f['snapshot']))) + self.assertEqual('saved',self.save()['state']) + # One ASCII byte in description changes only total size, within per-field bounds. + self.mutate("jsonb_set(content,'{description}',to_jsonb((content->>'description')||'x'))") + self.deny(self.save_sql(key='oversize',revision=1), '22023') + + def test_numeric_video_id_and_upper_count_limits(self): + self.sql("UPDATE public.context_resources SET provider_id='12345678901' WHERE id="+quote(self.f['resource'])+"; UPDATE public.context_requests SET input=jsonb_set(input,'{videoId}','\"12345678901\"') WHERE id="+quote(self.f['request'])+"; UPDATE public.context_source_versions SET media_manifest=jsonb_set(media_manifest,'{contextVideo,videoId}','\"12345678901\"') WHERE media_manifest ? 'contextVideo'") + self.mutate("jsonb_set(jsonb_set(jsonb_set(content,'{videoId}','12345678901'),'{sources,0,videoId}','12345678901'),'{sources,1,videoId}','12345678901')") + self.deny(self.save_sql(), '22023') + self.seed() + for path,count,item in [('performers',51,self.f['evidence']['performers'][0]),('captions.segments',501,self.f['evidence']['captions']['segments'][0])]: + self.seed();self.mutate("jsonb_set(content,"+quote('{'+path.replace('.',',')+'}')+','+quote(json.dumps([item]*count))+"::jsonb)") + self.deny(self.save_sql(), '22023') + + def test_signal_denial_never_substitutes_for_group_absence(self): + from unittest.mock import patch + with patch.object(os,'killpg',side_effect=PermissionError('synthetic')): + with patch.dict(globals(),{'_group_exists':lambda group:False}): + signal_owned_worker_group(4242,signal.SIGKILL) + with patch.dict(globals(),{'_group_exists':lambda group:True}): + with patch.object(time,'monotonic',side_effect=[0,6]): + with self.assertRaisesRegex(RuntimeError,'present owned worker group'): + signal_owned_worker_group(4242,signal.SIGKILL) + # The enumeration fallback itself must reject failed or malformed output. + with patch.object(os,'killpg',side_effect=PermissionError('synthetic')): + for code,stdout in [(1,''),(0,''),(0,'not-a-group')]: + with patch.object(subprocess,'run',return_value=subprocess.CompletedProcess([],code,stdout,'')): + with self.assertRaises(RuntimeError):_group_exists(4242) + + def test_denial_rejects_connection_executor_and_unexpected_errors(self): + from unittest.mock import patch + # Actual connection failure stays inside this fixture's owned Unix path. + missing_socket=self.args();missing_socket[missing_socket.index('-h')+1]=str(Path(self.socket)/'absent-socket') + with patch.object(Video,'args',return_value=missing_socket): + with self.assertRaises(PsqlExecutionFailure):self.deny('SELECT 1','42501') + # Actual missing executable is an executor failure, never a SQL refusal. + missing_binary=self.args();missing_binary[0]=str(Path(self.socket)/'absent-psql') + with patch.object(Video,'args',return_value=missing_binary): + with self.assertRaises(PsqlExecutionFailure):self.deny('SELECT 1','42501') + intended='ERROR: 42501: expected authority denial\n' + for code,stderr in [(1,intended),(2,intended),(-15,intended),(3,'connection lost'), + (3,'FATAL: 42501: connection refused\n'),(3,intended+intended)]: + with self.subTest(code=code,stderr=stderr): + result=subprocess.CompletedProcess([],code,'',stderr) + with patch.object(subprocess,'run',return_value=result): + with self.assertRaises(PsqlExecutionFailure):self.deny('SELECT 1','42501') + with patch.object(subprocess,'run',side_effect=subprocess.TimeoutExpired('psql',18)): + with self.assertRaises(PsqlExecutionFailure):self.deny('SELECT 1','42501') + for state in ['42601','42883','55P03','57014','22023']: + result=subprocess.CompletedProcess([],3,'',f'ERROR: {state}: unrelated failure mentioning 42501\n') + with patch.object(subprocess,'run',return_value=result): + with self.assertRaisesRegex(AssertionError,'Expected SQLSTATE 42501'): + self.deny('SELECT 1','42501') + result=subprocess.CompletedProcess([],3,'','ERROR: P0001: unrelated trigger failure\n') + with patch.object(subprocess,'run',return_value=result): + with self.assertRaisesRegex(AssertionError,'Unexpected server error message'): + self.deny('SELECT 1','P0001','fixture write fault') + # The intended verbose server diagnostic is accepted; stdout is still forbidden. + with patch.object(subprocess,'run',return_value=subprocess.CompletedProcess([],3,'',intended)): + self.deny('SELECT 1','42501','expected authority denial') + with patch.object(subprocess,'run',return_value=subprocess.CompletedProcess([],3,'leaked payload',intended)): + with self.assertRaisesRegex(AssertionError,'unexpected payload'):self.deny('SELECT 1','42501') + with patch.object(subprocess,'run',return_value=subprocess.CompletedProcess([],0,'','')): + with self.assertRaisesRegex(AssertionError,'SQL succeeded'):self.deny('SELECT 1','42501') + # Real unexpected server SQLSTATE cannot satisfy an authority expectation either. + with self.assertRaisesRegex(AssertionError,'got 42883'): + self.deny('SELECT video_fixture.no_such_function()','42501') + + def test_unqualified_profile_preserves_seeded_rows(self): + from unittest.mock import patch + snapshot="SELECT jsonb_build_array((SELECT jsonb_agg(a ORDER BY id) FROM public.accounts a),(SELECT jsonb_agg(r ORDER BY id) FROM public.context_resources r),(SELECT jsonb_agg(s ORDER BY id) FROM public.context_sources s),(SELECT jsonb_agg(f) FROM video_fixture.state f))" + before=self.sql(snapshot) + # Correct profile name alone cannot qualify this connection for destructive seeding. + with patch.dict(self.env,{'PGOPTIONS':'-c statement_timeout=12000 -c lock_timeout=10000'}): + self.deny('SELECT video_fixture.seed()', '42501', 'Video fixture seeding is test-only') + self.assertEqual(before,self.sql(snapshot)) + + def test_local_sql_manifest_rejects_changed_hash_before_export(self): + from unittest.mock import patch + manifest=json.loads((HERE/'sources.json').read_text()) + manifest['qualification']['prefix'][0]['sha256']='0'*64 + with patch.object(json,'loads',return_value=manifest), patch.object(subprocess,'run') as run: + with self.assertRaisesRegex(RuntimeError,'Local qualification source hash mismatch'): + export_sources(Path(self.socket).parent,self.env) + run.assert_not_called() + + def test_browser_roles_and_poison_environment(self): + for role in ['anon','authenticated']: + self.deny('SET ROLE '+role+'; SELECT context_retained_video_2131.read_projection(NULL,NULL,NULL)', '42501') + for role in ['service_role','anon','authenticated']: + self.deny('SET ROLE '+role+'; SELECT video_fixture.seed()', '42501') + self.assertEqual('UTF8',self.sql('SHOW server_encoding')) + forbidden=['PGSERVICE','PGSERVICEFILE','PGPASSWORD','PGPASSFILE','DATABASE_URL','SUPABASE_SERVICE_ROLE_KEY','DYLD_INSERT_LIBRARIES','LD_PRELOAD','PYTHONPATH','GIT_CONFIG_GLOBAL'] + for name in forbidden:self.assertNotIn(name,os.environ) + root=Path(self.socket).parent + self.assertEqual(str(root/'home'),os.environ['HOME']) + self.assertEqual(str(root/'tmp'),os.environ['TMPDIR']) + from unittest.mock import patch + with patch.dict(os.environ,{name:'synthetic-poison' for name in forbidden}): + clean=fixture_environment(root,self.bin) + observed=subprocess.run([sys.executable,'-I','-c','import os,json;print(json.dumps(dict(os.environ)))'],env=clean,capture_output=True,text=True,check=True,timeout=5) + child=json.loads(observed.stdout) + for name in forbidden:self.assertNotIn(name,child) + self.assertEqual(str(root/'home'),child['HOME']) + + +if __name__ == '__main__': + if sys.argv[1:] == ['--worker']: + socket, pg_bin = prepare_worker_environment() + fault = os.environ.get('VIDEO_QUALIFICATION_FAULT') + if fault == 'timeout': time.sleep(30) + if fault == 'worker-exit': + subprocess.Popen([str(pg_bin/'psql'), '-XqAt', '-h', socket, '-p', '55473', '-d', 'postgres', '-c', 'SELECT pg_sleep(30)'], env=fixture_environment(Path(socket).parent, pg_bin)) + time.sleep(.1) + raise SystemExit(86) + unittest.main(argv=[sys.argv[0]], verbosity=2) + elif len(sys.argv) == 1: + raise SystemExit(supervise()) + else: + raise SystemExit('No connection strings or external targets accepted')