Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
32 changes: 32 additions & 0 deletions .github/workflows/source-withdrawal.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,32 @@
name: Context source withdrawal transactions

on:
pull_request:
branches: [main]
push:
branches: [main]

permissions:
contents: read

jobs:
source-withdrawal:
runs-on: ubuntu-24.04
timeout-minutes: 10
strategy:
fail-fast: false
matrix:
postgres: [15, 16]
steps:
- uses: actions/checkout@v4
- name: Install PostgreSQL
run: |
sudo apt-get update
sudo apt-get install -y postgresql-common
sudo /usr/share/postgresql-common/pgdg/apt.postgresql.org.sh -y

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2: The bootstrap installs its default PostgreSQL server before the matrix-pinned package, adding an unintended server installation to each job and risking the 10-minute timeout. Pass the matrix version to the bootstrap.

(Based on your team's feedback about pinning the PGDG bootstrap version.)

View Feedback

Prompt for AI agents
Check if this issue is valid — if so, understand the root cause and fix it. When an issue isn't valid or won't be fixed in this PR, reply in its thread with the reason and then resolve the thread. At .github/workflows/source-withdrawal.yml, line 26:

<comment>The bootstrap installs its default PostgreSQL server before the matrix-pinned package, adding an unintended server installation to each job and risking the 10-minute timeout. Pass the matrix version to the bootstrap.

(Based on your team's feedback about pinning the PGDG bootstrap version.) </comment>

<file context>
@@ -0,0 +1,32 @@
+        run: |
+          sudo apt-get update
+          sudo apt-get install -y postgresql-common
+          sudo /usr/share/postgresql-common/pgdg/apt.postgresql.org.sh -y
+          sudo apt-get update
+          sudo apt-get install -y postgresql-${{ matrix.postgres }}
</file context>
Suggested change
sudo /usr/share/postgresql-common/pgdg/apt.postgresql.org.sh -y
sudo /usr/share/postgresql-common/pgdg/apt.postgresql.org.sh -y -v ${{ matrix.postgres }}

sudo apt-get update
sudo apt-get install -y postgresql-${{ matrix.postgres }}
- name: Test scoped source withdrawal
run: |
export PATH="/usr/lib/postgresql/${{ matrix.postgres }}/bin:$PATH"
python3 -m unittest discover -s tests -p 'test_source_withdrawal.py'
Original file line number Diff line number Diff line change
@@ -0,0 +1,92 @@
-- Scoped source withdrawal reachable from the shared Context operation. Reuses the
-- existing withdrawal primitive and membership check. Rows are kept: no original
-- bytes are deleted, nothing is corrected, refreshed or recomputed, and no provider
-- or model call is made. The primitive relabels every saved result on the source
-- as withdrawn, replacing its earlier status. Withdrawn evidence disappears from
-- authorized reads, document publication and saved briefs on the next read.
begin;

-- A withdrawn source URL stays withdrawn in its workspace. Writers re-create
-- sources through the active (owner, kind, URL) index, which ignores withdrawn
-- rows, so a slower or later job would otherwise record the same URL as a new
-- active source and publish it again. Such a row is born withdrawn instead: its
-- evidence is kept as history but can never be accepted or read, and writers that
-- require acceptance fail closed. Waiting on the active row first serializes with
-- an in-flight withdrawal, which holds that row FOR UPDATE until it commits. Other
-- workspaces and other source kinds are unaffected. There is no restore operation.
create index context_withdrawn_source_url on public.context_sources(owner_id,kind,source_url)
where withdrawn_at is not null and source_url is not null;
create function public.keep_context_source_withdrawn()
returns trigger language plpgsql set search_path='' as $$
begin
if new.source_url is null or new.withdrawn_at is not null then return new; end if;
perform 1 from public.context_sources s where s.owner_id=new.owner_id and s.kind=new.kind
and s.source_url=new.source_url and s.withdrawn_at is null for key share;
if exists(select 1 from public.context_sources s where s.owner_id=new.owner_id and s.kind=new.kind
and s.source_url=new.source_url and s.withdrawn_at is not null) then new.withdrawn_at:=now(); end if;
return new;
end $$;
revoke all on function public.keep_context_source_withdrawn() from public,anon,authenticated;
create trigger context_sources_keep_withdrawn before insert on public.context_sources
for each row execute function public.keep_context_source_withdrawn();

-- Callers name the source directly or by one of its versions (brief input manifests
-- and document reads expose version IDs). The source must be owned by the workspace
-- and recorded as an input of this request through saved result lineage; arbitrary
-- IDs are rejected. Locks: membership, request, source, then the primitive updates
-- results and documents. Writers that lock a document before reusing the active
-- source row take the opposite order, so a concurrent writer on a shared document
-- can deadlock with a withdrawal; PostgreSQL aborts one transaction and the caller
-- retries. Replay returns the same receipt without touching timestamps or revisions.
create function public.withdraw_context_request_source(p_actor uuid,p_owner uuid,p_request uuid,
p_source uuid default null,p_source_version uuid default null)
returns jsonb language plpgsql set search_path='' as $$
declare req public.context_requests; src public.context_sources; v_source uuid; already boolean;
results_count integer; documents_count integer; affected_requests jsonb;
begin
if (p_source is null)=(p_source_version is null) then
raise exception 'Provide exactly one source or source version' using errcode='22023';
end if;
perform public.authorize_context_case_actor(p_actor,p_owner);
select * into req from public.context_requests where id=p_request and owner_id=p_owner for update;

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P1: A pending enrichment using this source can still commit after this request lock is released: complete_context_enrichment locks only its attempt, then its partial-index upsert can create a new active source row for the withdrawn URL and accept_context_result can republish the result. Cancel or serialize matching pending attempts, or reject their commits before returning.

Prompt for AI agents
Check if this issue is valid — if so, understand the root cause and fix it. When an issue isn't valid or won't be fixed in this PR, reply in its thread with the reason and then resolve the thread. At supabase/migrations/20261010212400_context_source_withdrawal.sql, line 18:

<comment>A pending enrichment using this source can still commit after this request lock is released: `complete_context_enrichment` locks only its attempt, then its partial-index upsert can create a new active source row for the withdrawn URL and `accept_context_result` can republish the result. Cancel or serialize matching pending attempts, or reject their commits before returning.</comment>

<file context>
@@ -0,0 +1,54 @@
+ results_count integer; documents_count integer; affected_requests jsonb;
+begin
+ perform public.authorize_context_case_actor(p_actor,p_owner);
+ select * into req from public.context_requests where id=p_request and owner_id=p_owner for update;
+ if req.id is null then raise exception 'Context request not found'; end if;
+ select * into src from public.context_sources where id=p_source and owner_id=p_owner for update;
</file context>

if req.id is null then raise exception 'Context request not found'; end if;
v_source:=p_source;
if p_source_version is not null then
select v.source_id into v_source from public.context_source_versions v where v.id=p_source_version and v.owner_id=p_owner;
end if;
select * into src from public.context_sources where id=v_source and owner_id=p_owner for update;
if src.id is null or not exists(
select 1 from public.context_attempts a
join public.context_results r on r.attempt_id=a.id and r.owner_id=a.owner_id
join public.context_result_sources rs on rs.result_id=r.id and rs.owner_id=r.owner_id
join public.context_source_versions v on v.id=rs.source_version_id and v.owner_id=rs.owner_id
where a.request_id=p_request and a.owner_id=p_owner and v.source_id=src.id)
then raise exception 'Source is not an input of this request'; end if;
already:=src.withdrawn_at is not null;
if not already then
if not public.withdraw_context_source(p_owner,src.id) then raise exception 'Source withdrawal failed'; end if;
select withdrawn_at into src.withdrawn_at from public.context_sources where id=src.id and owner_id=p_owner;
end if;
-- Recorded input relationships, stable on replay: every saved result that depends
-- on any version of the source, every document (subject and topic) holding such a
-- result, and every request whose attempts produced those results. A document whose
-- current result depended on the source no longer has a current result.
with linked as (
select r.id as result_id,r.subject_id,r.topic,a.request_id
from public.context_attempts a
join public.context_results r on r.attempt_id=a.id and r.owner_id=a.owner_id
join public.context_result_sources rs on rs.result_id=r.id and rs.owner_id=r.owner_id
join public.context_source_versions v on v.id=rs.source_version_id and v.owner_id=rs.owner_id
where a.owner_id=p_owner and v.source_id=src.id)
select count(distinct l.result_id),
(select count(*) from public.context_documents d where d.owner_id=p_owner

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2: Custom agent: Flag AI Slop and Fabricated Changes

The receipt counts any document with historical lineage to the source, even when its current result is unrelated and remains published. Restrict this count to documents whose current result was withheld, such as by restoring the d.current_result_id is null predicate.

Prompt for AI agents
Check if this issue is valid — if so, understand the root cause and fix it. When an issue isn't valid or won't be fixed in this PR, reply in its thread with the reason and then resolve the thread. At supabase/migrations/20261010212400_context_source_withdrawal.sql, line 82:

<comment>The receipt counts any document with historical lineage to the source, even when its current result is unrelated and remains published. Restrict this count to documents whose current result was withheld, such as by restoring the `d.current_result_id is null` predicate.</comment>

<file context>
@@ -1,54 +1,92 @@
+  where a.owner_id=p_owner and v.source_id=src.id)
  select count(distinct l.result_id),
-  (select count(*) from public.context_documents d where d.owner_id=p_owner and d.current_result_id is null
+  (select count(*) from public.context_documents d where d.owner_id=p_owner
     and exists(select 1 from linked x where x.subject_id=d.subject_id and x.topic=d.topic)),
   coalesce((select jsonb_agg(distinct x.request_id) from linked x),'[]'::jsonb)
</file context>
Suggested change
(select count(*) from public.context_documents d where d.owner_id=p_owner
(select count(*) from public.context_documents d where d.owner_id=p_owner and d.current_result_id is null

and exists(select 1 from linked x where x.subject_id=d.subject_id and x.topic=d.topic)),
coalesce((select jsonb_agg(distinct x.request_id) from linked x),'[]'::jsonb)

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P3: The request_ids array has no defined order, so a replay can return a differently ordered receipt despite the stable-receipt contract. Add ORDER BY x.request_id to the aggregate.

Prompt for AI agents
Check if this issue is valid — if so, understand the root cause and fix it. When an issue isn't valid or won't be fixed in this PR, reply in its thread with the reason and then resolve the thread. At supabase/migrations/20261010212400_context_source_withdrawal.sql, line 46:

<comment>The `request_ids` array has no defined order, so a replay can return a differently ordered receipt despite the stable-receipt contract. Add `ORDER BY x.request_id` to the aggregate.</comment>

<file context>
@@ -0,0 +1,54 @@
+ select count(distinct l.result_id),
+  (select count(*) from public.context_documents d where d.owner_id=p_owner and d.current_result_id is null
+    and exists(select 1 from linked x where x.subject_id=d.subject_id and x.topic=d.topic)),
+  coalesce((select jsonb_agg(distinct x.request_id) from linked x),'[]'::jsonb)
+ into results_count,documents_count,affected_requests from linked l;
+ return jsonb_build_object('contract_version','context-source-withdrawal-v1','request_id',p_request,'source_id',p_source,
</file context>
Suggested change
coalesce((select jsonb_agg(distinct x.request_id) from linked x),'[]'::jsonb)
coalesce((select jsonb_agg(distinct x.request_id order by x.request_id) from linked x),'[]'::jsonb)

into results_count,documents_count,affected_requests from linked l;
return jsonb_build_object('contract_version','context-source-withdrawal-v1','request_id',p_request,'source_id',src.id,
'withdrawn_at',src.withdrawn_at,'already_withdrawn',already,
'affected',jsonb_build_object('results',results_count,'documents',documents_count,'request_ids',affected_requests));
end $$;
revoke all on function public.withdraw_context_request_source(uuid,uuid,uuid,uuid,uuid) from public,anon,authenticated;
grant execute on function public.withdraw_context_request_source(uuid,uuid,uuid,uuid,uuid) to service_role;
commit;
6 changes: 6 additions & 0 deletions tests/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -41,3 +41,9 @@ real transaction suite, preserved evidence, explicit enrollment and deployment l
Run `python3 -m unittest discover -s tests -p 'test_player_org_fans.py'` on disposable local PostgreSQL. The suite applies the actual `20261010100000_player_organization_contacts.sql` migration to legacy synthetic fan data and exercises backfill, stable relationship/session IDs, latest non-null contact profiles, parallel sign-ins, separate provider identities despite identical emails, cross-workspace isolation, browser-role denial and existing playback/reporting behavior. CI runs both player suites on PostgreSQL 15 and 16.

Rollout: apply the contact migration before deploying API readers of `contact_id` / `player_fan_contacts`. Existing `connect_player_fan` callers continue working through the new trigger. Artist relationships keep their original IDs; sessions and events are not reassigned. Workspace contacts never merge by email/name or across owners. The migration locks player_fans while backfilling and installing its composite identity FK; new connections can wait briefly during deployment. On API rollback, retain the additive contact schema and trigger so previously deployed capture still maintains contacts. Do not drop the new contacts or remove relationship/session history as a rollback shortcut. Live post-deployment verification is still required.

## Scoped Context source withdrawal

Run `python3 -m unittest discover -s tests -p 'test_source_withdrawal.py'` on disposable local PostgreSQL (CI: `.github/workflows/source-withdrawal.yml` on PostgreSQL 15 and 16). On macOS set `LC_ALL=C` if the postmaster reports it "became multithreaded during startup". The suite reuses the release-cases bootstrap, applies the brief-snapshot migration and `20261010212400_context_source_withdrawal.sql`, and covers: a member withdrawing one private source suppresses only the dependent result, document and saved brief while unrelated documents, a second request and all rows stay intact; the receipt has every documented key and JSON type; a source version ID resolves to its source, and both/neither identifiers, foreign and out-of-lineage versions are rejected; replay returns the same receipt without touching `withdrawn_at` or revisions; `affected` counts follow recorded lineage and do not drift after later withdrawals; `accept_context_result` refuses withdrawn evidence and still enforces its revision fence on live sources; the real `save_context_metadata` writer (captured before the withdrawal) and the shared source upsert cannot re-create the withdrawn URL as an active source, while other kinds and other workspaces are unaffected; a writer started while a withdrawal is uncommitted waits for it and then fails closed (two concurrent sessions in a throwaway database copy); shared lineage across two requests is reported in `affected.request_ids`; foreign and revoked actors get `42501`; sources outside the request's recorded lineage, another workspace's source and unknown requests are rejected without changes; browser roles cannot execute the functions.

Rollout: apply the migration before deploying the API `withdraw_source` operation. It adds one service-role-only invoker function over the existing `withdraw_context_source` primitive and `authorize_context_case_actor` membership check, plus a `BEFORE INSERT` trigger and a partial index on `context_sources`. Once a workspace withdraws a source, any later insert of the same owner, kind and URL is born withdrawn, so writers that need that evidence accepted fail closed (`Context acceptance conflict` / `Source no longer acceptable`) instead of republishing it; there is no restore operation yet. The existing primitive relabels every result on the source as `withdrawn`, replacing its earlier status. A writer that holds a document lock while a withdrawal holds the source can deadlock with it; PostgreSQL aborts one transaction and the caller retries. It does not delete original bytes or storage objects, correct or recompute evidence, or alter retention; those remain separate lifecycle work. Local fixture success is not hosted verification.
Loading
Loading