Skip to content

Mask agent session transcripts and supervisor grade evidence - #2114

Open
ppXD wants to merge 1 commit into
fix/keep-bearer-url-tokens-out-of-logs-and-viewsfrom
fix/redact-agent-transcripts-and-grade-evidence
Open

ppXD wants to merge 1 commit into
fix/keep-bearer-url-tokens-out-of-logs-and-viewsfrom
fix/redact-agent-transcripts-and-grade-evidence

Conversation

@ppXD

@ppXD ppXD commented Oct 12, 2026 •

Copy link
Copy Markdown
Owner

Summary

  • Session transcripts are masked wherever they are stored. The agent's resumable session transcript is now masked with the run's own redactor: the model key, broker and MCP tokens, and the workflow secrets from AgentTask.WorkflowSecrets.
    • This covers result_jsonb.sessionTranscript and its offload, the mid-run checkpoint artifacts, and the AgentRun resume payload.
    • It applies on the launch path and on the re-attach path (AgentRunExecutor.CaptureSessionTranscriptAsync, ArtifactSessionTranscriptCheckpointer).
    • SessionTranscriptRedaction masks inside JSON values, so every line still parses for --resume. It accepts lone-surrogate escapes. On a line it cannot parse (a torn tail), it also replaces each secret's JSON-escaped forms.
  • A warm revise round still gets the exact conversation. It continues the session in-process and its goal only says "continue", so it is handed the session exactly as the agent wrote it. That copy is held in memory beside the masked one and is never written to task_jsonb. Respawns and checkpoint resumes restore the masked copy and re-send the sealed goal.
  • The spool is owner-only. The per-run spool is created 0700 at launch, and out.log, err.log and the FIFOs are written 0600. They hold raw, pre-redaction output and were readable by every local user.
  • Supervisor grading masks secrets. All nine grade lanes, plus the captured and baseline requests, now receive the unit's sealed workflow secrets and the node-scope secrets. clone-failed, grade-error and patch-apply-failed details are masked too. A secret resolved in an operator-floor or model-authored acceptance argv no longer reaches the evidence artifact or the decision-tape tail.

There are no API, schema or wire changes. Rows written before this change keep what they hold.

Test plan

  • Unit: full suite (13676 passed, 1 skipped), including:
    • SessionTranscriptRedactionTests: escaped secrets, a complete line with a lone-surrogate escape, and torn tails in relaxed and ASCII-only escaping.
    • AgentRunExecutorReviseTests: a warm round restores the exact conversation.
    • Tests for spool modes and grade-lane secrets.
  • Integration (Postgres):
    • AgentSessionTranscriptMaskingFlowTests, launch and re-attach arms: result_jsonb, the resume payload and the checkpoints are masked.
    • AgentRunReviseLoopFlowTests warm-revision arm: the revision reads its secret from the exact conversation; result_jsonb, the resume payload and both rounds' checkpoints are masked; task_jsonb stays clean.
    • SupervisorGradeEvidenceMaskingFlowTests.
    • Related revise, checkpoint, re-attach, executor, broker and recovery classes: 409 passed.
  • E2E: in MaskedSessionResumeE2ETests, both attempts are brokered through the container's model-credential broker, the way a Sandbox:RequireConfinement deployment must launch them. The second attempt resumes from the masked session, the run token the first attempt echoed is masked in it, and the respawn's re-sent goal still carries the secret.
  • Mutation checks: each of the following turns a test red:
    • restoring the masked copy into the warm round;
    • leaving the redactor out of the revise round's capture;
    • leaving it out of the re-attach capture;
    • leaving it out of the re-attach checkpoint tick;
    • decoding with Utf8JsonReader.GetString;
    • leaving the escaped forms out of the fallback;
    • capturing the session unmasked (the E2E fails on the echoed run token).
  • Sandbox lane in CI (pinned Claude CLI):
    • GoalChannelE2ETests masked-resume arm, root and non-root: the restored turn reaches the model masked, and the re-sent goal still carries the value.
    • SpoolPermissionsE2ETests: the spool is owner-only under bubblewrap.
  • Engine lane in CI (Sandbox__RequireConfinement=true, bubblewrap): MaskedSessionResumeE2ETests runs brokered and confined.

An agent's resumable session file is its own: a tool call that printed
its environment, or a goal that carried a workflow secret, puts the
value there verbatim. The live stdout path was always masked, but the
session went around the run's redactor into result_jsonb's
sessionTranscript, the mid-run checkpoint artifacts and the resume
payload a retry is handed, so an echoed model key, broker or MCP token,
or workflow secret was kept at rest and restored into the next attempt.

The capture and the checkpointer now mask the session with the run's
own redactor (the same needles as its events, F1's workflow secrets
included) before it is kept, on the launch path and on the re-attach
that finishes a run whose worker went away. The mask is applied inside
the JSON values that hold a secret, not over the raw line: a value the
CLI escaped is not in the file as its own characters, and a raw match
that ends inside an escape leaves a line the CLI cannot resume from. A
line with no secret keeps its exact bytes; a lone surrogate escape,
which Node writes when it truncates inside an emoji, no longer throws a
complete line out of that path; and a line that is not JSON (a torn
tail) is masked by plain replacement of each secret and of the forms a
JSON writer escapes it to. A checkpoint's byte watermark still measures
the file, not the masked copy.

Only the kept copies are masked, not the live conversation. A warm
revise round continues the session in this process, and its goal only
says to continue, so a secret the original goal carried is in that
conversation and nowhere else. The round is handed the session exactly
as the agent wrote it, held in memory beside the masked copy the result
keeps and never written to task_jsonb. Respawns and checkpoint resumes
restore the masked copy and re-send the sealed goal.

The per-run spool held out.log and err.log, the agent's output before
redaction, at the worker's umask: a 0755 directory around 0644 logs,
beside the 0700 config home. The spool is now created 0700 at launch,
and the copiers write their logs and FIFOs owner-only under a umask of
their own, so the agent keeps the worker's umask for its own files.

The supervisor's nine grade lanes handed the grader no secrets, so an
operator-floor argv or a model-authored acceptance that resolved one
kept it in the evidence artifact a Viewer downloads, its tail on the
decision tape and the baseline's evidence. Each lane now passes the
secrets the unit's sealed task embeds and the node scope secrets its
spec carries; the captured and baseline requests carry them too. Every
grade lane masks its result on the way out, so clone-failed,
grade-error and patch-apply-failed details are masked like the
evidence, and the supervisor's own grade-error details are masked with
the node scope before the tape keeps them.

Rows written before this change keep what they hold; a retry restoring
one is masked again when its own session is captured.
@ppXD
ppXD force-pushed the fix/keep-bearer-url-tokens-out-of-logs-and-views branch from 7b289b0 to c115826 Compare October 13, 2026 07:38
@ppXD
ppXD force-pushed the fix/redact-agent-transcripts-and-grade-evidence branch from f99c9ff to 1e610fe Compare October 13, 2026 07:38
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant