Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
92 changes: 62 additions & 30 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -16,7 +16,7 @@ see [Antigravity 2.0 (IDE) sessions](#antigravity-20-ide-sessions).

By default, the daemon binds a different ephemeral port each `agy` session.
agy-reader discovers the live CLI endpoint and its CSRF token from local
processes on Linux, with session logs as a fallback for older versions.
processes on Linux and macOS, with `cli.log` as a fallback for the HTTP port.

The explicit `export-offline` command also reads supported SQLite sessions directly,
without a running agy daemon or proxy. See [Offline export](#offline-export).
Expand Down Expand Up @@ -185,6 +185,10 @@ point of the contract.

### Sidecar contract

Watch can also preserve a first transcript whose format the reader cannot yet
verify. Its `agyReader.completeness` contains `"state": "unverified"` and a
reason. Doctor counts these files even after agy removes the source.

For every syncable `~/.gemini/antigravity-cli/conversations/<uuid>.db` or `.pb`,
agy-reader writes `<uuid>.trajectory.json` in the same directory. When both
`<uuid>.db` and `<uuid>.pb` exist, as agy 1.2.17 leaves them for a forked
Expand Down Expand Up @@ -404,17 +408,18 @@ someone deliberately forging cache files or filesystem metadata. A changed activ
session still needs content verification and, after a successful sync, parent-link
resolution across the sidecar corpus.

On Linux, CLI discovery checks the daemon's open session log and owned listening
socket before using credentials. By default, discovery uses the
`ANTIGRAVITY_LS_ADDRESS` and `ANTIGRAVITY_CSRF_TOKEN` values agy exports to a
live tool process. A process environment is readable only by its own user, so
the token stays private to you. A running watcher keeps that token in memory
after the tool exits. A fresh watcher that you start yourself, such as the
systemd service, cannot authenticate until it catches another tool process;
commands shorter than its polling interval may be missed entirely.
On Linux and macOS, run `agy-reader --watch` under the same account as a normal
`agy` session. The reader checks the daemon's open session log and listening
socket, then uses the credentials agy exports to a readable tool process. On
macOS, this means a subprocess started by that `agy`, such as a shell command.
The watcher keeps the token in memory after the tool exits, so later text-only
turns can sync. A new watcher or `doctor` needs to see a live tool process to
discover the token for a plain launch. Commands shorter than the watch interval
can finish between polls. The macOS CLI path was verified with agy 1.5.10 on
Apple Silicon.

**Single-user machines only:** for reliable discovery even while agy is idle or
the watcher restarts, launch agy with a fresh token:
**Single-user Linux machines only:** for reliable discovery even while agy is
idle or the watcher restarts, launch agy with a fresh token:

```sh
agy --csrf_token="$(uuidgen)"
Expand Down Expand Up @@ -445,10 +450,11 @@ your user and root can read, and discovery reads it there. So a watcher you
start yourself, such as the systemd service, can authenticate to that agy even
while it is idle.

Every discovery path requires readable `/proc` entries in the same Linux network
namespace and a daemon log under the selected root (`cli.log` or `log/cli-*.log`).
This avoids stale `cli.log` ports left by short-lived CLI invocations. Other
platforms, restricted `/proc` access, and custom log locations can use
CLI process discovery requires an open daemon log under the selected root
(`cli.log` or `log/cli-*.log`). Linux uses `/proc` in the same network namespace.
macOS uses the system `/usr/sbin/lsof` to check the daemon's files and sockets,
and the OS process table to find readable tools under the same user. Restricted
process access, custom log locations, and other platforms can use the
`ANTIGRAVITY_DAEMON_URL` and `ANTIGRAVITY_CSRF_TOKEN` overrides.
Authentication rejection stops the batch
with one `authentication failed; sync blocked` message per poll instead of
Expand Down Expand Up @@ -686,6 +692,8 @@ It checks the following:
rejected until a token is found or pinned via `ANTIGRAVITY_CSRF_TOKEN`.
- **auth** — a reachable CLI daemon that rejects a read-only RPC is reported as
`rejected — sync blocked`, and fails doctor even if the IDE root is healthy.
With a plain CLI launch, run doctor while agy is running a readable external
tool so doctor can discover the current token itself.
- **agy version** — the running `agy --version` compared against the baseline
recorded in `COMPATIBILITY.md`. A skew means the format audit should re-run.
- **sidecars** — SQLite and legacy encrypted `.pb` sessions are verified,
Expand Down Expand Up @@ -741,7 +749,13 @@ matches, watch leaves the sidecar unchanged and retries later. One-shot reads
may return an existing cache with a warning; that fallback cannot refresh its
contents or freshness evidence.

Unknown source fields, schemas, enum values, or projections fail closed and
When the persisted identity is available and the reader encounters unknown
protobuf fields or an unsupported display projection, watch saves the identified
presentation if the sidecar path is empty. An existing file always wins over a
provisional capture. After a compatibility update, watch replaces the provisional
file on successful verification if the source and daemon are still available.

Unknown source fields, schemas, enum values, or projections fail verification and
need a compatibility update. When the source itself causes the refusal (it
cannot be read, lies outside the supported profile, or has a projection the
profile does not support), watch logs it once and does not fetch that
Expand All @@ -760,8 +774,8 @@ watcher or repair historical sidecars until the updated reader is run.

**`Auto-discovery failed and ANTIGRAVITY_DAEMON_URL is not set`**

On Linux, `agy-reader` first looks for the active CLI process and its launch
token or exported tool credentials. As a fallback, it scans `cli.log` inside the
On Linux and macOS, `agy-reader` first looks for the active CLI process and its
credentials. As a fallback, it scans `cli.log` inside the
session root directory
(`~/.gemini/antigravity-cli/` or `$ANTIGRAVITY_CLI_ROOT`) to locate the active
HTTP port. For an IDE root (`~/.gemini/antigravity`) it scans
Expand All @@ -776,7 +790,8 @@ will fail. You can troubleshoot by:
1. Manually overriding the port if necessary. Find the port using:
```bash
ss -tlnp 2>/dev/null | grep agy # Linux
lsof -iTCP -sTCP:LISTEN -anP | grep agy # macOS
pgrep -x agy # macOS: find the CLI PID
lsof -nPa -p <pid> -iTCP -sTCP:LISTEN # macOS: inspect that PID's listeners
```
Use the port explicitly logged as `for HTTP`, not the HTTPS/gRPC port.
For CLI versions that require CSRF, also supply the matching token exported
Expand All @@ -785,6 +800,15 @@ will fail. You can troubleshoot by:
export ANTIGRAVITY_DAEMON_URL=http://127.0.0.1:<port>
```

**`authentication failed; sync blocked` on macOS**

Start the watcher under the same macOS account as `agy`. If the daemon is idle,
wait for agy to run a shell command or another external tool. A shorter interval
such as `--watch-interval=2s` helps catch tools that finish quickly. If you set
an endpoint or token override for a previous session, unset it and restart the
watcher. Use `agy-reader doctor --root "$HOME/.gemini/antigravity-cli"` while the
tool is running to check authentication and sidecar coverage.

**`connection refused`**

The daemon only listens while `agy` (Antigravity CLI) is running, and the port
Expand Down Expand Up @@ -813,7 +837,7 @@ default locations `~/.gemini/antigravity-cli` / `~/.gemini/antigravity`.
| ------------------------ | ------------------------------------------------------------- | ----------------------------------- |
| `ANTIGRAVITY_DAEMON_URL` | Daemon base URL override (optional, auto-detected by default) | unset (optional fallback) |
| `ANTIGRAVITY_CLI_ROOT` | Pin a single session root (suppresses store discovery) | unset (each default store that exists) |
| `ANTIGRAVITY_CSRF_TOKEN` | CSRF token override for daemons that enforce one | unset (auto-discovered for Linux CLI and the IDE) |
| `ANTIGRAVITY_CSRF_TOKEN` | CSRF token override for daemons that enforce one | unset (auto-discovered for Linux and macOS CLI and the IDE) |
| `AGY_READER_LIVE` | Enable live daemon smoke test | unset (test skips) |
| `AGY_READER_TEST_UUID` | Cascade id to use in the live test | unset |

Expand Down Expand Up @@ -851,7 +875,9 @@ Do not combine it with the event-driven service configuration.

### launchd (macOS)

`~/Library/LaunchAgents/dev.mjacobs.agy-reader.plist`:
Save the following as `~/Library/LaunchAgents/dev.mjacobs.agy-reader.plist`.
Use the logged-in user's LaunchAgent so the watcher can see that account's
`agy` processes. Replace both example paths with absolute paths on your Mac.

```xml
<?xml version="1.0" encoding="UTF-8"?>
Expand All @@ -863,24 +889,30 @@ Do not combine it with the event-driven service configuration.
<string>dev.mjacobs.agy-reader</string>
<key>ProgramArguments</key>
<array>
<string>/usr/local/bin/agy-reader</string>
<string>/path/to/agy-reader</string>
<string>--watch</string>
<string>--root</string>
<string>/path/to/home/.gemini/antigravity-cli</string>
</array>
<key>RunAtLoad</key>
<true/>
<key>KeepAlive</key>
<true/>
<key>StandardOutPath</key>
<string>/tmp/agy-reader.out</string>
<key>StandardErrorPath</key>
<string>/tmp/agy-reader.err</string>
</dict>
</plist>
```

Load with
`launchctl load -w ~/Library/LaunchAgents/dev.mjacobs.agy-reader.plist`.
Replace `/usr/local/bin/agy-reader` with your installed binary's absolute path.
Load and inspect the agent from your macOS login session:

```sh
launchctl bootstrap "gui/$(id -u)" "$HOME/Library/LaunchAgents/dev.mjacobs.agy-reader.plist"
launchctl print "gui/$(id -u)/dev.mjacobs.agy-reader"
```

Start `agy` as usual. The agent waits for the daemon and can sync once it sees
an external tool with readable credentials. Run only one watcher for the store.
To switch to a foreground watcher for troubleshooting, first unload the agent
with `launchctl bootout "gui/$(id -u)/dev.mjacobs.agy-reader"`.

## Testing

Expand Down Expand Up @@ -975,7 +1007,7 @@ symlinks it into the gitignored agent skill directories (`.claude/skills/`,

Active development. Currently supports:

- CLI port and CSRF discovery from Linux processes, with `cli.log` fallback;
- CLI port and CSRF discovery from Linux and macOS processes, with `cli.log` fallback;
IDE discovery from `language_server.log` and `main.log`.
- Antigravity 2.0 (IDE) conversations, discovered automatically alongside the
CLI store, including CSRF token discovery for the IDE daemon.
Expand Down
11 changes: 11 additions & 0 deletions docs/compatibility/agy-1.5.10.md
Original file line number Diff line number Diff line change
Expand Up @@ -75,3 +75,14 @@ linked to the pinned compression format. Fedora's system Python uses
zlib-ng, which fails the same earlier overlay check on the baseline commit.
The separate agy-pair suite passes all 64 tests on system Python, including
the pidfd cases skipped by the schema verification interpreter.

## Policy guardian configuration in a later test turn

A follow-up turn advanced the watcher sidecar from 317 to 319 steps. SQLite
and the daemon reached 320, including the completed reply. Three unknown
`PolicyGuardianConfig` fields in the new executor metadata prevented the
final sidecar write. The reader now decodes `deployment` (15),
`enable_irreversible_vetting` (18), and
`enable_reversibility_adjudication` (19) from the authenticated snapshot.
The new profile verifies all 320 steps against both daemon responses and
preserves the three fields, including explicit boolean values, in offline JSON.
10 changes: 9 additions & 1 deletion docs/completeness.md
Original file line number Diff line number Diff line change
Expand Up @@ -11,7 +11,7 @@ partial or same-step stale response.

## Supported representation

Profile `sqlite-v1-format0-agy-1.3.4-5b93565c5f04-planner50-permission2-prompt23-tools143-permission1-tools144-skill1-history2-ide225-options145-compaction146-skill146-finish147-replay147-failedread146-subagents148-media149-tools149-sendmessage149-files1411-media1410-parentmsgs1410-paths1410-denied1412-cwd1412-unrun1412-config150-skill150-frontmatter150-bgcancel150-readlines150-config152-tools153-tools154-tools155-images155-tools156-retained156-legacychoices156-runtime157-planexecute157-tools158-workflow159` supports SQLite
Profile `sqlite-v1-format0-agy-1.3.4-5b93565c5f04-planner50-permission2-prompt23-tools143-permission1-tools144-skill1-history2-ide225-options145-compaction146-skill146-finish147-replay147-failedread146-subagents148-media149-tools149-sendmessage149-files1411-media1410-parentmsgs1410-paths1410-denied1412-cwd1412-unrun1412-config150-skill150-frontmatter150-bgcancel150-readlines150-config152-tools153-tools154-tools155-images155-tools156-retained156-legacychoices156-runtime157-planexecute157-tools158-workflow159-schedule1510-policy1510` supports SQLite
`user_version=1`, the seven known tables and their exact columns, contiguous
zero-based indices, and `step_format=0`. All persisted protobuf fields and enum
values must be known, except the declared source-only fields below.
Expand Down Expand Up @@ -1748,3 +1748,11 @@ the MIME type, and the stored byte count matches the inline data. The projected
step preserves the filename, full byte sequence and MIME type. Markdown names
the media and reports its size. See the [1.5.10 audit](compatibility/agy-1.5.10.md)
for the native evidence and its limits.

The `policy1510` profile adds `PolicyGuardianConfig.deployment` (string
field 15), `enable_irreversible_vetting` (optional boolean field 18), and
`enable_reversibility_adjudication` (optional boolean field 19). The fields
occur together in a stored command assessor configuration in executor
metadata. The full and ordinary responses preserve all three, including
explicit boolean presence. SQLite verification and offline JSON retain
them under the same equality rules as the surrounding metadata.
57 changes: 48 additions & 9 deletions internal/cache/sidecar.go
Original file line number Diff line number Diff line change
Expand Up @@ -23,6 +23,34 @@ import (
"github.com/mjacobs/agy-reader/internal/daemon"
)

// Capture publishes a provisional transcript only in an empty destination.
func Capture(path string, t *daemon.Trajectory, id, reason string, stamp time.Time) (bool, error) {
if t == nil || t.FromCache || id == "" || id != t.CascadeID || reason == "" {
return false, errors.New("cannot capture unidentified presentation")
}
var top map[string]json.RawMessage
if err := json.Unmarshal(t.RawJSON, &top); err != nil {
return false, err
}
if top == nil {
return false, errors.New("missing presentation")
}
reader, err := json.Marshal(map[string]any{"completeness": map[string]string{"state": "unverified", "reason": reason}})
if err != nil {
return false, err
}
top["agyReader"] = reader
data, err := json.MarshalIndent(top, "", " ")
if err != nil {
return false, err
}
err = publishAtomic(path, append(data, '\n'), stamp, os.Chtimes, os.Link)
if errors.Is(err, fs.ErrExist) {
return false, nil
}
return err == nil, err
}

// Write atomically writes a trajectory to sidecarPath. Parent dir must
// already exist (we don't create directories outside the supplied path).
//
Expand Down Expand Up @@ -219,6 +247,11 @@ func setParentCascadeID(sidecarPath, parentCascadeID string) (changed bool, err
if err != nil {
return false, err
}
// A provisional file may be promoted concurrently. Never republish its
// bytes from this read-modify-write path.
if a := completeness.ReadAttestation(data); a != nil && a.State == "unverified" {
return false, nil
}
var top map[string]json.RawMessage
if err := json.Unmarshal(data, &top); err != nil {
return false, fmt.Errorf("unmarshal sidecar for parent stamp: %w", err)
Expand Down Expand Up @@ -333,10 +366,15 @@ func writeAtomicWithStamp(path string, data []byte, mtime time.Time, stamp func(
return false, false, fmt.Errorf("read existing sidecar: %w", readErr)
}

err = publishAtomic(path, data, mtime, stamp, os.Rename)
return err == nil, err == nil, err
}

func publishAtomic(path string, data []byte, mtime time.Time, stamp func(string, time.Time, time.Time) error, publish func(string, string) error) error {
dir := filepath.Dir(path)
tmp, err := os.CreateTemp(dir, ".trajectory-*.json.tmp")
if err != nil {
return false, false, fmt.Errorf("create temp sidecar: %w", err)
return fmt.Errorf("create temp sidecar: %w", err)
}
tmpPath := tmp.Name()
cleanup := func() {
Expand All @@ -345,27 +383,28 @@ func writeAtomicWithStamp(path string, data []byte, mtime time.Time, stamp func(
}
if err := tmp.Chmod(0o600); err != nil {
cleanup()
return false, false, fmt.Errorf("chmod temp sidecar: %w", err)
return fmt.Errorf("chmod temp sidecar: %w", err)
}
if _, err := tmp.Write(data); err != nil {
cleanup()
return false, false, fmt.Errorf("write temp sidecar: %w", err)
return fmt.Errorf("write temp sidecar: %w", err)
}
if err := stamp(tmpPath, mtime, mtime); err != nil {
cleanup()
return false, false, fmt.Errorf("stamp temp sidecar mtime: %w", err)
return fmt.Errorf("stamp temp sidecar mtime: %w", err)
}
if err := tmp.Sync(); err != nil {
cleanup()
return false, false, fmt.Errorf("sync temp sidecar: %w", err)
return fmt.Errorf("sync temp sidecar: %w", err)
}
if err := tmp.Close(); err != nil {
_ = os.Remove(tmpPath)
return false, false, fmt.Errorf("close temp sidecar: %w", err)
return fmt.Errorf("close temp sidecar: %w", err)
}
if err := os.Rename(tmpPath, path); err != nil {
if err := publish(tmpPath, path); err != nil {
_ = os.Remove(tmpPath)
return false, false, fmt.Errorf("rename sidecar: %w", err)
return fmt.Errorf("publish sidecar: %w", err)
}
return true, true, nil
_ = os.Remove(tmpPath)
return nil
}
26 changes: 26 additions & 0 deletions internal/cache/sidecar_test.go
Original file line number Diff line number Diff line change
Expand Up @@ -476,3 +476,29 @@ func TestWriteVerifiedDoesNotRewindMtimeForPermissionOnlyRepair(t *testing.T) {
t.Fatalf("mtime = %v, want the existing newer mtime %v", info.ModTime(), newer)
}
}

func TestConcurrentCaptureKeepsVerifiedPublication(t *testing.T) {
p := filepath.Join(t.TempDir(), "a.trajectory.json")
traj := &daemon.Trajectory{CascadeID: "a", RawJSON: []byte(`{"cascadeId":"a","steps":[]}`)}
stamp := time.Now()
done := make(chan error)
go func() { _, err := cache.Capture(p, traj, "a", "unsupported projection", stamp); done <- err }()
verified := []byte("verified payload")
tmp := p + ".verified"
if err := os.WriteFile(tmp, verified, 0600); err != nil {
t.Fatal(err)
}
if err := os.Rename(tmp, p); err != nil {
t.Fatal(err)
}
if err := <-done; err != nil {
t.Fatal(err)
}
if created, err := cache.Capture(p, traj, "a", "unsupported projection", stamp); err != nil || created {
t.Fatal(created, err)
}
got, err := os.ReadFile(p)
if err != nil || !bytes.Equal(got, verified) {
t.Fatal(string(got), err)
}
}
15 changes: 14 additions & 1 deletion internal/completeness/apply_overlays.py
Original file line number Diff line number Diff line change
Expand Up @@ -262,6 +262,19 @@ class OverlayStep:
compressed_bytes=138810,
gzip_os=3,
),
OverlayStep(
name="policy1510",
module_name="generate_policy1510",
snapshot_path="artifacts/proto-snapshots/agy-1.5.10-20d6f74672ce/closure.protoset",
snapshot_kind="protoset",
snapshot_sha256="5e2adf25951e87fe151b3fd097ab2c772990ecfb167f3b2025cab40119aae1b1",
binary_sha256=None,
base_raw_sha256="f142a51963d6af1df06072e4c35d92be4a75dce1c545eeb421a7d4837b9483a2",
raw_sha256="07b862d0d5e8a4ed511f2d3e4d703148f87e2b29c72d32d055259b0a7316fd11",
compressed_sha256="d4d50760ceaff4e64e79c14d1b3f37ea8f5a8559574bbdddfd556c212e15fa23",
compressed_bytes=138900,
gzip_os=3,
),
)


Expand Down Expand Up @@ -478,7 +491,7 @@ def strip_step(step: OverlayStep, bundle):
require(actual_in == step.raw_sha256, f"{step.name}: strip input hash mismatch")
stripped = descriptor_pb2.FileDescriptorSet()
stripped.CopyFrom(bundle)
if step.name in ("tools158", "workflow159", "schedule1510"):
if step.name in ("tools158", "workflow159", "schedule1510", "policy1510"):
module = importlib.import_module(step.module_name)
stripped = module.strip(stripped)
elif step.name in _STRIPPERS:
Expand Down
Loading
Loading