Found during an adversarial dogfooding pass on main @ 6a40258 (release binary, Linux).
Repro
d=$(mktemp -d)
for i in $(seq 1 12); do
( APS_HOME=$d aps set secret "s$i" >/dev/null 2>&1 || echo "set s$i FAILED rc=$?" ) &
done
wait
APS_HOME=$d aps get secret
Observed across repeated runs (12 concurrent first-use writers each):
- Trial A: 5 of 12 sets failed with exit 69 (
secret_unlock_failed)
- Trial B: 1 of 12 sets failed with exit 73 (
persistence_failed)
- Roughly 1 in 3 trials shows at least one failure; single-process use never fails
Why
On a fresh state root, every racer lands in SecretStore.loadOrCreateKeyFile(): no secret.key yet, so each process generates its own X25519 key and writes the file non-atomically (Data.write(to:) without .atomic; mode 0600 is applied only after the write). Meanwhile set() (#89) unlocks any existing envelope before rewriting and read-back verifies with whatever secret.key is current at that moment. Peers can swap the key file between a writer's envelope write and its read-back, or between another writer's unlock and re-seal, producing the spurious 69/73 failures.
Worst case (the interleaving exists; not yet hit in 10x16 races): one process writes its envelope last while a peer's key-file write lands last, leaving secret.enc undecryptable until aps reset secret.
Side note: because secret.key is created with default umask and chmod 0600 happens after the write, there is a brief window where the private key file is group/world readable.
Suggestions
- Write
secret.key atomically (temp file + rename) with restrictive permissions at creation time (O_EXCL/mode 0600), and
- Serialize first-use key creation across processes (the
SchemaFileLock pattern already exists; a per-store lock file would do), or tolerate-and-retry on the first-use path.
This matters for the repo's own multi-agent workflow: AGENTS.md has agents share ~/.aps and encourages parallel key operations, so two agents setting secrets concurrently in a fresh state root can hit this.
Found during an adversarial dogfooding pass on main @ 6a40258 (release binary, Linux).
Repro
Observed across repeated runs (12 concurrent first-use writers each):
secret_unlock_failed)persistence_failed)Why
On a fresh state root, every racer lands in
SecretStore.loadOrCreateKeyFile(): nosecret.keyyet, so each process generates its own X25519 key and writes the file non-atomically (Data.write(to:)without.atomic; mode 0600 is applied only after the write). Meanwhileset()(#89) unlocks any existing envelope before rewriting and read-back verifies with whateversecret.keyis current at that moment. Peers can swap the key file between a writer's envelope write and its read-back, or between another writer's unlock and re-seal, producing the spurious 69/73 failures.Worst case (the interleaving exists; not yet hit in 10x16 races): one process writes its envelope last while a peer's key-file write lands last, leaving
secret.encundecryptable untilaps reset secret.Side note: because
secret.keyis created with default umask and chmod 0600 happens after the write, there is a brief window where the private key file is group/world readable.Suggestions
secret.keyatomically (temp file + rename) with restrictive permissions at creation time (O_EXCL/mode 0600), andSchemaFileLockpattern already exists; a per-store lock file would do), or tolerate-and-retry on the first-use path.This matters for the repo's own multi-agent workflow: AGENTS.md has agents share
~/.apsand encourages parallel key operations, so two agents setting secrets concurrently in a fresh state root can hit this.