diff --git a/docs/README.md b/docs/README.md index 5d450cf6..e988dac6 100644 --- a/docs/README.md +++ b/docs/README.md @@ -2,6 +2,7 @@ Current maintained [Python API](information/python-api.md). Optional executor context: [metrics diagnostics](information/executor-metrics.md). +Production analysis: [#321 activation and the SQLite index](analysis/planfile-321-activation.md). For maintained documentation navigation, start with [`NAVIGATION.md`](information/navigation.md). The content below is a generated diff --git a/docs/analysis/planfile-321-activation.md b/docs/analysis/planfile-321-activation.md new file mode 100644 index 00000000..9ff453cd --- /dev/null +++ b/docs/analysis/planfile-321-activation.md @@ -0,0 +1,125 @@ +--- +{ + "schema": "wellmanifest.docs/document/v1", + "id": "planfile-321-activation", + "kind": "analysis", + "version": 1, + "title": "Planfile #321 activation and the SQLite index finding", + "status": "implemented", + "owner": "semcod/planfile", + "scope": "repository", + "created": "2026-10-30", + "updated": "2026-10-30", + "review_after": "2026-11-30", + "source_revision": "57d085698cd2a92f6da7bf69a4e6a270571aec0e", + "affected_repositories": [ + "semcod/planfile" + ], + "evidence": [ + "https://github.com/semcod/planfile/pull/321", + "https://github.com/semcod/planfile/issues/317", + "https://github.com/semcod/planfile/issues/313" + ] +} +--- + +# Planfile #321 activation and the SQLite index finding + + +## Question + +Did activating #321 stop the planfile resource problem, and what did? + + +Purpose: record, in the owning repository, what was observed when planfile PR #321 was +activated in production and what actually reduced the resource use. Moved from +an operator-local note; it grants no deployment authority. + + +## Scope + +Production planfile service on 2026-10-30. Covers #321 (connection handling and +retryable store-lock refusals), the SQLite ticket index and the later Compose +deployment. Out of scope: PR #322 (status-position index), whose native +migration is recorded separately in +`/home/tom/.local/state/ecosystem-throughput-20261030/planfile262-live-migration-receipt.json`. + + +## Method + +Live measurements of request latency, container CPU and memory, a canary on a +copy of the data with and without the index, a soak test, and a read of the +index-enable code path. Production was not modified by this analysis. + + +## Evidence + +- Source: #321 merged as 57d0856 by the independent Validator. Wheel 0.1.127, + sha256 612120c33b3eb82544481ccc2c09d6cddc99e72f32ebffa3021bd18c848fc30d + (192 files verified). +- Before: container about 182-190% CPU and 8-10.9 GiB; workers held only 20-25 + open file descriptors and none were SQLite. +- Canary on a copy of the data (isolated port, 2 CPU / 12 GiB): 2,771 reads and + 32 writes, 0 failures, worker RSS flat at 400-630 MiB. +- Activation at about 14:50Z: standalone replacement container, same address, + alias `planfile`, port 18769 and volumes; other clients resumed within about + 90 s. Immediately afterwards CPU and memory were again high (about 184% CPU, + 4.7-5.7 GiB). +- Cause measured on the live service: `GET /tickets?sprint=all&offset=0&limit=1000&view=operational` + took about 35 s and returned 6.1 MB, against 0.05 s for `sprint=current`. + `planfile storage status` reported `"index": "none"`; the index file existed + (487 MB, 20,925 tickets) but was disabled and stale, so #321 had no SQLite + connections to close. +- Canary with the index enabled (`planfile storage index-enable --project /workspace`, + about 125 s): the same call dropped from 24.5 s to 0.01-1.7 s, idle CPU 7.5%. + Soak at about 20 req/s (about 50 times production) plus 33 writes: 3,596 reads, + p50 0.08 s, p95 0.64 s, 7 typed 503 on the full `status=open` list during + repair and 12 background lock timeouts that self-recovered. +- Later: the platform owners replaced the standalone container with the + `subactor-platform` Docker Compose deployment at 16:21Z. The SQLite index was + enabled in production at 17:12:23Z by an actor that was not identified (no + config-change record). Measured afterwards: `sprint=all` 0.15-0.33 s, + container about 29% CPU and 2.2 GiB, 21,016 tickets, index current. + + +## Facts + +The large drop in CPU, memory and read latency followed enabling the SQLite +index, not #321 alone. #321 supplies the connection handling and the retryable +503 for a busy store lock that make the index safe under concurrent writes; the +two work together. What the index change touches: one config key +(`store.storage.index = sqlite` in `.planfile/config.yaml`) and the disposable +projection `.planfile/index/tickets.sqlite3`, rebuilt from the YAML and evidence +journals. Ticket, sprint and evidence YAML, the container, image, network and +volumes are untouched. Undo: `python -m planfile storage index-disable --project /workspace` +(sets the key back to `none`, keeps the database file). + + +## Hypotheses + +- The drop in CPU and memory came from the index being enabled; #321 alone was + not shown to resolve it. Not proven: whether a long-running leak existed. +- The unidentified actor was probably an owner-side operation; no record + confirms it. + + +## Limitations + +- A minutes-long canary cannot prove an hours-long memory leak is gone. +- The actor that enabled the index in production is unknown. +- A previous attempt on 2026-10-29 was refused with + `ticket_index_sources_changed_during_rebuild` under write churn. +- Index freshness and contention were observed at a point in time; current + configuration alone does not prove either. +- The standalone activation script that did the first replacement has been + retired (it refuses a Compose-managed container); its original note is + operator-local and is not reproduced here. + + +## Recommendations + +- Owners decide whether to summarize this in planfile#317 and #313; nothing was + posted there. +- Remove the unused runtime tree + `~/.local/share/planfile/runtime/57d085698cd2-ticket260-wheel0.1.127` when + convenient; production no longer mounts it. diff --git a/project/TICKETS.md b/project/TICKETS.md index b0f5448a..bb6eefe4 100644 --- a/project/TICKETS.md +++ b/project/TICKETS.md @@ -162,4 +162,5 @@ This file indexes governance tickets without taking ownership of | **ticket-258** | [`README.md`](./ticket-258/README.md) | - | - | - | - | - | | **ticket-259** | [`README.md`](./ticket-259/README.md) | - | - | - | - | - | | **ticket-260** | [`README.md`](./ticket-260/README.md) | - | - | - | - | - | +| **ticket-262** | [`README.md`](./ticket-262/README.md) | - | - | - | - | - | diff --git a/project/ticket-263/README.md b/project/ticket-263/README.md new file mode 100644 index 00000000..2fe27b73 --- /dev/null +++ b/project/ticket-263/README.md @@ -0,0 +1,22 @@ +# Ticket 263: Record planfile 321 activation evidence + +- **ID**: ticket-263 +- **Owner**: claude +- **Status**: IN_PROGRESS +- **Workflow state**: EDIT +- **Created**: 2026-10-30 + +## Goal and scope + +SESSION_EXECUTION_AUTHORIZATION: user said "tak, kontynuuj" to recording the planfile #321 activation evidence in the owning repository. Nothing is posted to issues; no deployment action. + +Add docs/analysis/planfile-321-activation.md and link it from docs/README.md. + +## Acceptance criteria + +- [x] AC-01: The analysis exists under docs/analysis/ and is linked from docs/README.md. + +## Tracking boundary + +This directory contains the minimal reviewed intent. Optional participant prose +and raw command logs are not required delivery output. diff --git a/project/ticket-263/intent.json b/project/ticket-263/intent.json new file mode 100644 index 00000000..e649ae2b --- /dev/null +++ b/project/ticket-263/intent.json @@ -0,0 +1,25 @@ +{ + "schema": "new-project.intent/v3", + "ticket": "ticket-263", + "summary": "Record planfile 321 activation evidence", + "workstream": "integration", + "classification": { + "kind": "SERVICE", + "priority": "P2", + "origin": "health" + }, + "allowedPaths": [ + "project/ticket-263/**", + "TODO.md", + "project/TICKETS.md", + "docs/analysis/planfile-321-activation.md", + "docs/README.md" + ], + "forbiddenPaths": [ + "project/ticket-*/user-*.md" + ], + "stacks": [], + "dependsOn": [], + "conflictsWith": [], + "integrationTicket": null +}