Skip to content

fix(store): make request-log purge drop child rows for real - #462

Merged
caigee-cmd merged 1 commit into
caigee-cmd:mainfrom
NaimioChan:fix/log-retention-cascade
Nov 3, 2026
Merged

caigee-cmd merged 1 commit into
caigee-cmd:mainfrom
NaimioChan:fix/log-retention-cascade

Conversation

@NaimioChan

Copy link
Copy Markdown
Contributor

问题

请求日志的保留期清理(PurgeRequestLogs,每小时一次)只删 request_logs 主表,靠 ON DELETE CASCADE 带走子表。但级联其实从来没生效过。

PRAGMA foreign_keys 是连接级设置,不是数据库文件属性。OpenStore 里是这样设的:

db, err := sql.Open("sqlite", path)
db.SetMaxOpenConns(1)
db.Exec("PRAGMA foreign_keys = ON; PRAGMA busy_timeout = 5000;")

这一句 Exec 只配置了 database/sql 当时恰好给它的那一条连接。之后连接因为 idle 回收或连接级错误被丢弃重开时,新连接的 foreign_keys 是 OFF,级联静默失效。

SetMaxOpenConns(1) 挡不住这件事:它限制并发连接数上限,不保证连接不被替换。

实测影响

线上部署(v0.7.26)的库副本:

检查项 结果
request_usage_details 孤儿行(父 log 已不存在) 9249 / 13474 = 69%
request_attempts 孤儿 9463
request_stream_diagnostics 孤儿 6322
最老孤儿 2026-10-20,早于 7 天保留窗口

复现(旧代码,连接回收后):foreign_keys = 0,purge 后孤儿 attempts = 1。

改动

  1. pragma 进 DSN:sqliteDSN() 追加 ?_foreign_keys=on,驱动在每次 open 时应用,新连接也生效。启动时那句 Exec 只留 busy_timeout。
  2. purge 显式删子表:PurgeRequestLogs / ClearRequestLogs 改为在同一个事务里先删三张子表的关联行再删主表。这样保留逻辑完全不依赖连接状态,将来驱动或 DSN 再出岔子也不会留孤儿。
  3. exportWindowHours() 接上 logs.RetentionDays():原来写死 24 * 7,usage.go 的注释自己写着「等 CLI2API_LOG_RETENTION_DAYS 落地后接 RetentionDays()」,但 feat(logs): configurable request-log retention (from #336) #386 合并时没回来改。后果是调大保留期后用量页「费用」选项卡的 from 仍被夹回 7 天,看不到更早的历史。

测试

新增三个测试:

  • TestForeignKeyPragmaSurvivesConnectionRecycle — 连接回收后 PRAGMA foreign_keys 仍为 1。
  • TestPurgeRequestLogsDropsChildRowsWithoutCascade — 强制 foreign_keys = OFF 后 purge,三张子表都不留孤儿(不依赖级联也能清干净)。
  • TestExportWindowHoursFollowsRetention — 默认 7 天、CLI2API_LOG_RETENTION_DAYS=90 时 90 天、非法值回退默认。

前两个在旧代码上会失败,已确认(旧代码 foreign_keys = 0、孤儿 attempts = 1)。

全量:docker run --rm -v .:/src -w /src golang:1.25 go test ./internal/... 全部 ok。Windows 宿主上直接 go test ./... 的 TempDir 清理失败是老问题(SQLite 文件被占用),与本次改动无关。

已存在的孤儿

这次改动只保证今后不再产生孤儿,线上库里的历史残渣(约 2.5 万行)不会自动消失。需要的话可以单独跑一次清理;我没把它做进迁移里,因为要不要动现有数据应当由部署方决定。

PRAGMA foreign_keys is per-connection, not per-database. OpenStore ran it
once through db.Exec, which configured only whichever pooled connection that
Exec happened to take. After an idle recycle or a connection-level error,
database/sql replaced it with a fresh connection where foreign_keys was OFF,
so the ON DELETE CASCADE clauses on request_attempts, request_usage_details
and request_stream_diagnostics silently stopped firing. The hourly purge kept
deleting request_logs rows and leaving the children behind: on a live
deployment 9249 of 13474 request_usage_details rows (69%) were orphans, the
oldest reaching back past the 7-day window.

- Carry the pragma in the DSN (_foreign_keys=on) so the driver applies it at
  open time on every connection, and stop relying on the startup Exec.
- Purge and Clear now delete child rows explicitly inside one transaction,
  so retention no longer depends on connection state at all.
- exportWindowHours follows logs.RetentionDays() instead of a hard-coded 7
  days; a raised CLI2API_LOG_RETENTION_DAYS was previously invisible to the
  Usage cost panel.

Agent: hermes
Verify: docker golang:1.25 go test ./internal/... all ok (Windows host fails on TempDir cleanup only)

@caigee-cmd caigee-cmd left a comment

Copy link
Copy Markdown
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM. DSN 里带 _foreign_keys=on 才真正保连接回收后仍生效,purge/clear 再显式删子表是稳妥的双保险;exportWindowHours 接上 RetentionDays() 也把 #386 遗留补上了。CI 全绿,合入。历史孤儿按你说的先不动,部署方自行清即可。

@caigee-cmd
caigee-cmd merged commit f9b671a into caigee-cmd:main Nov 3, 2026
6 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants