Skip to content

Commit 2107320

Browse files
committed
fix: harden sync resume tokens and occ lookups
1 parent 7c23d9e commit 2107320

19 files changed

Lines changed: 317 additions & 30 deletions

‎CHANGELOG.md‎

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -1,6 +1,6 @@
11
# CHANGELOG
22

3-
> Summary index — current release details are packaged in [changelogs/v2.0.7.md](./changelogs/v2.0.7.md); historical details live in the repository changelog archive.
3+
> Summary index — current v2.0.7 release-candidate details are packaged in [changelogs/v2.0.7.md](./changelogs/v2.0.7.md); historical details live in the repository changelog archive.
44
> **Last updated**: 2026-06-17
55
66
---
@@ -9,7 +9,7 @@
99

1010
| Version | Date | Summary | Details |
1111
|---------|------|---------|---------|
12-
| [v2.0.7](./changelogs/v2.0.7.md) | 2026-06-16 | Patch: release-readiness alignment plus core correctness fixes for model OCC, transaction cache invalidation, sync ordering, soft-delete reads, driver option forwarding, distributed invalidation, query bounds, and aggregate write-cache invalidation | [View](./changelogs/v2.0.7.md) |
12+
| [v2.0.7](./changelogs/v2.0.7.md) | 2026-06-16 | Patch candidate: release-readiness alignment plus core correctness fixes for model OCC, transaction cache invalidation, sync ordering, soft-delete reads, driver option forwarding, distributed invalidation, query bounds, and aggregate write-cache invalidation | [View](./changelogs/v2.0.7.md) |
1313
| [v2.0.6](./changelogs/v2.0.6.md) | 2026-06-15 | Patch: dependency alignment to `schema-dsl@2.0.11` so downstream frameworks inherit the ESM/CJS shared custom type registry fix | [View](./changelogs/v2.0.6.md) |
1414
| [v2.0.5](./changelogs/v2.0.5.md) | 2026-06-13 | Patch: Model schema adapter delegates DSL type authority to `schema-dsl@2.0.10`, removing the duplicated monSQLize allowlist while preserving legacy aliases and business literals | [View](./changelogs/v2.0.5.md) |
1515
| [v2.0.4](./changelogs/v2.0.4.md) | 2026-06-12 | Patch: production-safe Model index rollout controls, `schema-dsl@2.0.9`, capability-index wording cleanup, and documentation home refinements | [View](./changelogs/v2.0.4.md) |

‎changelogs/README.md‎

Lines changed: 5 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -10,7 +10,7 @@
1010
changelogs/
1111
├── README.md # 本说明文档
1212
├── TEMPLATE.md # 变更文档模板
13-
├── v2.0.7.md # v2.0.7 详细变更
13+
├── v2.0.7.md # v2.0.7 待发布详细变更
1414
├── v2.0.6.md # v2.0.6 详细变更
1515
├── v2.0.5.md # v2.0.5 详细变更
1616
├── v2.0.4.md # v2.0.4 详细变更
@@ -50,7 +50,8 @@ changelogs/
5050

5151
### 按版本类型
5252

53-
- **正式发布**: v2.0.7, v2.0.6, v2.0.5, v2.0.4, v2.0.3, v2.0.2, v2.0.1, v2.0.0, v1.0.0
53+
- **待发布**: v2.0.7
54+
- **正式发布**: v2.0.6, v2.0.5, v2.0.4, v2.0.3, v2.0.2, v2.0.1, v2.0.0, v1.0.0
5455
- **功能版本**: v1.x
5556
- **修复版本**: v1.x.y
5657

@@ -157,7 +158,7 @@ git push origin main --tags
157158

158159
| 版本 | 文件 | 状态 | 发布日期 |
159160
|------|------|------|---------|
160-
| v2.0.7 | v2.0.7.md | ✅ 已发布 | 2026-06-16 |
161+
| v2.0.7 | v2.0.7.md | ⏳ 待发布 | 2026-06-16 |
161162
| v2.0.6 | v2.0.6.md | ✅ 已发布 | 2026-06-15 |
162163
| v2.0.5 | v2.0.5.md | ✅ 已发布 | 2026-06-13 |
163164
| v2.0.4 | v2.0.4.md | ✅ 已发布 | 2026-06-12 |
@@ -170,6 +171,6 @@ git push origin main --tags
170171
---
171172

172173
**目录版本**: 2.0
173-
**最后更新**: 2026-06-16
174+
**最后更新**: 2026-06-17
174175
**维护者**: monSQLize Team
175176

‎changelogs/unreleased.md‎

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -1,6 +1,7 @@
11
# Unreleased
22

33
- Added automatic optimistic locking for versioned Model single-document writes, including automatic direct-`_id` version lookup, explicit `expectedVersion` overrides, versioned `save()` replacement guards, and configurable `updateMany` version modes.
4+
- Forwarded transaction/read options such as `session` into automatic model OCC version pre-reads, hardened file-backed Change Stream resume tokens with atomic replacement plus strict load validation, and made unexpected Change Stream closes visible through `isRunning: false` / `lastError`.
45
- Awaited async query-cache reads/writes so Redis/MultiLevel cache backends no longer make first cached reads resolve to `undefined`, and made Change Stream resume-token saves strict by default with retry knobs plus explicit `strictSave: false` legacy mode.
56
- Fixed `findPage` cursor anchors for nested dot-path sort fields, accepted `project` as a query projection alias across read helpers, and documented process-level Model registration plus ObjectId `maxDepth` conversion boundaries.
67
- Fixed `incrementOne` driver-option forwarding, closed runtime-owned Redis cache adapters on `runtime.close()`, made SSH tunnels fail fast for multi-host/SRV MongoDB URIs and post-ready disconnects, enforced slow-query batch `maxBufferSize` during in-flight flushes, wired prewarmed `findPage` bookmarks into page-jump reads, and reduced redundant rebuilds in the memory-server validation matrix.

‎changelogs/v2.0.7.md‎

Lines changed: 5 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -1,6 +1,7 @@
11
# monSQLize v2.0.7
22

3-
> Release date: 2026-06-16
3+
> Prepared date: 2026-06-16
4+
> Release status: Prepared, not yet published
45
> Type: Patch
56
> Compatibility: Backward compatible
67
@@ -15,9 +16,10 @@ v2.0.7 closes the release-readiness gap after the v2.0.6 review and hardens the
1516
- `npm run test:unit` now delegates to `test/run-tests.cjs unit`, matching the maintained unit suite instead of a stale hand-written file list.
1617
- Validation ledgers now reflect the current 56 runnable TypeScript documentation examples.
1718
- `initializeModelV1Methods()` now reports factory failures through the runtime logger when available instead of writing directly to `console.warn`.
18-
- Versioned models now enforce true single-document optimistic concurrency control: callers must provide the expected version in the filter or `expectedVersion`, stale writes throw `WRITE_CONFLICT`, and `updateMany()` is rejected for versioned models.
19+
- Versioned models now enforce true optimistic concurrency control for single-document writes: direct `_id` filters automatically read the current version, callers may override with `expectedVersion` / `version`, and stale writes throw `WRITE_CONFLICT`.
20+
- Versioned `updateMany()` now supports explicit modes: `counter` for native batch version counters, `strict` for per-document conditional updates, and `off` for compatibility escape hatches.
1921
- Transaction cache invalidations are recorded during the transaction and replayed only after a successful commit; commit retry now handles `UnknownTransactionCommitResult`.
20-
- Change Stream sync events are processed serially so target writes and resume token persistence stay ordered.
22+
- Change Stream sync events are processed serially so target writes and resume token persistence stay ordered; resume token files use atomic replacement, strict load validation, and backup files in file mode; unexpected stream close events now mark sync as stopped in stats.
2123
- Soft-delete filters now cover the standard model read surface, including `findPage`, ID reads, `distinct`, `aggregate`, `stream`, and `explain`.
2224
- MongoDB read paths now forward driver options such as `session`, `readConcern`, `readPreference`, `collation`, `hint`, `maxTimeMS`, and aggregation options instead of dropping them through a narrow whitelist.
2325
- Query caches now avoid session-scoped reads and build stable cache keys only from result-shaping options.

‎docs/en/connection.md‎

Lines changed: 2 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -782,6 +782,7 @@ const msq = new MonSQLize({
782782
// Resume Token configuration.
783783
storage: 'file',
784784
path: './.sync-resume-token',
785+
strictLoad: true,
785786
strictSave: true
786787
}
787788
}
@@ -837,6 +838,7 @@ const msq = new MonSQLize({
837838
| `slowQueryLog` | object | - | Persistent slow-query log storage. |
838839
| `models` | object | - | Model auto-loading configuration. |
839840
| `sync` | object | - | Change Stream synchronization. |
841+
| `sync.resumeToken.strictLoad` | boolean | same as `strictSave` | When true, unreadable or corrupted stored tokens stop Change Stream startup instead of starting from no token. |
840842
| `sync.resumeToken.strictSave` | boolean | `true` | When true, token save failures stop Change Stream sync before the resume token can advance. |
841843
| `sync.resumeToken.saveRetries` | number | `0` | Retry attempts before a strict token save fails. |
842844
| `sync.resumeToken.saveRetryDelayMs` | number | `100` | Delay between token-save retries. |

‎docs/en/release-preflight.md‎

Lines changed: 6 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -13,9 +13,12 @@ You can also manually trigger the **Release Preflight** workflow through GitHub
1313
1. Check whether `package-lock.json` is consistent with the current `package.json` version and does not contain `file:` / `workspace:` / sibling local path residue.
1414
2. Check whether the changelog file corresponding to the current `package.json` version exists.
1515
3. Check whether the necessary documents for project management exist:
16-
- `docs/support-matrix.md`
17-
- `docs/file-dependency-governance.md`
18-
- `docs/verification-entrypoints.md`
16+
- `docs/en/support-matrix.md`
17+
- `docs/en/file-dependency-governance.md`
18+
- `docs/en/verification-entrypoints.md`
19+
- `docs/zh/support-matrix.md`
20+
- `docs/zh/file-dependency-governance.md`
21+
- `docs/zh/verification-entrypoints.md`
1922
4. Run `npm run verify:fast`
2023
5. Run `npm test`
2124
6. Run `npm pack --dry-run`

‎docs/en/sync-backup.md‎

Lines changed: 7 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -120,11 +120,12 @@ await msq.close();
120120
| `storage` | string | ❌ | `'file'` | Storage type: `'file'` or `'redis'` |
121121
| `path` | string | ❌ | `./.sync-resume-token` | File path (file mode) |
122122
| `redis` | Object | ❌ | - | Redis instance (Redis mode) |
123+
| `strictLoad` | boolean | ❌ | same as `strictSave` | Treat unreadable or corrupted stored tokens as fatal instead of starting from no token |
123124
| `strictSave` | boolean | ❌ | `true` | Treat token save failures as fatal so CDC never advances without a persisted token |
124125
| `saveRetries` | number | ❌ | `0` | Retry attempts before a strict token save fails |
125126
| `saveRetryDelayMs` | number | ❌ | `100` | Delay between token-save retries |
126127

127-
Resume token persistence is strict by default. After all eligible targets apply an event successfully, monSQLize saves the event resume token; `syncedCount` advances only after that save succeeds. If token persistence fails after the configured retries, the manager records `tokenSaveErrorCount` / `lastTokenSaveError`, closes the live stream, marks `isRunning: false`, and does not process later queued events. Set `strictSave: false` only for legacy best-effort behavior, where a restart may replay already-applied events. Built-in MongoDB targets are idempotent (`replaceOne(..., { upsert: true })` / `deleteOne()`); custom `apply` targets should still deduplicate by change event `_id`.
128+
Resume token persistence is strict by default. File storage writes to a same-directory temporary file, fsyncs it, keeps the previous token as `<path>.bak`, and then atomically renames the temporary file into place. Startup also validates the stored token: a corrupted token fails fast when `strictLoad` is true, instead of silently starting without a resume token. After all eligible targets apply an event successfully, monSQLize saves the event resume token; `syncedCount` advances only after that save succeeds. If token persistence fails after the configured retries, the manager records `tokenSaveErrorCount` / `lastTokenSaveError`, closes the live stream, marks `isRunning: false`, and does not process later queued events. Set `strictSave: false` and `strictLoad: false` only for legacy best-effort behavior, where a restart may replay already-applied events or start without the previously stored token. Built-in MongoDB targets are idempotent (`replaceOne(..., { upsert: true })` / `deleteOne()`); custom `apply` targets should still deduplicate by change event `_id`.
128129

129130
---
130131

@@ -241,6 +242,9 @@ console.log(stats);
241242
// errorCount: 4,
242243
// startTime: 2026-01-17T...,
243244
// lastEventTime: 2026-01-17T...,
245+
// lastError: null,
246+
// tokenSaveErrorCount: 0,
247+
// lastTokenSaveError: null,
244248
// targets: [...]
245249
// }
246250
```
@@ -303,13 +307,14 @@ rs.initiate()
303307
**Automatic processing**:
304308
- Failure of a single target does not affect other targets
305309
- A resume-token save failure stops the sync manager before the token can advance
306-
- Change Stream driver errors are logged and reflected in stats; monitor `isRunning` / `errorCount` and restart the manager or runtime from your supervisor when needed
310+
- Change Stream driver errors and unexpected stream closes are logged and reflected in stats; monitor `isRunning`, `errorCount`, and `lastError`, then restart the manager or runtime from your supervisor when needed
307311

308312
**Manual processing**:
309313
```javascript
310314
// View error statistics
311315
const stats = msq.getSyncStats();
312316
console.log(stats.errorCount);
317+
console.log(stats.lastError);
313318
console.log(stats.targets[0].lastError);
314319
```
315320

‎docs/zh/connection.md‎

Lines changed: 2 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -746,6 +746,7 @@ const msq = new MonSQLize({
746746
resumeToken: {
747747
storage: 'file', // 'file' | 'redis'【默认: 'file'】
748748
path: './.sync-resume-token', // 文件模式 token 路径
749+
strictLoad: true, // token 损坏或不可读时停止启动【默认: 同 strictSave】
749750
strictSave: true // token 保存失败时停止同步【默认: true】
750751
}
751752
},
@@ -830,6 +831,7 @@ const msq = new MonSQLize({
830831
| `slowQueryLog` | object | - | 慢查询日志持久化 |
831832
| `models` | object | - | Model 自动加载 |
832833
| `sync` | object | - | Change Stream 同步 |
834+
| `sync.resumeToken.strictLoad` | boolean | 同 `strictSave` | 为 true 时,已保存 token 损坏或不可读会阻止 Change Stream 启动,而不是按无 token 启动 |
833835
| `sync.resumeToken.strictSave` | boolean | `true` | 为 true 时,token 保存失败会在 resume token 推进前停止 Change Stream 同步 |
834836
| `sync.resumeToken.saveRetries` | number | `0` | strict token save 失败前的重试次数 |
835837
| `sync.resumeToken.saveRetryDelayMs` | number | `100` | token 保存重试间隔(毫秒) |

‎docs/zh/release-preflight.md‎

Lines changed: 6 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -13,9 +13,12 @@ npm run release:preflight
1313
1. 检查 `package-lock.json` 是否与当前 `package.json` 版本一致,且不包含 `file:` / `workspace:` / sibling 本地路径残留。
1414
2. 检查当前 `package.json` 版本对应的 changelog 文件是否存在。
1515
3. 检查工程治理必需文档是否存在:
16-
- `docs/support-matrix.md`
17-
- `docs/file-dependency-governance.md`
18-
- `docs/verification-entrypoints.md`
16+
- `docs/en/support-matrix.md`
17+
- `docs/en/file-dependency-governance.md`
18+
- `docs/en/verification-entrypoints.md`
19+
- `docs/zh/support-matrix.md`
20+
- `docs/zh/file-dependency-governance.md`
21+
- `docs/zh/verification-entrypoints.md`
1922
4. 运行 `npm run verify:fast`
2023
5. 运行 `npm test`
2124
6. 运行 `npm pack --dry-run`

‎docs/zh/sync-backup.md‎

Lines changed: 7 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -114,11 +114,12 @@ await msq.close();
114114
| `storage` | string | ❌ | `'file'` | 存储类型:`'file'` 或 `'redis'` |
115115
| `path` | string | ❌ | `./.sync-resume-token` | 文件路径(文件模式) |
116116
| `redis` | Object | ❌ | - | Redis 实例(Redis 模式) |
117+
| `strictLoad` | boolean | ❌ | 同 `strictSave` | 将不可读取或损坏的已保存 token 视为致命错误,而不是按无 token 启动 |
117118
| `strictSave` | boolean | ❌ | `true` | 将 token 保存失败视为致命错误,避免 CDC 在 token 未持久化时继续推进 |
118119
| `saveRetries` | number | ❌ | `0` | strict token save 失败前的重试次数 |
119120
| `saveRetryDelayMs` | number | ❌ | `100` | token 保存重试间隔(毫秒) |
120121

121-
Resume Token 持久化默认是 strict。匹配的 target 全部成功应用事件后,monSQLize 会先保存该事件的 resume token;只有保存成功后才推进 `syncedCount`。如果 token 保存在配置的重试后仍失败,manager 会记录 `tokenSaveErrorCount` / `lastTokenSaveError`,关闭当前 change stream,将 `isRunning` 标记为 `false`,并停止处理后续排队事件。只有兼容旧 best-effort 行为时才设置 `strictSave: false`;此时进程重启后可能重放已应用事件。内置 MongoDB target 是幂等的(`replaceOne(..., { upsert: true })` / `deleteOne()`);自定义 `apply` target 仍建议按 change event `_id` 做幂等或去重。
122+
Resume Token 持久化默认是 strict。文件模式会先写入同目录临时文件、fsync、把上一份 token 备份为 `<path>.bak`,再通过原子 rename 替换正式文件。启动时也会校验已保存 token:`strictLoad` 为 true 时,损坏 token 会快速失败,而不是静默按无 resume token 启动。匹配的 target 全部成功应用事件后,monSQLize 会先保存该事件的 resume token;只有保存成功后才推进 `syncedCount`。如果 token 保存在配置的重试后仍失败,manager 会记录 `tokenSaveErrorCount` / `lastTokenSaveError`,关闭当前 change stream,将 `isRunning` 标记为 `false`,并停止处理后续排队事件。只有兼容旧 best-effort 行为时才设置 `strictSave: false` 与 `strictLoad: false`;此时进程重启后可能重放已应用事件,或在旧 token 损坏时按无 token 启动。内置 MongoDB target 是幂等的(`replaceOne(..., { upsert: true })` / `deleteOne()`);自定义 `apply` target 仍建议按 change event `_id` 做幂等或去重。
122123

123124
---
124125

@@ -230,6 +231,9 @@ console.log(stats);
230231
// errorCount: 4,
231232
// startTime: 2026-01-17T...,
232233
// lastEventTime: 2026-01-17T...,
234+
// lastError: null,
235+
// tokenSaveErrorCount: 0,
236+
// lastTokenSaveError: null,
233237
// targets: [...]
234238
// }
235239
```
@@ -286,13 +290,14 @@ rs.initiate()
286290
**自动处理**:
287291
- 单个目标失败不影响其他目标
288292
- Resume Token 保存失败会在推进 token 前停止 sync manager
289-
- Change Stream driver 错误会记录日志并体现在 stats 中;生产应监控 `isRunning` / `errorCount`,必要时由进程管理器或应用重启 manager/runtime
293+
- Change Stream driver 错误与意外 stream close 会记录日志并体现在 stats 中;生产应监控 `isRunning`、`errorCount` 与 `lastError`,必要时由进程管理器或应用重启 manager/runtime
290294
291295
**手动处理**:
292296
```javascript
293297
// 查看错误统计
294298
const stats = msq.getSyncStats();
295299
console.log(stats.errorCount);
300+
console.log(stats.lastError);
296301
console.log(stats.targets[0].lastError);
297302
```
298303

0 commit comments

Comments
 (0)