Skip to content

Commit c3a0ffb

Browse files
committed
chore(db): regenerate the sync lock token migration with drizzle-kit
The migration was hand-written, which left it inconsistent with every other migration in the repo and, more importantly, without a schema snapshot. Drizzle diffs against the latest snapshot to decide what a migration needs to contain, so the next generate would have seen the column as still missing and emitted it a second time. Regenerated properly: drizzle-kit now owns the SQL, the journal entry, and 0297_snapshot.json. The emitted statement matches the house pattern for an additive nullable column, and check:migrations still reports backward-compatible.
1 parent 3c0a07b commit c3a0ffb

3 files changed

Lines changed: 31 additions & 33 deletions

File tree

apps/sim/lib/knowledge/connectors/sync-engine.test.ts

Lines changed: 13 additions & 14 deletions
Original file line numberDiff line numberDiff line change
@@ -1518,7 +1518,7 @@ describe('writeTerminalConnectorState', () => {
15181518
})
15191519
})
15201520

1521-
describe('applySupersededOutcome', () => {
1521+
describe('markSyncSuperseded', () => {
15221522
const result = {
15231523
docsAdded: 3,
15241524
docsUpdated: 1,
@@ -1527,29 +1527,28 @@ describe('applySupersededOutcome', () => {
15271527
docsFailed: 0,
15281528
}
15291529

1530-
it('leaves a run that kept its lock untouched', async () => {
1531-
const { applySupersededOutcome } = await import('@/lib/knowledge/connectors/sync-engine')
1532-
1533-
expect(applySupersededOutcome(result, true)).toEqual(result)
1534-
})
1535-
15361530
it('flags a discarded run so the task wrapper does not report it as clean', async () => {
1537-
const { applySupersededOutcome, SUPERSEDED_SYNC_ERROR } = await import(
1531+
const { markSyncSuperseded, SUPERSEDED_SYNC_ERROR } = await import(
15381532
'@/lib/knowledge/connectors/sync-engine'
15391533
)
15401534

1541-
const superseded = applySupersededOutcome(result, false)
1542-
15431535
// The task wrapper reports `success: !result.error`.
1544-
expect(superseded.error).toBe(SUPERSEDED_SYNC_ERROR)
1545-
expect(Boolean(superseded.error)).toBe(true)
1536+
expect(markSyncSuperseded(result).error).toBe(SUPERSEDED_SYNC_ERROR)
15461537
})
15471538

15481539
it('preserves the document counters of the discarded run', async () => {
1549-
const { applySupersededOutcome } = await import('@/lib/knowledge/connectors/sync-engine')
1540+
const { markSyncSuperseded } = await import('@/lib/knowledge/connectors/sync-engine')
15501541

15511542
// Those writes landed — only the connector-level bookkeeping was discarded.
1552-
expect(applySupersededOutcome(result, false)).toMatchObject(result)
1543+
expect(markSyncSuperseded(result)).toMatchObject(result)
1544+
})
1545+
1546+
it('does not mutate the result it was handed', async () => {
1547+
const { markSyncSuperseded } = await import('@/lib/knowledge/connectors/sync-engine')
1548+
1549+
markSyncSuperseded(result)
1550+
1551+
expect(result).not.toHaveProperty('error')
15531552
})
15541553
})
15551554

apps/sim/lib/knowledge/connectors/sync-engine.ts

Lines changed: 16 additions & 17 deletions
Original file line numberDiff line numberDiff line change
@@ -477,9 +477,11 @@ export async function completeSyncLog(
477477
* proves the lock is still this run's. `status` is kept alongside as defence in
478478
* depth and to cover a user pausing the connector mid-run.
479479
*
480-
* Guards both terminal paths. The failure path needs it as much as the success
481-
* path: a reclaimed run's failure would double-increment a counter the sweep
482-
* already advanced and overwrite its backoff with a shorter one.
480+
* Guards every write a run makes to its own connector row: both terminal paths
481+
* and the mid-run heartbeat. The failure path needs it as much as the success
482+
* path — a reclaimed run's failure would double-increment a counter the sweep
483+
* already advanced and overwrite its backoff with a shorter one — and reusing it
484+
* for the heartbeat is what turns a beat into an ownership probe.
483485
*/
484486
export function stillHoldsSyncLock(connectorId: string, syncLockToken: string) {
485487
return and(
@@ -509,7 +511,7 @@ export function buildSyncLockAcquisition(syncLogId: string, now: Date) {
509511
/**
510512
* Whether a running sync is due to refresh its lock.
511513
*
512-
* Time-based rather than batch-count-based: batches vary hugely in cost, so a
514+
* Time-based rather than batch-count-based: batches vary hugely in cost, so an
513515
* every-N-batches beat would fire constantly on small documents and barely at
514516
* all on large ones — exactly the runs that need it.
515517
*/
@@ -570,9 +572,10 @@ export async function writeTerminalConnectorState(
570572
}
571573

572574
/**
573-
* Reported when a run's terminal write matched no rows because the run no longer
574-
* held its lock. Its document writes still landed; only its connector-level
575-
* bookkeeping was discarded, in favour of whoever reclaimed the row.
575+
* Reported when a run loses its connector's lock mid-flight — either because a
576+
* heartbeat found the lock reclaimed, or because its terminal write matched no
577+
* rows. Its document writes still landed; only its connector-level bookkeeping
578+
* was discarded, in favour of whoever reclaimed the row.
576579
*/
577580
export const SUPERSEDED_SYNC_ERROR = 'sync_superseded'
578581

@@ -581,11 +584,7 @@ export const SUPERSEDED_SYNC_ERROR = 'sync_superseded'
581584
* report a discarded run as a clean sync — the same reason a lock-contended run
582585
* returns `sync_in_progress` rather than an empty success.
583586
*/
584-
export function applySupersededOutcome(
585-
result: SyncResult,
586-
terminalWriteLanded: boolean
587-
): SyncResult {
588-
if (terminalWriteLanded) return result
587+
export function markSyncSuperseded(result: SyncResult): SyncResult {
589588
return { ...result, error: SUPERSEDED_SYNC_ERROR }
590589
}
591590

@@ -708,8 +707,8 @@ export function classifySuspectListing(
708707
* immediately. A genuinely emptied source keeps reconciling: its second sync
709708
* corroborates the first and tombstones everything, and a later sync — once the
710709
* tombstoned set is again absent — completes the two-strike purge, subject to
711-
* {@link capReconciliationDeletions}, which holds a pass whose deletion count
712-
* exceeds the per-sync blast-radius cap.
710+
* {@link capReconciliationDeletions}, which withholds any generation whose
711+
* deletion count exceeds the per-sync blast-radius cap.
713712
*
714713
* A forced `fullSync` overrides the guard, matching its existing meaning
715714
* elsewhere here — an explicit human request to reconcile against this listing
@@ -2155,7 +2154,7 @@ export async function executeSync(
21552154
syncLogId,
21562155
...result,
21572156
})
2158-
return applySupersededOutcome(result, false)
2157+
return markSyncSuperseded(result)
21592158
}
21602159

21612160
logger.info('Sync completed', { connectorId, ...result })
@@ -2172,7 +2171,7 @@ export async function executeSync(
21722171
syncLogId,
21732172
...result,
21742173
})
2175-
return applySupersededOutcome(result, false)
2174+
return markSyncSuperseded(result)
21762175
}
21772176

21782177
if (error instanceof ConnectorDeletedException) {
@@ -2230,7 +2229,7 @@ export async function executeSync(
22302229
)
22312230

22322231
/**
2233-
* Deliberately does NOT get {@link applySupersededOutcome}. `result.error`
2232+
* Deliberately does NOT get {@link markSyncSuperseded}. `result.error`
22342233
* is set to the real failure cause below and the task wrapper already
22352234
* reports this run as unsuccessful, so overwriting it with
22362235
* `sync_superseded` would destroy the diagnostic without changing the

apps/sim/lib/knowledge/connectors/sync-limits.ts

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -38,8 +38,8 @@ export const CONNECTOR_SYNC_STALE_LOCK_TTL_MS = CONNECTOR_SYNC_MAX_DURATION_SECO
3838
*
3939
* Shared because two independent writers advance this counter: `executeSync`'s
4040
* in-process failure path, and the scheduler's out-of-process stale-lock
41-
* reclaim (a SIGKILL skips `catch`/`finally`, so only the reaper ever sees that
42-
* failure). A connector that only ever dies hard must still reach the threshold,
41+
* reclaim (a SIGKILL unwinds nothing, so the in-process `catch` never runs and
42+
* only the reaper ever sees that failure). A connector that only ever dies hard must still reach the threshold,
4343
* which it cannot if the two disagree on what the threshold is.
4444
*/
4545
export const MAX_CONSECUTIVE_FAILURES = 10

0 commit comments

Comments
 (0)