Skip to content

improvement(background): right-size the two knowledge task machines - #7212

Merged
waleedlatif1 merged 1 commit into
stagingfrom
ci/trigger-rightsize
Aug 28, 2026
Merged

improvement(background): right-size the two knowledge task machines#7212
waleedlatif1 merged 1 commit into
stagingfrom
ci/trigger-rightsize

Conversation

@waleedlatif1

Copy link
Copy Markdown
Collaborator

Summary

  • knowledge-connector-sync: large-2xlarge-1x. Peak sampled RSS 2.6 GB and peak 1.4 vCPU, so 8 GB / 4 vCPU keeps ~3x memory and ~2.8x CPU headroom against a preset that was reserving 16 GB.
  • knowledge-process-document: large-1xmedium-2x. Peak sampled RSS 902 MB and peak 1.2 vCPU with no document exceeding 2 GB, so 4 GB / 2 vCPU keeps ~4x memory and ~1.7x CPU headroom. Replaces a stale comment claiming 8 GB was "needed for large PDF processing".
  • Halves the reserved compute for the first task and quarters it for the second. No retry, queue, or concurrency semantics change.

Notes

  • CPU figures are core-normalized. OTel process.cpu.utilization divides by cores available, so the raw gauge is multiplied by each preset's vCPU count before comparing. Neither task loses headroom it was actually using, and neither can be throttled by the smaller preset.
  • Deliberately no outOfMemory escalation on knowledge-connector-sync. An OOM is a SIGKILL, so the run never reaches the terminal write that clears syncLockToken; the escalated attempt would find the row still syncing, fail to acquire the lock, and return a skipReason that classifyConnectorSyncResult maps to skipped — a success outcome that would mask the OOM while syncing nothing. The stale-lock reaper owns that recovery path.
  • Tasks left alone on purpose: schedule-execution and webhook-execution both sit around 65% of their current memory ceiling and are correctly sized; workflow-execution cannot retry (maxAttempts: 1), so dropping it to 2 GB would trade unrecoverable failures for a smaller preset.

Type of Change

  • Improvement

Testing

bun run type-check, bun run lint, check:api-validation:strict, the block-registry audit, and the full check:audits suite (38 audits) all pass. background/knowledge-processing.test.ts and background/knowledge-connector-sync.test.ts pass (35 tests).

Checklist

  • Code follows project style guidelines
  • Self-reviewed my changes
  • Tests added/updated and passing
  • No new warnings introduced
  • I confirm that I have read and agree to the terms outlined in the Contributor License Agreement (CLA)

Both knowledge tasks reserved machine presets well above their measured
ceilings. Sized each from production telemetry on both memory and CPU:

- knowledge-connector-sync: large-2x -> large-1x. Peak sampled RSS 2.6 GB
  and peak 1.4 vCPU, so 8 GB/4 vCPU keeps ~3x memory and ~2.8x CPU headroom
  against a preset that reserved 16 GB.
- knowledge-process-document: large-1x -> medium-2x. Peak sampled RSS 902 MB
  and peak 1.2 vCPU, with no document exceeding 2 GB, so 4 GB/2 vCPU keeps
  ~4x memory and ~1.7x CPU headroom.

CPU figures are core-normalized (OTel process.cpu.utilization divides by
cores available), so neither task loses headroom it was actually using and
neither can be throttled by the smaller preset.

No retry or concurrency semantics change.
@vercel

vercel Bot commented Aug 28, 2026

Copy link
Copy Markdown

The latest updates on your projects. Learn more about Vercel for GitHub.

1 Skipped Deployment
Project Deployment Actions Updated (UTC)
docs Skipped Skipped Aug 28, 2026 8:57am

Request Review

@greptile-apps

greptile-apps Bot commented Aug 28, 2026

Copy link
Copy Markdown
Contributor

Greptile Summary

The PR reduces reserved compute for two knowledge background tasks based on production telemetry while preserving their retry, queue, and concurrency behavior.

  • Changes knowledge-connector-sync from large-2x to large-1x and documents why stale-lock recovery remains responsible for OOM termination.
  • Changes knowledge-process-document from large-1x to medium-2x while retaining escalation to large-2x after an OOM.

Confidence Score: 5/5

The PR appears safe to merge, with no concrete blocking or independently actionable non-blocking issue identified.

The changes only adjust initial machine presets, retain existing retry and queue behavior, preserve document-processing OOM escalation, and remain above the production resource peaks described by the change.

Important Files Changed

Filename Overview
apps/sim/background/knowledge-connector-sync.ts Reduces the task's memory reservation without reducing CPU capacity; no concrete workload was found that violates the new limit.
apps/sim/background/knowledge-processing.ts Reduces the initial machine preset while retaining the larger-machine OOM retry safety net and existing execution semantics.

Reviews (1): Last reviewed commit: "improvement(background): right-size the ..." | Re-trigger Greptile

@waleedlatif1
waleedlatif1 merged commit 9e79dd2 into staging Aug 28, 2026
24 checks passed
@waleedlatif1
waleedlatif1 deleted the ci/trigger-rightsize branch August 28, 2026 08:59
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant