fix(trigger): let workers see that Trigger.dev is available

Workers run Trigger.dev by definition, but the flag saying so is read from the
environment and had only ever been set on the app container. isTriggerAvailable()
was therefore false inside every task run, so work a task dispatched silently
took the in-process fallback instead of the queue it was written for.

Document processing is where this showed: a connector sync chunked and embedded
its documents itself, five at a time, rather than handing them to the
document-processing queue. The queue's concurrency limit, the per-document task's
machine, retry policy and duration budget all sat unused, and a sync with
thousands of documents ran until it hit its own max duration. It also explains
why that task has no runs for connector-synced knowledge bases at all.

Asserting the flag here is safe because the same check still requires
TRIGGER_SECRET_KEY, which only the Trigger.dev runtime provides: anywhere
dispatching is not actually possible the flag stays ineffective and behaviour is
unchanged.

Dispatch failure is now recoverable rather than silent. Only a total failure
raised before, so one failed batch left its documents at pending with nothing
recording why. Those are processed in-process instead, which costs the caller
the time it hoped to hand to the queue but does not drop the work. That path was
unreachable from a worker until this change made dispatching happen there.

Author: waleedlatif1

apps/sim/lib/knowledge/documents/service.ts

@@ -721,6 +721,7 @@ async function dispatchViaBatchTrigger(
 ): Promise<number> {
   let dispatched = 0
   const batchIds: string[] = []
+  const undispatched: DocumentProcessingPayload[] = []
   const region = await resolveTriggerRegion()
   for (let i = 0; i < jobPayloads.length; i += TRIGGER_BATCH_SIZE) {
     const chunk = jobPayloads.slice(i, i + TRIGGER_BATCH_SIZE)
@@ -747,11 +748,27 @@ async function dispatchViaBatchTrigger(
       logger.error(`[${requestId}] Failed to batchTrigger ${chunk.length} document jobs`, {
         error: getErrorMessage(error),
       })
+      undispatched.push(...chunk)
     }
   }
   if (batchIds.length > 0) {
     logger.info(`[${requestId}] Trigger.dev batches dispatched`, { batchIds })
   }
+
+  /**
+   * Only a total dispatch failure raises, so a chunk that failed on its own used
+   * to leave its documents sitting at `pending` with nothing recording why —
+   * invisible until the stuck-document sweep happened to pick them up, and never
+   * if they aged out of its window first. Running them here costs the caller time
+   * it hoped to hand to the queue, which is the point: the work still happens.
+   */
+  if (undispatched.length > 0) {
+    logger.warn(
+      `[${requestId}] Processing ${undispatched.length} documents in-process after failed enqueue`
+    )
+    dispatched += await dispatchInProcess(undispatched, requestId)
+  }
+
   return dispatched
 }
 

apps/sim/trigger.config.ts

@@ -73,7 +73,23 @@ export default defineConfig({
       '@daytona/sdk',
     ],
     extensions: [
-      syncEnvVars(() => [{ name: 'DB_APP_NAME', value: 'sim-trigger' }]),
+      syncEnvVars(() => [
+        { name: 'DB_APP_NAME', value: 'sim-trigger' },
+        /**
+         * Workers run Trigger.dev by definition, but the flag that says so is
+         * read from the environment and was only ever set on the app container.
+         * `isTriggerAvailable()` therefore returned false inside every worker, so
+         * anything a task dispatched — document processing above all — silently
+         * took the in-process path instead of the queue it was written for. A
+         * connector sync ended up chunking and embedding thousands of documents
+         * itself, five at a time, and running until it hit its max duration.
+         *
+         * Safe to assert here because the check also requires TRIGGER_SECRET_KEY,
+         * which only the Trigger.dev runtime provides: where dispatching is not
+         * actually possible this stays false and nothing changes.
+         */
+        { name: 'TRIGGER_DEV_ENABLED', value: 'TRUE' },
+      ]),
       additionalFiles({
         files: [
           './lib/execution/isolated-vm-worker.cjs',