Skip to content

fix(v3): remove executing tasks waiting to deploy #4432

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
deepshekhardas wants to merge 2 commits into from
Closed

fix(v3): remove executing tasks waiting to deploy #4432

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
2 commits
Select commit Hold shift + click to select a range
f6f7225
fix(v3): remove executing tasks waiting to deploy
Jul 11, 2026
634ec91
chore: add changeset for v3 executing tasks fix
Apr 3, 2026
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
7 changes: 7 additions & 0 deletions .changeset/fix-v3-executing-tasks.md
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,7 @@
---
"@trigger.dev/core": patch
---

fix(v3): remove executing tasks waiting to deploy

Removes the executing tasks that are waiting to deploy from the task list. This fixes an issue where tasks in EXECUTING state but waiting for deployment were incorrectly displayed.
Comment on lines +1 to +7

@devin-ai-integration devin-ai-integration Bot Jul 31, 2026

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟡 Release-notes entry claims a published package changed when only server code did

A changeset targeting the published core package was added ("@trigger.dev/core": patch in .changeset/fix-v3-executing-tasks.md:2) even though the PR only touches server code, so an unrelated package gets a version bump and a misleading release note.
Impact: Users see a package release describing a change that isn't in that package, and the note is written in internal implementation terms rather than user-facing behaviour.

Repository rule for server-only changes

AGENTS.md and CONTRIBUTING.md state: when modifying only server components (apps/webapp/, apps/supervisor/, etc.) with no package changes, add a .server-changes/ file (with area/type frontmatter) instead of a changeset. They also require the description be written for users, not maintainers, and never name internal tooling/infra — the current text ("fix(v3): remove executing tasks waiting to deploy", "tasks in EXECUTING state but waiting for deployment") is implementation-oriented and inaccurate.

Prompt for agents 
This PR changes only apps/webapp. Per AGENTS.md and CONTRIBUTING.md, delete .changeset/fix-v3-executing-tasks.md and add a .server-changes/ markdown file instead, with frontmatter `area: webapp` and `type: fix`, and a single plain user-facing sentence describing the behaviour change (no internal service names).
Open in Devin Review

Was this helpful? React with 👍 or 👎 to provide feedback.

6 changes: 1 addition & 5 deletions apps/webapp/app/services/worker.server.ts
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -41,7 +41,6 @@ import {
} from "~/v3/services/cancelDevSessionRuns.server";
import { CancelTaskAttemptDependenciesService } from "~/v3/services/cancelTaskAttemptDependencies.server";
import { EnqueueDelayedRunService } from "~/v3/services/enqueueDelayedRun.server";
import { ExecuteTasksWaitingForDeployService } from "~/v3/services/executeTasksWaitingForDeploy";
import { ExpireEnqueuedRunService } from "~/v3/services/expireEnqueuedRun.server";
import { ResumeBatchRunService } from "~/v3/services/resumeBatchRun.server";
import { ResumeTaskDependencyService } from "~/v3/services/resumeTaskDependency.server";
Expand Down Expand Up @@ -220,9 +219,6 @@ function getWorkerQueue() {
priority: 0,
maxAttempts: 5,
handler: async (payload, job) => {
const service = new ExecuteTasksWaitingForDeployService();

return await service.call(payload.backgroundWorkerId);
},
},
// @deprecated, moved to ScheduleEngine
Expand Down Expand Up @@ -288,7 +284,7 @@ function getWorkerQueue() {
"v3.requeueTaskRun": {
priority: 0,
maxAttempts: 3,
handler: async (payload, job) => {}, // This is now handled by redisWorker
handler: async (payload, job) => { }, // This is now handled by redisWorker

@devin-ai-integration devin-ai-integration Bot Jul 31, 2026

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟡 Formatting change would fail the automated code-quality check

An empty function body was reformatted with a space inside the braces (async (payload, job) => { } at apps/webapp/app/services/worker.server.ts:287), which the repository's auto-formatter would rewrite, so the CI formatting check fails.
Impact: The pull request cannot pass CI until formatting is re-run.

Rule reference

AGENTS.md / CONTRIBUTING.md require running pnpm run format (oxfmt) before pushing; the code-quality CI check fails if it produces a diff. The unrelated whitespace change from {} to { } reverts formatter output.

Suggested change
handler: async (payload, job) => { }, // This is now handled by redisWorker
handler: async (payload, job) => {}, // This is now handled by redisWorker
Open in Devin Review

Was this helpful? React with 👍 or 👎 to provide feedback.

},
// @deprecated, moved to commonWorker.server.ts
"v3.retryAttempt": {
Expand Down
3 changes: 0 additions & 3 deletions apps/webapp/app/v3/commonWorker.server.ts
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -20,7 +20,6 @@ import { BatchTriggerV3Service } from "./services/batchTriggerV3.server";
import { CancelDevSessionRunsService } from "./services/cancelDevSessionRuns.server";
import { CancelTaskAttemptDependenciesService } from "./services/cancelTaskAttemptDependencies.server";
import { EnqueueDelayedRunService } from "./services/enqueueDelayedRun.server";
import { ExecuteTasksWaitingForDeployService } from "./services/executeTasksWaitingForDeploy";
import { ExpireEnqueuedRunService } from "./services/expireEnqueuedRun.server";
import { ResumeBatchRunService } from "./services/resumeBatchRun.server";
import { ResumeTaskDependencyService } from "./services/resumeTaskDependency.server";
Expand Down Expand Up @@ -252,8 +251,6 @@ function initializeWorker() {
await service.call(payload.deploymentId, payload.fromStatus, payload.errorMessage);
},
"v3.executeTasksWaitingForDeploy": async ({ payload }) => {
const service = new ExecuteTasksWaitingForDeployService();
await service.call(payload.backgroundWorkerId);
},
Comment on lines 253 to 254

@devin-ai-integration devin-ai-integration Bot Jul 31, 2026

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🔴 Runs held back until a deploy exists are never started again

The step that releases runs waiting on a deployment has been emptied out ("v3.executeTasksWaitingForDeploy" handler at apps/webapp/app/v3/commonWorker.server.ts:253-254) while runs are still being put into that waiting state, so those runs sit there forever and never execute.
Impact: On legacy-engine projects, any run triggered before its task was deployed stays stuck in "waiting for deploy" indefinitely, even after a successful deploy or promotion.

How waiting runs lose their only drain path

apps/webapp/app/v3/marqs/sharedQueueConsumer.server.ts:1435-1446 still transitions runs to WAITING_FOR_DEPLOY. Previously, ExecuteTasksWaitingForDeployService flipped those runs back to PENDING and re-enqueued them in MarQS, and it was triggered from three places:

  • deploy completion (ExecuteTasksWaitingForDeployService.enqueue(backgroundWorker.id) removed at apps/webapp/app/v3/services/createDeploymentBackgroundWorkerV3.server.ts:211),
  • promotion/rollback (removed at apps/webapp/app/v3/services/changeCurrentDeployment.server.ts:176),
  • the worker jobs in commonWorker.server.ts / apps/webapp/app/services/worker.server.ts:218-223.

This PR deletes the service file and empties all three, so nothing transitions WAITING_FOR_DEPLOY runs out of that state. Any jobs already enqueued in Redis for v3.executeTasksWaitingForDeploy also become no-ops.

Prompt for agents 
The PR deletes apps/webapp/app/v3/services/executeTasksWaitingForDeploy.ts and empties the v3.executeTasksWaitingForDeploy handlers in commonWorker.server.ts and services/worker.server.ts, and removes the enqueue calls in createDeploymentBackgroundWorkerV3.server.ts and changeCurrentDeployment.server.ts. However SharedQueueConsumer (#markRunAsWaitingForDeploy in apps/webapp/app/v3/marqs/sharedQueueConsumer.server.ts) still sets legacy V1 runs to WAITING_FOR_DEPLOY, and no other code transitions them out. Either restore the drain service and its enqueue call sites, or remove the code path that puts runs into WAITING_FOR_DEPLOY in the first place so no run can become permanently stuck.
Open in Devin Review

Was this helpful? React with 👍 or 👎 to provide feedback.

"v3.retryAttempt": async ({ payload }) => {
const service = new RetryAttemptService();
Expand Down
12 changes: 0 additions & 12 deletions apps/webapp/app/v3/services/changeCurrentDeployment.server.ts
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -10,7 +10,6 @@ import {
import { taskMetadataCacheInstance } from "~/services/taskMetadataCacheInstance.server";
import { BaseService, ServiceValidationError } from "./baseService.server";
import { syncDeclarativeSchedules } from "./createBackgroundWorker.server";
import { ExecuteTasksWaitingForDeployService } from "./executeTasksWaitingForDeploy";
import { compareDeploymentVersions } from "../utils/deploymentVersions";

export type ChangeCurrentDeploymentDirection = "promote" | "rollback";
Expand Down Expand Up @@ -175,17 +174,6 @@ export class ChangeCurrentDeploymentService extends BaseService {
});
}

// Only V1 engine workers need the WAITING_FOR_DEPLOY drain — V2 runs sit
// in PENDING_VERSION and are handled out of band, so enqueuing here for V2
// just produces empty scans of the TaskRun status index.
const worker = await this._prisma.backgroundWorker.findFirst({
where: { id: deployment.workerId },
select: { engine: true },
});

if (worker?.engine === "V1") {
await ExecuteTasksWaitingForDeployService.enqueue(deployment.workerId);
}
}

async #syncSchedulesForDeployment(deployment: WorkerDeployment) {
Expand Down
2 changes: 0 additions & 2 deletions apps/webapp/app/v3/services/createDeploymentBackgroundWorkerV3.server.ts
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -15,7 +15,6 @@ import {
stripBackgroundWorkerMetadataForStorage,
syncDeclarativeSchedules,
} from "./createBackgroundWorker.server";
import { ExecuteTasksWaitingForDeployService } from "./executeTasksWaitingForDeploy";
import { projectPubSub } from "./projectPubSub.server";
import { TimeoutDeploymentService } from "./timeoutDeployment.server";
import { CURRENT_DEPLOYMENT_LABEL, BackgroundWorkerId } from "@trigger.dev/core/v3/isomorphic";
Expand Down Expand Up @@ -209,7 +208,6 @@ export class CreateDeploymentBackgroundWorkerServiceV3 extends BaseService {
});
}

await ExecuteTasksWaitingForDeployService.enqueue(backgroundWorker.id);
await PerformDeploymentAlertsService.enqueue(deployment.id);
await TimeoutDeploymentService.dequeue(deployment.id, this._prisma);

Expand Down
142 changes: 0 additions & 142 deletions apps/webapp/app/v3/services/executeTasksWaitingForDeploy.ts
View file
Open in desktop

This file was deleted.