Skip to content

fix(automations): Say that send_message is how an automation run posts #2213

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
HazAT merged 1 commit into from
Oct 23, 2026
Merged

fix(automations): Say that send_message is how an automation run posts #2213

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
3 changes: 3 additions & 0 deletions packages/junior-evals/evals/scheduler/README.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -38,6 +38,9 @@ Each failure seen in production has one realistic case:
`evals/integration/scheduler/delivery.eval.ts`.
- "me" does not reach the creator (#2014): the creator mention case in
`evals/integration/scheduler/delivery.eval.ts`.
- A reminder that says "post in this channel" ends as `misconfigured`
because no Slack posting tool exists: the post-in-this-channel case in
`delivery.eval.ts`.
- Unattended runs ask questions, or post when a condition is not met, and the
silence marker leaks into Slack (#2014, #1741): the condition cases in
`evals/integration/coding/event-automations/delivery.eval.ts`.
Expand Down
19 changes: 19 additions & 0 deletions packages/junior-evals/evals/scheduler/delivery.eval.ts
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -74,6 +74,25 @@ describe("Scheduled Delivery", () => {
expect(delivery.replies[0]!.text).not.toMatch(/<@[UW]/);
});

test("when a due reminder says to post in this channel, deliver it instead of looking for a posting tool", async ({
run,
}) => {
await insertScheduledAutomation({
credentialMode: "system",
destination: slackChannel(),
due: true,
task: "Post a reminder in this channel asking everyone to reply in this message's thread with their weekly update. A couple of sentences is enough.",
});

const delivery = await run(heartbeat());

expect(
completedToolCalls("finishAutomationRun", delivery).map(toolOutput),
).toEqual([expect.objectContaining({ result: "send_message" })]);
expect(delivery.replies).toHaveLength(1);
expect(delivery.replies[0]!.text).toMatch(/weekly update/i);
});

test("when a due reminder goes to its creator's DM, deliver the reminder itself", async ({
run,
}) => {
Expand Down
2 changes: 1 addition & 1 deletion packages/junior/src/chat/tools/runtime/finish-automation-run.ts
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -29,7 +29,7 @@ export function createFinishAutomationRunTool(options: {
readOnlyHint: false,
},
description: options.sendsMessage
? `End this automation run. Use \`send_message\` with the finished reminder, digest, alert, or answer; the runtime posts it to the stored destinations. ${RESULT_GUIDANCE}`
? `End this automation run. Use \`send_message\` with the finished reminder, digest, alert, or answer; the runtime posts it to the stored destinations. This is the only way that the run posts, so a missing Slack posting tool does not make the run \`misconfigured\`. ${RESULT_GUIDANCE}`
: `End this automation run. This automation posts nothing. ${RESULT_GUIDANCE}`,
inputSchema: z.object({
result: z.enum(results),
Expand Down
Loading