Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
86 changes: 86 additions & 0 deletions packages/agent-runtime/src/__tests__/call-main-prompt.test.ts
Original file line number Diff line number Diff line change
@@ -0,0 +1,86 @@
import { TEST_USER_ID } from '@codebuff/common/old-constants'
import { createTestAgentRuntimeParams } from '@codebuff/common/testing/fixtures/agent-runtime'
import { getInitialSessionState } from '@codebuff/common/types/session-state'
import { afterEach, describe, expect, it, mock, spyOn } from 'bun:test'

import { callMainPrompt } from '../main-prompt'
import * as agentRegistry from '../templates/agent-registry'

import type { ProjectFileContext } from '@codebuff/common/util/file'

describe('callMainPrompt', () => {
afterEach(() => {
mock.restore()
})

const mockFileContext: ProjectFileContext = {
projectRoot: '/test',
cwd: '/test',
fileTree: [],
fileTokenScores: {},
knowledgeFiles: {},
gitChanges: {
status: '',
diff: '',
diffCached: '',
lastCommitMessages: '',
},
changesSinceLastChat: {},
shellConfigFiles: {},
agentTemplates: {},
customToolDefinitions: {},
systemInfo: {
platform: 'test',
shell: 'test',
nodeVersion: 'test',
arch: 'test',
homedir: '/home/test',
cpus: 1,
chromeAvailable: false,
},
}

it('returns early without calling mainPrompt when agent config validation fails', async () => {
const sentActions: Array<{ type: string }> = []
const sendAction = ({ action }: { action: { type: string } }) => {
sentActions.push(action)
}

Comment on lines +43 to +48

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P1 Badge Exercise validation with an invalid config fixture

This test replaces assembleLocalAgentTemplates through an ES-module spy even though the validation path can be exercised directly by placing an invalid template in mockFileContext.agentTemplates. That unnecessarily couples the regression test to module-binding behavior and conflicts with the repository convention to prefer dependency injection over module mocking; use a real invalid fixture here, or inject the assembler if isolation is required.

AGENTS.md reference: AGENTS.md:L28-L28

Useful? React with 👍 / 👎.

Comment on lines +43 to +48

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P1 Badge Exercise validation with an invalid config fixture

This test replaces assembleLocalAgentTemplates through an ES-module spy even though the validation path can be exercised directly by placing an invalid template in mockFileContext.agentTemplates. That unnecessarily couples the regression test to module-binding behavior and conflicts with the repository convention to prefer dependency injection over module mocking; use a real invalid fixture here, or inject the assembler if isolation is required.

AGENTS.md reference: AGENTS.md:L28-L28

Useful? React with 👍 / 👎.

spyOn(agentRegistry, 'assembleLocalAgentTemplates').mockReturnValue({
agentTemplates: {},
validationErrors: [
{ message: 'bad agent config', agentId: 'test-agent' },
] as any,
})

const sessionState = getInitialSessionState(mockFileContext)
const baseParams = createTestAgentRuntimeParams()

const result = await callMainPrompt({
...baseParams,
promptId: 'test-prompt',
sendAction,
logger: baseParams.logger,
signal: new AbortController().signal,
action: {
type: 'prompt' as const,
prompt: 'Hello',
sessionState,
fingerprintId: 'test',
costMode: 'normal' as const,
promptId: 'test-prompt',
toolResults: [],
},
repoUrl: undefined,
repoId: undefined,
clientSessionId: 'test-session',
userId: TEST_USER_ID,
} as any)

const actionTypes = sentActions.map((a) => a.type)
expect(actionTypes).toContain('prompt-error')
expect(actionTypes).toContain('prompt-response')
expect(actionTypes).not.toContain('response-chunk')
expect(result.output.type).toBe('error')
})
})
21 changes: 20 additions & 1 deletion packages/agent-runtime/src/main-prompt.ts
Original file line number Diff line number Diff line change
Expand Up @@ -171,13 +171,32 @@ export async function callMainPrompt(
assembleLocalAgentTemplates({ fileContext, logger })

if (validationErrors.length > 0) {
const errorMessage = `Invalid agent config: ${validationErrors.map((err) => err.message).join('\n')}`
sendAction({
action: {
type: 'prompt-error',
message: `Invalid agent config: ${validationErrors.map((err) => err.message).join('\n')}`,
message: errorMessage,
userInputId: promptId,
},
})

const errorResult = {
sessionState: action.sessionState,
output: { type: 'error' as const, message: errorMessage },
Comment on lines +183 to +185

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 Badge Preserve the rejected prompt in the returned session state

When validation fails before loopAgentSteps runs, the user's current prompt has not yet been inserted into messageHistory, but this result simply returns the unchanged incoming session. Consequently, an SDK caller that fixes the config and passes this failed result as previousRun loses the original request—for example, a follow-up such as “retry” has no request to retry. The SDK's thrown-error path explicitly adds the prompt when the runtime made no progress (sdk/src/run.ts lines 509-527); this early-return path should preserve it equivalently before sending the terminal state.

Useful? React with 👍 / 👎.

Comment on lines +183 to +185

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 Badge Preserve the rejected prompt in the returned session state

When validation fails before loopAgentSteps runs, the user's current prompt has not yet been inserted into messageHistory, but this result simply returns the unchanged incoming session. Consequently, an SDK caller that fixes the config and passes this failed result as previousRun loses the original request—for example, a follow-up such as “retry” has no request to retry. The SDK's thrown-error path explicitly adds the prompt when the runtime made no progress (sdk/src/run.ts lines 509-527); this early-return path should preserve it equivalently before sending the terminal state.

Useful? React with 👍 / 👎.

}

sendAction({
action: {
type: 'prompt-response',
Comment on lines +188 to +190

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 Badge Wait for the async error event before resolving the run

When local validation fails in CodebuffClient.run and the supported handleEvent callback returns a promise, the preceding prompt-error starts handlePromptResponse, which awaits that callback (sdk/src/run.ts lines 1243-1245), but sendAction does not await the handler. This immediately following prompt-response enters the synchronous response branch and resolves the run first (lines 1256-1287), so await client.run() can return while its error event is still being processed. Emit only one terminal action or serialize these dispatches so asynchronous error handling completes before the run resolves.

Useful? React with 👍 / 👎.

Comment on lines +188 to +190

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 Badge Wait for the async error event before resolving the run

When local validation fails in CodebuffClient.run and the supported handleEvent callback returns a promise, the preceding prompt-error starts handlePromptResponse, which awaits that callback (sdk/src/run.ts lines 1243-1245), but sendAction does not await the handler. This immediately following prompt-response enters the synchronous response branch and resolves the run first (lines 1256-1287), so await client.run() can return while its error event is still being processed. Emit only one terminal action or serialize these dispatches so asynchronous error handling completes before the run resolves.

Useful? React with 👍 / 👎.

promptId,
sessionState: errorResult.sessionState,
toolCalls: [],
toolResults: [],
output: errorResult.output,
},
})

return errorResult
}

sendAction({
Expand Down
Loading