Design: structured prompt builder to replace string template substitution

[zjshen14]

Problem

The current system instruction is assembled by string substitution on a Markdown template (DEFAULT_SYSTEM_INSTRUCTION in src/agent/prompt.ts):

template
  .replaceAll("{CWD}", ctx.cwd)
  .replaceAll("{SESSION_TMP}", ctx.tmpDir)
  .replaceAll("{TOOL_CATALOG}", toolCatalog)
  .replaceAll("{GIT_CONTEXT}", gitContext)

This approach has several limitations:

1. No structure — the prompt is an opaque string. Sections (workflow, tools, git context, mode suffix) can't be individually toggled, reordered, or overridden without editing the raw text.
2. Manual sync burden — buildPlanSuffix has to hardcode the write tool list (write, edit, bash, todo_write). When new write tools are added, it must be updated manually.
3. Hard to extend for new modes — adding a new run mode (e.g. review, debug) means another suffix string concatenated in core.ts.
4. No per-section caching granularity — context.ts caches by tool signature. If only the git context changes (e.g. a commit happens mid-session), the whole instruction is stale but there's no targeted invalidation.
5. Custom system prompts (OPENCLI_SYSTEM_MD) bypass all structure — a file override replaces everything, making it impossible to partially customise (e.g. override just the tone section).

Prior art

Several agent frameworks use a structured prompt model:

• LangChain — ChatPromptTemplate with named slots; sections are SystemMessagePromptTemplate, HumanMessagePromptTemplate, etc. Each section is typed and composable.
• DSPy — prompts are declared as typed signatures (inputs → outputs); the framework assembles and optimises the prompt text automatically.
• Anthropic SDK — no built-in builder, but Claude Code's internal prompt is assembled from a section registry where each section has a name, content() function, and optional condition().
• PromptLayer / Helicone — prompt versioning and A/B testing require structured sections to diff and swap.

Proposed design

A lightweight PromptBuilder in src/agent/prompt.ts — no new dependencies, fits the existing architecture.

Core idea

interface PromptSection {
  id: string;
  content: (ctx: SystemInstructionContext) => string;
  condition?: (ctx: SystemInstructionContext) => boolean;
}

class PromptBuilder {
  private sections: PromptSection[] = [];

  add(section: PromptSection): this { ... }
  remove(id: string): this { ... }
  build(ctx: SystemInstructionContext): string { ... }
}

Default sections

id	Content
identity	"You are OpenCLI, an expert software engineer…"
cwd	Working directory + session tmp
git_context	Repository snapshot (branch, status, recent commits)
workflow	Research → Plan → Execute process
engineering_standards	Code quality rules
tool_usage	Parallelism, targeting, one-edit-per-file
git_rules	Never commit without request, etc.
security	Credential handling, destructive ops
tone	Senior peer, direct, no filler
files	See vs create, scratch dir
tool_catalog	Available tools list (dynamic)

Plan mode

Instead of appending PLAN_SYSTEM_SUFFIX, plan mode replaces or adds sections:

const planBuilder = defaultBuilder()
  .remove("tool_catalog")
  .add({ id: "plan_mode", content: buildPlanModeSection });

Benefits

• Adding a new tool or mode doesn't require editing prompt strings
• Sections can be individually tested
• OPENCLI_SYSTEM_MD could override specific sections by id rather than replacing everything
• Natural foundation for future prompt versioning / A/B testing

Scope

This is a refactor of src/agent/prompt.ts and src/agent/context.ts. The LLMClient interface, tool system, and REPL are unaffected. The external behaviour (what the model receives) should be identical after the change — tests should pass without modification except for the renderSystemInstruction unit tests which would be replaced by builder tests.

References

• Follows from the prompt assembly centralisation in ce0a636
• Related: src/agent/prompt.ts, src/agent/context.ts, src/agent/core.ts