Skip to content

fix(init): wire Custom provider through OpenAI-compatible runtime path (closes #83)#93

Open
initializ-mk wants to merge 1 commit into
mainfrom
fix/issue-83-custom-provider-wiring
Open

fix(init): wire Custom provider through OpenAI-compatible runtime path (closes #83)#93
initializ-mk wants to merge 1 commit into
mainfrom
fix/issue-83-custom-provider-wiring

Conversation

@initializ-mk
Copy link
Copy Markdown
Contributor

Summary

Closes #83.

The wizard's Custom provider option is documented as "point Forge at any OpenAI-compatible endpoint" (OpenRouter, litellm, vLLM, self-hosted Kimi/Llama, etc.). It collected the right inputs but wrote artifacts the runtime could not consume:

  1. forge.yaml got provider: custom, which the LLM client factory rejects (forge-core/llm/providers/factory.go has no case "custom":). The runner falls back to StubExecutor and every task fails with agent execution not configured for framework "forge".
  2. .env got MODEL_BASE_URL / MODEL_API_KEY, which ResolveModelConfig never reads — only per-provider names (OPENAI_BASE_URL, etc.) are consulted.
  3. Even after hand-fixing (1) and (2), stored ChatGPT OAuth credentials silently override an explicit OPENAI_BASE_URLcreateProviderClient replaces cfg.BaseURL with chatgpt.com/backend-api/codex. ChatGPT then 400s on the operator's model, with no signal that the endpoint was hijacked.

Fix

Wire 1+2 — forge-cli/cmd/init.go

New normalizeCustomProvider() called at the top of scaffold(). Single entry point covers both the TUI wizard and the Web UI POST. Rewrites ModelProvider=customopenai and migrates MODEL_BASE_URL / MODEL_API_KEYOPENAI_BASE_URL / OPENAI_API_KEY.

Handles all three shapes that may arrive:

  • Legacy MODEL_* env vars (current TUI wizard, older Web UI).
  • New OPENAI_* env vars written directly (post-fix Web UI).
  • Non-interactive --api-key flag where opts.APIKey is set but no env var is.

buildEnvVars extended to emit OPENAI_BASE_URL when set; dead case "custom": removed.

OAuth-precedence guardrail — forge-cli/runtime/runner.go + forge-cli/cmd/ui.go

When provider == "openai" AND cfg.BaseURL != "" AND cfg.APIKey == "", refuse rather than fall through to OAuth. Error explicitly names:

  • the configured base URL
  • the missing env var (OPENAI_API_KEY)
  • the override (chatgpt.com/backend-api/codex) that would otherwise occur silently

Applied to both the agent runtime (runner.go) and the skill builder chat handler (cmd/ui.go). Anthropic / Gemini / Ollama paths unaffected.

Web UI — forge-ui/handlers_create.go + types.go + static/app.js

Wizard metadata advertises BaseURLEnv: \"OPENAI_BASE_URL\" (was \"MODEL_BASE_URL\"); the React form binds to OPENAI_BASE_URL directly. POSTs whose ModelProvider == \"custom\" still work — they hit the same scaffold() entry point and get normalized identically.

Regression tests

10 new tests, all -race green:

forge-cli/cmd/init_custom_provider_test.go (6 tests)

  • TestNormalizeCustomProvider_RewritesLegacyEnvVars — old MODEL_* shape → OPENAI_*
  • TestNormalizeCustomProvider_NoOpForOtherProviders — table across openai/anthropic/gemini/ollama
  • TestNormalizeCustomProvider_PreExistingOpenAIVarsPreserved — newer Web UI shape isn't clobbered
  • TestNormalizeCustomProvider_APIKeyFallsBackToOptsField — non-interactive --api-key path
  • TestScaffold_CustomProviderProducesOpenAIShape — end-to-end; pins forge.yaml + .env content; asserts no MODEL_* leak
  • TestScaffold_CustomProviderWebUIShape — parity with Web UI POST shape

forge-cli/runtime/runner_oauth_guardrail_test.go (4 tests)

  • TestCreateProviderClient_BaseURLSetWithoutAPIKey_RefusesOAuth — pins the guardrail + verifies the error message mentions OPENAI_BASE_URL, the configured URL, and OPENAI_API_KEY
  • TestCreateProviderClient_BaseURLSetWithAPIKey_BypassesOAuth — happy path still works
  • TestCreateProviderClient_NoBaseURL_AllowsOAuthPath — existing openai.com OAuth use unaffected
  • TestCreateProviderClient_AnthropicWithBaseURL_Unaffected — guardrail scoped to openai

Gates

  • go test -race -count=1 ./forge-cli/cmd/... ./forge-cli/runtime/... ./forge-ui/... — green
  • golangci-lint run — 0 issues across forge-cli, forge-ui, forge-core
  • gofmt -l forge-cli/ forge-ui/ forge-core/ — clean

Migration

CHANGELOG "Unreleased" section calls out the rename for users with checked-in provider: custom configs:

-provider: custom
+provider: openai
-MODEL_BASE_URL=...
-MODEL_API_KEY=...
+OPENAI_BASE_URL=...
+OPENAI_API_KEY=...

Test plan

  • go test -race -count=1 ./forge-cli/cmd/... ./forge-cli/runtime/... ./forge-ui/... green
  • golangci-lint run clean across the three modules
  • gofmt -l clean
  • Manual: TUI Custom flow writes provider: openai + OPENAI_BASE_URL/OPENAI_API_KEY (pinned by TestScaffold_CustomProviderProducesOpenAIShape)
  • Manual: Web UI Custom flow writes the same shape (pinned by TestScaffold_CustomProviderWebUIShape)
  • Manual: OAuth-credentials + OPENAI_BASE_URL without OPENAI_API_KEY → clear error, no silent ChatGPT redirect (pinned by TestCreateProviderClient_BaseURLSetWithoutAPIKey_RefusesOAuth)
  • End-to-end smoke against a real OpenAI-compatible endpoint (operator-side; the reporter's eurl config against OpenRouter / Kimi)

…ath (closes #83)

The wizard's Custom provider option is documented as "point Forge at any
OpenAI-compatible endpoint" (OpenRouter, litellm, vLLM, self-hosted Kimi/
Llama, etc.). It collected the right inputs but wrote artifacts the runtime
could not consume:

  1. forge.yaml got `provider: custom`, which the LLM client factory
     (forge-core/llm/providers/factory.go) rejects with "unknown LLM
     provider". Runner falls back to StubExecutor; every task fails with
     'agent execution not configured for framework "forge"'.

  2. .env got MODEL_BASE_URL / MODEL_API_KEY, which forge-core/runtime/
     config.go's resolveAPIKey() never reads (only per-provider names
     like OPENAI_BASE_URL / OPENAI_API_KEY are consulted).

  3. Even when users worked around (1) and (2) by hand-editing forge.yaml
     to `provider: openai` + writing OPENAI_BASE_URL, stored ChatGPT
     OAuth credentials silently won over the explicit base URL — the
     runner's createProviderClient overrides cfg.BaseURL with
     chatgpt.com/backend-api/codex. ChatGPT then returns 400 rejecting
     the operator's model, with no signal that the endpoint was hijacked.

Fix at three boundaries:

- forge-cli/cmd/init.go: new normalizeCustomProvider() called at the top
  of scaffold() (single entry point for both TUI and Web UI). Rewrites
  ModelProvider=custom -> openai and migrates MODEL_BASE_URL /
  MODEL_API_KEY -> OPENAI_BASE_URL / OPENAI_API_KEY. Handles all three
  shapes that may arrive: legacy MODEL_* env vars, newer Web UI direct
  OPENAI_* env vars, and the non-interactive --api-key flag path.
  buildEnvVars's "openai" case now emits OPENAI_BASE_URL when set; the
  dead "custom" case is removed.

- forge-cli/runtime/runner.go + forge-cli/cmd/ui.go: OAuth-precedence
  guardrail. When provider="openai" + cfg.BaseURL != "" + cfg.APIKey ==
  "", refuse rather than fall through to OAuth. Error names the
  configured base URL, the missing env var (OPENAI_API_KEY), and the
  override (chatgpt.com/backend-api/codex) that would otherwise occur.
  Anthropic and other providers unaffected.

- forge-ui/handlers_create.go + types.go + static/app.js: wizard
  metadata advertises BaseURLEnv="OPENAI_BASE_URL" (not MODEL_BASE_URL);
  the React form binds to OPENAI_BASE_URL directly. POSTs whose
  ModelProvider="custom" still work — they hit the same scaffold()
  entry point and get the same normalization.

Regression tests:

  forge-cli/cmd/init_custom_provider_test.go (6 tests):
    TestNormalizeCustomProvider_RewritesLegacyEnvVars
    TestNormalizeCustomProvider_NoOpForOtherProviders
    TestNormalizeCustomProvider_PreExistingOpenAIVarsPreserved
    TestNormalizeCustomProvider_APIKeyFallsBackToOptsField
    TestScaffold_CustomProviderProducesOpenAIShape
    TestScaffold_CustomProviderWebUIShape

  forge-cli/runtime/runner_oauth_guardrail_test.go (4 tests):
    TestCreateProviderClient_BaseURLSetWithoutAPIKey_RefusesOAuth
    TestCreateProviderClient_BaseURLSetWithAPIKey_BypassesOAuth
    TestCreateProviderClient_NoBaseURL_AllowsOAuthPath
    TestCreateProviderClient_AnthropicWithBaseURL_Unaffected

go test -race ./forge-cli/cmd/... ./forge-cli/runtime/... ./forge-ui/...
golangci-lint and gofmt clean across all three modules.

Migration note in CHANGELOG for users with checked-in `provider: custom`
forge.yaml files (manual rename to provider: openai + .env key rename).
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

forge init 'Custom' provider produces forge.yaml the runtime cannot consume (custom URL never reaches LLM client)

1 participant