Skip to content

feat: add Agnes AI provider support (text, image, video) - #218

Open
susirial wants to merge 1 commit into
0xsline:mainfrom
susirial:feat/agnes-provider-support
Open

susirial wants to merge 1 commit into
0xsline:mainfrom
susirial:feat/agnes-provider-support

Conversation

@susirial

Copy link
Copy Markdown

Summary

Adds support for three Agnes AI models as configurable providers in OpenChatCut:

Model Type Provider ID Default Model ID
Agnes 3.0 Flash Text / LLM (OpenAI-compatible Chat Completions) agnes agnes-3.0-flash
Agnes Image 2.5 Flash Image generation (OpenAI Images-compatible) agnes-image agnes-image-2.5-flash
Agnes Video 2.5 Flash Video generation (OpenAI Videos-compatible, async) agnes-video agnes-video-2.5-flash

All three share the same base URL (https://apihub.agnes-ai.com/v1) and API key, configurable in Settings → Agent Brain / AI Generation → Image / AI Generation → Video.

Changes

Text (LLM)

  • Registered Agnes as an openai-compatible LLM provider preset in shared/llm-providers.ts (auto-generates LLM_AGNES_API_KEY / LLM_AGNES_BASE_URL / LLM_AGNES_MODEL config keys).
  • Added the three LLM_AGNES_* keys to KEY_NAMES so they can be persisted via the settings API.

Image

  • New callAgnesImageProvider in server/plugins/image-provider-clients.ts — calls /v1/images/generations with size tier (1K/2K/3K/4K), ratio aspect ratio, and optional image[] references (Data URI for img2img). Returns data[0].url or data[0].b64_json.
  • Wired agnes-image into ValidImageRequest, validateImageRequest (max 4 outputs, size tiers, aspect-ratio whitelist), the dispatch branch, and ImagePluginOptions.
  • Added AGNES_IMAGE_* keystore keys, settings vendor page, route option, and caps detection.

Video

  • New generateAgnesVideo in server/plugins/video.ts — POST /v1/videos to create a task, then polls GET /v1/videos/{id}?model_name= until completed. Supports text, keyframe (first/last frame), and reference (up to 5 images + 3 audios) modes. Fixed 720p, 4–12s duration.
  • Wired agnes-video into VideoRequest/ValidVideoRequest, validateAgnes, runVideoOperation dispatch, the resumer provider list, and VideoOptions.
  • Added AGNES_VIDEO_* keystore keys, settings vendor page, route option, and caps detection.

Shared & agent-facing

  • computeCaps() now detects AGNES_IMAGE_API_KEY / AGNES_VIDEO_API_KEY so the settings UI correctly shows configured state at runtime.
  • __CONFIGURED_CAPS__ build snapshot updated in config/vite.config.ts.
  • Added Agnes entries to agent tool schemas (generate-schemas.ts), tool-input mapping (generate-tool-input.ts), capabilities routing (capabilities.ts), skill docs (image-gen/video-gen SKILL.md + new agnes-image.md/agnes-video.md references), and connection probes (key-probes.ts).
  • Fixed Fal model routing: the Fal model catalog now only surfaces in the agent prompt when Fal is the actively routed image/video provider, so it no longer shadows another default such as Agnes.

Verification

  • npx tsc -b --force — passes
  • npm run build — succeeds
  • Settings UI confirmed: Agnes appears in Agent Brain, Image, and Video vendor lists; saved API keys persist and show "configured"
  • curl POST /api/keys with Agnes keys returns source: runtime and caps.image/caps.video: true

Notes

  • package-lock.json was intentionally excluded from this PR (the local npm install only reformatted the lockfile with the installed npm version; no dependency changes).

Add Agnes 3.0 Flash (LLM), Agnes Image 2.5 Flash, and Agnes Video 2.5 Flash
as configurable providers.

Text (LLM):
- Register Agnes as an openai-compatible LLM provider preset
  (base URL https://apihub.agnes-ai.com/v1, model agnes-3.0-flash)
- Add LLM_AGNES_* keys to KEY_NAMES so the API key can be persisted

Image:
- Add callAgnesImageProvider (OpenAI Images-compatible /images/generations)
  supporting text-to-image and image-to-image (size tiers 1K-4K, aspect ratios)
- Wire agnes-image model into validation, dispatch, settings, and caps

Video:
- Add generateAgnesVideo (OpenAI Videos-compatible async: POST /videos then
  GET /videos/{id}?model_name= polling) supporting text, keyframe, and
  reference modes (4-12s, 720p only)
- Wire agnes-video model into validation, dispatch, resumer, settings, and caps

Shared:
- Add AGNES_IMAGE_*/AGNES_VIDEO_* keystore keys and NON_SECRET_NAMES entries
- Update computeCaps() to detect Agnes image/video keys for runtime UI status
- Add vendor icon, settings pages, route selection, and env.example entries
- Extend agent tool schemas, capabilities, skill docs, and connection probes
- Fix Fal model routing so the Fal catalog only surfaces when Fal is the
  active provider (not alongside Agnes or another default)
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant