feat: add Amazon Bedrock as inference provider#963
feat: add Amazon Bedrock as inference provider#963mvincentbb wants to merge 1 commit intoNVIDIA:mainfrom
Conversation
Add Bedrock's OpenAI-compatible endpoint as a first-class provider,
enabling NemoClaw to run on AWS without GPU infrastructure.
- Add bedrock provider config, endpoint URL, and 7 curated models
- Add bedrock to non-interactive mode and interactive menu
- Skip endpoint validation (Bedrock API key format incompatible with probe)
- Add bedrock case to inference-config.js
Tested models (all support tool calling):
nvidia.nemotron-super-3-120b, nvidia.nemotron-nano-3-30b,
deepseek.v3.2, openai.gpt-oss-120b, mistral.devstral-2-123b,
moonshotai.kimi-k2.5, minimax.minimax-m2.5
Usage:
export NEMOCLAW_PROVIDER=bedrock
export OPENAI_API_KEY=<bedrock-api-key>
export BEDROCK_REGION=us-west-2
nemoclaw onboard --non-interactive
📝 WalkthroughWalkthroughAmazon Bedrock is added as a first-class inference provider to NemoClaw. The integration includes provider configuration in inference-config.js with OpenAI-compatible routing, and extends onboard.js with Bedrock endpoint setup, curated models, and validation bypass logic. Changes
Estimated code review effort🎯 3 (Moderate) | ⏱️ ~20 minutes Poem
🚥 Pre-merge checks | ✅ 4 | ❌ 1❌ Failed checks (1 warning)
✅ Passed checks (4 passed)
✏️ Tip: You can configure your own custom pre-merge checks in the settings. ✨ Finishing Touches🧪 Generate unit tests (beta)
Comment |
![coderabbitai[bot]](https://avatars.githubusercontent.com/in/347564?s=60&v=4){kind=small}
There was a problem hiding this comment.
Actionable comments posted: 2
🤖 Prompt for all review comments with AI agents
Verify each finding against the current code and only fix it if needed.
Inline comments:
In `@bin/lib/onboard.js`:
- Line 50: The BEDROCK_ENDPOINT_URL currently falls back to "us-west-2" when
neither process.env.BEDROCK_REGION nor process.env.AWS_REGION is set, which can
silently misconfigure region-bound Bedrock API keys; update the onboarding logic
around BEDROCK_ENDPOINT_URL to require an explicit region by checking
process.env.BEDROCK_REGION (and optionally process.env.AWS_REGION) and, if
neither is present, abort with a clear error message (throw or process.exit with
logging) instructing the user to set BEDROCK_REGION instead of defaulting to
"us-west-2"; locate the constant BEDROCK_ENDPOINT_URL and the surrounding
initialization code to add this validation and error handling.
- Around line 116-126: The Bedrock provider entry currently sets skipVerify:
true and bypasses model checks; change it to perform the same validation used by
other OpenAI-compatible providers by removing or setting skipVerify to false on
the bedrock provider object and invoking validateOpenAiLikeModel(...) for the
selectedModel (the same call used for OpenAI/Gemini) inside the provider
selection/validation flow rather than breaking out early; also remove the early
break that skips verification in the provider selection loop and add calls to
validateOpenAiLikeModel at the other two places where Bedrock was skipped so
onboarding validates model availability and region/model mismatches up front.
🪄 Autofix (Beta)
Fix all unresolved CodeRabbit comments on this PR:
- Push a commit to this branch (recommended)
- Create a new PR with the fixes
ℹ️ Review info
⚙️ Run configuration
Configuration used: Path: .coderabbit.yaml
Review profile: CHILL
Plan: Pro
Run ID: fa5f150e-6144-4eff-b46f-9a1a5b867aff
📒 Files selected for processing (2)
bin/lib/inference-config.jsbin/lib/onboard.js
| const OPENAI_ENDPOINT_URL = "https://api.openai.com/v1"; | ||
| const ANTHROPIC_ENDPOINT_URL = "https://api.anthropic.com"; | ||
| const GEMINI_ENDPOINT_URL = "https://generativelanguage.googleapis.com/v1beta/openai/"; | ||
| const BEDROCK_ENDPOINT_URL = `https://bedrock-mantle.${process.env.BEDROCK_REGION || process.env.AWS_REGION || "us-west-2"}.api.aws/v1`; |
There was a problem hiding this comment.
🧩 Analysis chain
🏁 Script executed:
# First, let's look at the file structure and locate the file in question
fd -t f "onboard.js" --path bin/lib/Repository: NVIDIA/NemoClaw
Length of output: 292
🏁 Script executed:
# Let's examine the context around line 50 and the BEDROCK_ENDPOINT_URL usage
cat -n bin/lib/onboard.js | sed -n '45,60p'Repository: NVIDIA/NemoClaw
Length of output: 875
🏁 Script executed:
# Search for how BEDROCK_ENDPOINT_URL is used throughout the codebase
rg "BEDROCK_ENDPOINT_URL" -A 3 -B 3Repository: NVIDIA/NemoClaw
Length of output: 1015
🏁 Script executed:
# Look for validation or setup logic related to Bedrock
rg "bedrock|BEDROCK" -i bin/lib/onboard.js | head -30Repository: NVIDIA/NemoClaw
Length of output: 1208
🏁 Script executed:
# Check if there's any Bedrock validation logic in the file
rg "setupNim|setupInference" -A 10 bin/lib/onboard.js | grep -A 10 "bedrock"Repository: NVIDIA/NemoClaw
Length of output: 985
🏁 Script executed:
# Find the setupNim function and the Bedrock validation skip
rg "setupNim" -A 50 bin/lib/onboard.js | grep -B 5 -A 10 "bedrock"Repository: NVIDIA/NemoClaw
Length of output: 845
🏁 Script executed:
# Search for the specific validation skip comment and context
rg "Skip endpoint validation for Bedrock" -B 5 -A 10 bin/lib/onboard.jsRepository: NVIDIA/NemoClaw
Length of output: 800
🏁 Script executed:
# Check if there's any validation or error handling specific to Bedrock
rg "skipVerify|SKIP_VERIFY" -B 3 -A 3 bin/lib/onboard.jsRepository: NVIDIA/NemoClaw
Length of output: 1184
🌐 Web query:
AWS Bedrock Mantle endpoint regional behavior API key region bound
💡 Result:
AWS Bedrock Mantle endpoints (bedrock-mantle.{region}.api.aws) are region-specific, as are bedrock-runtime.{region}.amazonaws.com endpoints. Both support OpenAI-compatible APIs (Responses/Chat Completions) and native Bedrock APIs respectively, with model availability varying by region. Amazon Bedrock API keys (Bearer tokens via AWS_BEARER_TOKEN_BEDROCK) are region-bound for short-term keys: "Can be used only in the AWS Region from which you generated it." Long-term keys (service-specific credentials for bedrock.amazonaws.com) documentation does not explicitly state cross-region restrictions, but examples always use region-specific endpoints matching generation region, and Bedrock service quotas/endpoints are regional. API keys are supported in specific regions listed in docs. To use Mantle or any Bedrock endpoint, generate/use an API key in the target region, as keys inherit regional IAM permissions/session scope. For cross-region, regenerate or use IAM credentials with multi-region policies. AWS recommends short-term keys or IAM for production.
Citations:
- 1: https://docs.aws.amazon.com/bedrock/latest/userguide/endpoints.html
- 2: https://docs.aws.amazon.com/bedrock/latest/userguide/models-endpoint-availability.html
- 3: https://docs.aws.amazon.com/general/latest/gr/bedrock.html
- 4: https://docs.aws.amazon.com/bedrock/latest/userguide/api-keys-supported.html
- 5: https://docs.aws.amazon.com/bedrock/latest/userguide/api-keys.html
- 6: https://docs.aws.amazon.com/bedrock/latest/userguide/bedrock-mantle.html
- 7: https://docs.aws.amazon.com/bedrock/latest/userguide/api-keys-generate.html
Require an explicit Bedrock region instead of defaulting to us-west-2.
Bedrock API keys are region-bound and can only be used in the region where they were generated. Bedrock Mantle endpoints are region-specific (bedrock-mantle.{region}.api.aws). If neither BEDROCK_REGION nor AWS_REGION is set, this silently pins every Bedrock onboarding to Oregon, creating a regional mismatch for users working in other regions. Since endpoint validation is intentionally skipped for Bedrock ("validation happens at inference time"), this misconfiguration won't be caught until the first inference request fails.
🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed.
In `@bin/lib/onboard.js` at line 50, The BEDROCK_ENDPOINT_URL currently falls back
to "us-west-2" when neither process.env.BEDROCK_REGION nor
process.env.AWS_REGION is set, which can silently misconfigure region-bound
Bedrock API keys; update the onboarding logic around BEDROCK_ENDPOINT_URL to
require an explicit region by checking process.env.BEDROCK_REGION (and
optionally process.env.AWS_REGION) and, if neither is present, abort with a
clear error message (throw or process.exit with logging) instructing the user to
set BEDROCK_REGION instead of defaulting to "us-west-2"; locate the constant
BEDROCK_ENDPOINT_URL and the surrounding initialization code to add this
validation and error handling.
| bedrock: { | ||
| label: "Amazon Bedrock (OpenAI-compatible endpoint)", | ||
| providerName: "bedrock", | ||
| providerType: "openai", | ||
| credentialEnv: "OPENAI_API_KEY", | ||
| endpointUrl: BEDROCK_ENDPOINT_URL, | ||
| helpUrl: "https://docs.aws.amazon.com/bedrock/latest/userguide/api-keys-generate.html", | ||
| modelMode: "curated", | ||
| defaultModel: "nvidia.nemotron-super-3-120b", | ||
| skipVerify: true, | ||
| }, |
There was a problem hiding this comment.
🧩 Analysis chain
🏁 Script executed:
# Check the file exists and get its size
wc -l bin/lib/onboard.jsRepository: NVIDIA/NemoClaw
Length of output: 82
🏁 Script executed:
# First, let's examine the Bedrock configuration around lines 116-126
sed -n '110,135p' bin/lib/onboard.jsRepository: NVIDIA/NemoClaw
Length of output: 707
🏁 Script executed:
# Search for early breaks related to Bedrock
rg -n "bedrock" bin/lib/onboard.js -A 5 -B 5 | head -100Repository: NVIDIA/NemoClaw
Length of output: 4269
🏁 Script executed:
# Search for validateOpenAiLikeModel function
rg -n "validateOpenAiLikeModel" bin/lib/onboard.jsRepository: NVIDIA/NemoClaw
Length of output: 248
🏁 Script executed:
# Search for --no-verify flag
rg -n "\-\-no-verify|skipVerify" bin/lib/onboard.jsRepository: NVIDIA/NemoClaw
Length of output: 446
🏁 Script executed:
# Examine validateOpenAiLikeModel function
sed -n '942,1000p' bin/lib/onboard.jsRepository: NVIDIA/NemoClaw
Length of output: 1934
🏁 Script executed:
# Check the validation loop context around line 1750 for other providers
sed -n '1740,1790p' bin/lib/onboard.jsRepository: NVIDIA/NemoClaw
Length of output: 2392
🏁 Script executed:
# Check what other providers use skipVerify and how they're handled
sed -n '1700,1800p' bin/lib/onboard.jsRepository: NVIDIA/NemoClaw
Length of output: 4424
🏁 Script executed:
# Check the setupInference flow for bedrock around lines 2025-2040
sed -n '2020,2065p' bin/lib/onboard.jsRepository: NVIDIA/NemoClaw
Length of output: 2556
🌐 Web query:
AWS Bedrock Mantle OpenAI compatible /v1/models endpoint API
💡 Result:
Yes, AWS Bedrock's Mantle inference engine provides OpenAI-compatible API endpoints, including the /v1/models endpoint. This allows listing available models using the OpenAI Models API. Key details: - Endpoint base URL: https://bedrock-mantle.{region}.api.aws/v1 (e.g., us-east-1, us-east-2). - Authentication: Bearer token with Amazon Bedrock API key (via OPENAI_API_KEY env var). - Usage with OpenAI SDK: Set OPENAI_BASE_URL to the Mantle endpoint and use client.models.list. - HTTP example: GET $OPENAI_BASE_URL/v1/models with Authorization: Bearer $OPENAI_API_KEY. - Supported APIs: Models, Responses (/v1/responses), Chat Completions (/v1/chat/completions). - Models: Includes openai.gpt-oss-120b and others; check via the endpoint for current list. - Purpose: Migrate OpenAI apps easily; supports stateful conversations, streaming, tools. Official docs confirm full OpenAI compatibility for these endpoints on bedrock-mantle.
Citations:
- 1: https://docs.aws.amazon.com/bedrock/latest/userguide/bedrock-mantle.html
- 2: https://docs.aws.amazon.com/bedrock/latest/userguide/endpoints.html
- 3: https://medium.com/@mattgillard/getting-started-with-amazon-bedrock-mantle-openai-compatible-apis-on-aws-17cb8a9f2b9d
- 4: https://docs.aws.amazon.com/bedrock/latest/userguide/build.html
🌐 Web query:
AWS Bedrock Mantle REST API GET /v1/models authentication
💡 Result:
The AWS Bedrock Mantle REST API (OpenAI-compatible endpoints at bedrock-mantle.{region}.api.aws/v1) uses Bearer token authentication for the GET /v1/models endpoint. Authentication methods: - Amazon Bedrock API key: Set as OPENAI_API_KEY environment variable. Include in header: Authorization: Bearer $OPENAI_API_KEY - AWS credentials (SigV4): Generate short-term bearer token using AWS SDK or tools like aws-bedrock-token-generator, then use Authorization: Bearer $TOKEN Example curl request: curl -X GET $OPENAI_BASE_URL/v1/models \ -H "Authorization: Bearer $OPENAI_API_KEY" This lists available models powered by Mantle. For production, prefer short-term AWS credentials over long-term API keys.
Citations:
- 1: https://docs.aws.amazon.com/bedrock/latest/userguide/bedrock-mantle.html
- 2: https://medium.com/@mattgillard/getting-started-with-amazon-bedrock-mantle-openai-compatible-apis-on-aws-17cb8a9f2b9d
Bedrock onboarding should validate model availability like other OpenAI-compatible providers.
Bedrock currently skips model validation during onboarding due to skipVerify: true at line 116-126 and an early break at line 1778-1783. However, AWS Bedrock Mantle exposes an OpenAI-compatible /v1/models endpoint that supports Bearer token authentication with the same OPENAI_API_KEY credential. The code's comment claiming the API key format is incompatible with standard OpenAI probe requests does not align with Bedrock Mantle's documented OpenAI-compatible API. Bedrock should reuse validateOpenAiLikeModel() (like OpenAI and Gemini providers do) to validate model availability before leaving the selection loop, catching invalid model IDs or region/model mismatch issues at onboarding time rather than at inference.
Also applies to: 1778-1783, 2025-2038
🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed.
In `@bin/lib/onboard.js` around lines 116 - 126, The Bedrock provider entry
currently sets skipVerify: true and bypasses model checks; change it to perform
the same validation used by other OpenAI-compatible providers by removing or
setting skipVerify to false on the bedrock provider object and invoking
validateOpenAiLikeModel(...) for the selectedModel (the same call used for
OpenAI/Gemini) inside the provider selection/validation flow rather than
breaking out early; also remove the early break that skips verification in the
provider selection loop and add calls to validateOpenAiLikeModel at the other
two places where Bedrock was skipped so onboarding validates model availability
and region/model mismatches up front.
1 participant
Add Bedrock's OpenAI-compatible endpoint as a first-class provider,
enabling NemoClaw to run on AWS without GPU infrastructure.
Tested models (all support tool calling):
nvidia.nemotron-super-3-120b, nvidia.nemotron-nano-3-30b, deepseek.v3.2, openai.gpt-oss-120b, mistral.devstral-2-123b, moonshotai.kimi-k2.5, minimax.minimax-m2.5
Usage:
export NEMOCLAW_PROVIDER=bedrockexport OPENAI_API_KEY=<bedrock-api-key>export BEDROCK_REGION=us-west-2 nemoclaw onboard --non-interactiveSummary
Add Amazon Bedrock's OpenAI-compatible endpoint as a first-class inference provider, enabling NemoClaw to run on AWS without GPU infrastructure. Follows the same pattern as existing providers (Gemini, OpenAI, Anthropic).
Related Issue
Closes #962
Changes
bin/lib/inference-config.js: Addbedrockcase togetProviderSelectionConfig()bin/lib/onboard.js:BEDROCK_ENDPOINT_URLconstant (usesBEDROCK_REGIONorAWS_REGIONenv var)bedrocktoREMOTE_PROVIDER_CONFIG(type: openai, skipVerify: true)REMOTE_MODEL_OPTIONS(all verified to work with NemoClaw)bedrocktogetNonInteractiveProvider()valid providers setbedrockoption to interactive menu insetupNim()bedrockto provider check insetupInference()Type of Change
Testing
Manually tested all 7 curated models via Telegram bridge on EC2 (t4g.large Graviton ARM64, us-west-2):
Manually tested all 7 curated models via Telegram bridge on EC2 (t4g.large Graviton ARM64, us-west-2):
Checklist
General
Code Changes
npx prek run --all-filesauto-fixes formatting (ormake formatfor targeted runs).Doc Changes
update-docsagent skill to draft changes while complying with the style guide. For example, prompt your agent with "/update-docscatch up the docs for the new changes I made in this PR."Summary by CodeRabbit