Model list discrepancy between CLI and notebook environments

[GitHub4LP]

The model list returned by kaggle models list shows only a small subset of models (primarily lower-tier ones), while Kaggle notebook environments can successfully call a much broader range of models, including many high-tier commercial ones.

This inconsistency is undocumented and affects discoverability. Users have no way to know which models are actually available in notebooks without trial and error.

Expected behavior:

• CLI should expose the same model list available in notebook environments, or
• Documentation should clearly explain the difference in model access between CLI and notebooks

[stevemessick]

Please take a closer look at the options from kaggle models list --help. The ones of interest are probably --page-token and --search. Does that help?

[GitHub4LP]

I use llm for benchmarks task .
kaggle benchmarks init -y

LLMS_AVAILABLE=anthropic/claude-haiku-4-5@20251001,deepseek-ai/deepseek-v3.2,google/gemini-3-flash-preview,google/gemini-3.1-flash-lite-preview,openai/gpt-oss-120b,qwen/qwen3-next-80b-a3b-instruct,zai/glm-5

in notebook
anthropic/claude-haiku-4-5@20251001,anthropic/claude-opus-4-1@20250805,anthropic/claude-opus-4-5@20251101,anthropic/claude-opus-4-6@default,anthropic/claude-opus-4-7@default,anthropic/claude-opus-4-8@default,anthropic/claude-sonnet-4-5@20250929,anthropic/claude-sonnet-4-6@default,anthropic/claude-sonnet-4@20250514,deepseek-ai/deepseek-r1-0528,deepseek-ai/deepseek-v3.1,deepseek-ai/deepseek-v3.2,google/gemini-2.5-flash,google/gemini-2.5-pro,google/gemini-3-flash-preview,google/gemini-3.1-flash-lite-preview,google/gemini-3.1-pro-preview,google/gemini-3.5-flash,google/gemma-4-26b-a4b,google/gemma-4-31b,openai/gpt-5.4-2026-03-05,openai/gpt-5.4-mini-2026-03-17,openai/gpt-5.4-nano-2026-03-17,openai/gpt-5.5-2026-04-23,openai/gpt-oss-120b,openai/gpt-oss-20b,qwen/qwen3-235b-a22b-instruct-2507,qwen/qwen3-coder-480b-a35b-instruct,qwen/qwen3-next-80b-a3b-instruct,qwen/qwen3-next-80b-a3b-thinking,xai/grok-4.20-0309-non-reasoning,xai/grok-4.20-0309-reasoning,zai/glm-5

[stevemessick]

@dolaameng Is this something the kaggle benchmarks command set can help with?

[GitHub4LP]

Do you plan to support the same model availability in both the CLI and notebook environments going forward?

[dolaameng]

@GitHub4LP, thanks for reporting this! This is actually by design, though I agree the docs could be clearer.

When you run kaggle benchmarks init -y, the .env file is set up with a small subset of models in LLMS_AVAILABLE — these are intended for local development and testing, and are intentionally limited to avoid abuse.

To access the full set of models (same ones available in notebooks), use the Kaggle Benchmarks task commands:

• kaggle benchmarks tasks models — lists all available models
• kaggle benchmarks tasks run — runs your task on Kaggle with the full model set

The idea is: develop and test locally with the smaller model set, then use kaggle b t run to run against the full lineup on Kaggle's infrastructure. We will update the docs! Thanks!