Fixing UnsupportedParamsError in litellm when using groq model with OpenAI

[szymonrucinski]

Dear Maintainers,

I encountered an issue when running the command:

python -m routellm.openai_server --routers mf --strong-model openai/gpt4o --weak-model groq/llama3-8b-8192

The error message indicated that the groq model does not support certain parameters:

litellm.UnsupportedParamsError: groq does not support parameters: {'presence_penalty': 0.0, 'frequency_penalty': 0.0}, for model=llama3-8b-8192.

To resolve this, I added the following command to set drop_params to True:

import litellm
litellm.drop_params = True

This change successfully addresses the issue by ensuring that unsupported parameters are dropped, allowing the command to execute without errors.

I would like to submit these changes for review and merge into the main branch. Please let me know if you need any further information or modifications.

Thanks!

[iojw]

Is there a reason why the parameters are set for gpt4o but not groq? Since you're setting the values to 0 (default), you can simply remove the params from the RouteLLM controller call to avoid this issue!

[pybanaszak]

Same issue here when running the local version of the example.

python -m routellm.openai_server --routers mf --weak-model ollama_chat/llama3 --config config.example.yaml

import openai

client = openai.OpenAI(
  base_url="http://localhost:6060/v1",
  api_key="no_api_key"
)

response = client.chat.completions.create(
  model="router-mf-0.11593",
  messages=[
    {"role": "user", "content": "Hello!"}
  ]
)

print(response)

raise UnsupportedParamsError(
litellm.exceptions.UnsupportedParamsError: litellm.UnsupportedParamsError: ollama_chat does not support parameters: {'presence_penalty': 0.0}, for model=llama3. To drop these, set `litellm.drop_params=True` or for proxy:

`litellm_settings:
 drop_params: true`