[WIP] Add Gemini and GPT-Audio

[zhopto3]

Description

• Model Name: gemini-2.5-flash
• Model Type: speechLLM
• Checkpoint: N/A
• Paper/Technical Report: https://storage.googleapis.com/deepmind-media/gemini/gemini_v2_5_report.pdf
• License: Proprietary
• Motivation: Most recent non-preview Gemini release that supports audio input.

Outputs

List of benchmarks for which outputs are provided:

• FLEURS
• CoVoST2
• Europarl-ST
• WMT

Exclusions

Excluded non-generic benchmarks due to budget constraints

[Gldkslfmsd]

Description

• Model Name: gpt-audio
• Model Type: speechLLM
• Checkpoint: N/A
• Paper/Technical Report:
• License:
• Motivation:

Outputs

List of benchmarks for which outputs are provided:

• FLEURS
• CoVoST2
• Europarl-ST
• WMT
• WinoST
• CommonAccent
• ManDi
• CS-Dialogue
• CS-FLEURS
• LibriStutter
• NoisyFLEURS (ambient)
• NoisyFLEURS (babble)
• EmotionTalk
• mExpresso
• ACL 60/60 (long)
• ACL 60/60 (short)
• MCIF (long)
• MCIF (short)

Exclusions

too expensive

[JAVI897]

Can we merge it? @zhopto3 @sarapapi

[sarapapi]

Can we merge it? @zhopto3 @sarapapi

No, I've asked @zhopto3 to create a clean PR including only Gemini inference code, outputs, and evals. Then, we'll close this one.