Skip to content

fix: correct misleading ValueError message in Attention._call_#617

Open
Dhakshin2007 wants to merge 1 commit intogoogle-deepmind:mainfrom
Dhakshin2007:Dhakshin2007-patch-1
Open

fix: correct misleading ValueError message in Attention._call_#617
Dhakshin2007 wants to merge 1 commit intogoogle-deepmind:mainfrom
Dhakshin2007:Dhakshin2007-patch-1

Conversation

@Dhakshin2007
Copy link
Copy Markdown

Bug Fix: Incorrect ValueError message in Attention.__call__

Problem

In gemma/gm/nn/_modules.py, the ValueError raised when attn_type is neither AttentionType.LOCAL_SLIDING nor AttentionType.GLOBAL contains a misleading error message:

# Before (buggy)
raise ValueError(
    'Attn_type must be either AttentionType.GLOBAL or'
    f' AttentionType.GLOBAL not {self.attn_type}'
)

Both sides of "or" show AttentionType.GLOBAL — this is a copy-paste error. The first value should be AttentionType.LOCAL_SLIDING.

Fix

# After (fixed)
raise ValueError(
    'Attn_type must be either AttentionType.LOCAL_SLIDING or'
    f' AttentionType.GLOBAL not {self.attn_type}'
)

Impact

This does not affect runtime behavior (the condition itself is correct), but the misleading error message makes debugging difficult for users who encounter unsupported attention types.

The error message raised when `attn_type` is neither LOCAL_SLIDING
nor GLOBAL incorrectly said "AttentionType.GLOBAL or AttentionType.GLOBAL"
due to a copy-paste mistake. The first enum name should be
AttentionType.LOCAL_SLIDING, not AttentionType.GLOBAL.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant