Skip to content

fix: correct ground_truth comparison in BanditEnv - #190

Open
bigsawman wants to merge 1 commit into
TextArena:mainfrom
bigsawman:fix/bandit-ground-truth-comparison
Open

fix: correct ground_truth comparison in BanditEnv#190
bigsawman wants to merge 1 commit into
TextArena:mainfrom
bigsawman:fix/bandit-ground-truth-comparison

Conversation

@bigsawman

Copy link
Copy Markdown

Summary

  • ground_truth in BanditEnv is a dict mapping button names to probabilities (e.g. {"red": 0.6, "blue": 0.3, ...}), but the final-turn winner check compares a button string directly against this dict (button == self.state.game_state['ground_truth']), which always evaluates to False.
  • This means the player can never be recognized as having chosen the correct button — every game ends with an incorrect outcome and a regret-based reward.
  • Fix: find the button with the highest probability via max(..., key=...), then compare the player's choice against that.

Reproduction

ground_truth = {"red": 0.6, "blue": 0.3}
button = "red"
print(button == ground_truth)  # False — string vs dict, always False

Test plan

  • Verified that _regret() already correctly uses ground_truth as a dict (calls .values() and indexes by button name), confirming this is a dict, not a string.
  • Confirmed the fix matches the intended semantics: reward 1.0 when the player picks the highest-probability button.

`ground_truth` is a dict mapping button names to probabilities, but
the final-turn check compared a button string directly against this
dict (`button == self.state.game_state['ground_truth']`), which always
evaluates to False. This means the player can never win.

Fix: find the button with the highest probability first, then compare.
@borgr

borgr commented Aug 8, 2026

Copy link
Copy Markdown
Collaborator

Ready to merge. Correctly diagnosed: ground_truth is a dict {button: prob}, so the old final-turn check button == self.state.game_state['ground_truth'] (str == dict) was always False — the reward=1.0 branch was dead code, and even a correct pick fell through to the regret branch. The fix max(ground_truth, key=lambda b: ground_truth[b]) is the right one and matches how reset() builds the distribution (the intended button always gets the highest probability, so the argmax is exact). Minimal and in-scope.

@borgr

borgr commented Aug 10, 2026

Copy link
Copy Markdown
Collaborator

Verified. ground_truth is a dict {button: prob}, so the old button == self.state.game_state['ground_truth'] compared a str to a dict — always False. The +1 "correct button" reward was therefore unreachable, and even the optimal button fell through to the regret branch (scoring 0.0, its own regret). Taking argmax over the probability dict is correct: by construction (reset assigns the ground-truth button 0.5 + p_gap/2 and every other button < 0.5 - p_gap/2) the ground-truth button is always the unique highest-mean button.

LGTM, recommend merge.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants