[farook-edev]

this PR adds a couple fixes from mobile_app_open's ifeval. First, the Character counting instead of byte counting. Second, it removes special tokens before running the evaluation

[github-actions]

MLCommons CLA bot All contributors have signed the MLCommons CLA ✍️ ✅

[freedomtan]

@AhmedTElthakeb please take a look at this one.

[AhmedTElthakeb]

Couple comments on the special-token removal part:

• I do not recommend to change the reference python implementation (generally) and specially for this case, I would rather try to strip the special tokens in the response generate pipeline (e.g. detokenize with skip_special_tokens) or in a separate preprocessing step, and keep the reference python as identical to google-research.

• Also, the token list is model-specific and incomplete in this case. It covers some special tokens <pad>/<eos>/<bos>/<mask>, but misses Gemma's most special tokens such as <start_of_turn>/<end_of_turn>. Doing this at the tokenizer level, which knows its own special tokens, avoids keeping a hard coded list.