Search results

Perplexity metric fails with GPT-2 tokenizer

Status: Open.

#766 In huggingface/evaluate;

· zanvari opened on Jun 17, 2026

New community metric: RAIL Score — responsible AI evaluation across 8 dimensions

Status: Open.

#742 In huggingface/evaluate;

· SumitVermakgp opened on Apr 1, 2026

Task Text Summarization does not work

Status: Open.

#741 In huggingface/evaluate;

· DominikHil opened on Mar 29, 2026

BERTScore: OverflowError with transformers>=5 due to undefined model_max_length

Status: Open.

#739 In huggingface/evaluate;

· uriyapesso opened on Mar 24, 2026

Proposal: Retrieval-Conditioned Confidence Metric (RCCS) for RAG evaluation

Status: Open.

#736 In huggingface/evaluate;

· devarakondasrikanth opened on Feb 26, 2026

BLEU: improve input validation and document edge-case behavior)

Status: Open.

#729 In huggingface/evaluate;

· Vangmay opened on Jan 28, 2026

Proposal: Make SQuAD QA F1 logic reusable as a generic QA metric in evaluate

Status: Open.

#724 In huggingface/evaluate;

· Ayesha-Imr opened on Dec 27, 2025

FileNotFoundError: Couldn't find a module script at /workspace/weiw13@xiaopeng.com/VLM/evaluate/meteor/meteor.py. Module 'meteor' doesn't exist on the Hugging Face Hub either.

Status: Open.

#723 In huggingface/evaluate;

· viviansmlie opened on Dec 18, 2025

Incompatibility with Gradio >= 4.0.0

Status: Open.

#713 In huggingface/evaluate;

· shivanraptor opened on Nov 3, 2025

bug: huggingface-hub 1.0.0 release breaks existing evaluate releases

Status: Open.

#712 In huggingface/evaluate;

· nvluxiaoz opened on Oct 28, 2025

Unable to reproduce example results using the Unbabel/wmt22-comet-da model

Status: Open.

#711 In huggingface/evaluate;

· starbuild7505 opened on Oct 16, 2025

DownloadConfig is not passed to LocalEvaluationModuleFactory when loading from local file, causing uncontrollable download behavior

Status: Open.

#709 In huggingface/evaluate;

· Very-White opened on Sep 30, 2025