Repository navigation
fix(bfcl): keep the bfcl extra importable by capping pyarrow below 26 - #2091
Merged
Merged
Conversation
JerryChaox
marked this pull request as ready for review
November 15, 2026 04:25
Contributor
Author
|
@Yunnglin Ready for review: this keeps |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Following the documented BFCL setup (
pip install evalscope, thenpip install bfcl-eval==2025.10.27.1) currently leaves an environment where evalscope cannot be imported:Cause
bfcl-evalpinsnumpy==1.26.4. Its latest release, 2026.3.23, still does.import datasets(and with itimport evalscope) fails.bfcl-evalfrom the full CI job.Fix
pyarrow<26in thebfclextra, next to thebfcl-evalpin, with a comment explaining the reason. It can be removed oncebfcl-evalstops pinning NumPy 1.x.pip install 'evalscope[bfcl]', so that installingbfcl-evalalways brings the cap with it. Updated: both BFCL guides (en/zh), and the v3/v4 adapter descriptions, with the generated meta and benchmark pages regenerated throughbenchmark-info --update/--generate-docs. The zh text is a manual one-line edit written throughtranslate_benchmarks, because I don't have a DashScope key. No evaluation semantics change, soevaluation_versionis unchanged.Verification (Python 3.11, pip)
pip install evalscope(1.15.0), thenpip install bfcl-eval==2025.10.27.1(current docs)import evalscope.runfailspip install -e '.[bfcl]'on this branchpip checkclean, BFCL-v4load()works forsimple_pythonandmemory_kvpip install -e '.[bfcl]'test_bfcl_v3_summary,test_bfcl_v4,test_bfcl_v4_pipeline,test_bfcl_v4_state) andtests/api/test_benchmark_registry_contract.pyin the fixed environment: 55 passed, 1 failed. The failure istest_bfcl_v4.py::test_patch_against_real_bfcl_eval_handler, which also fails on unmodified main when it runs after the pipeline/state tests in the same session, and passes on its own. It is order-dependent and unrelated to this change.pytest tests/cli/test_all.py::TestRun::test_ci_lite: pass.Note: the BFCL tests in a clean environment also need
soundfile, whichbfcl-eval'sqwen_agentimport chain uses but does not declare. I left that out of this PR to keep the scope tight.