Skip to content

chore(python): upgrade eval fixture dependencies#105

Merged
Abhijeet Prasad (AbhiPrasad) merged 1 commit into
mainfrom
abhi-chore-upgrade-python-eval-fixtures
Jul 2, 2026
Merged

chore(python): upgrade eval fixture dependencies#105
Abhijeet Prasad (AbhiPrasad) merged 1 commit into
mainfrom
abhi-chore-upgrade-python-eval-fixtures

Conversation

@AbhiPrasad

Copy link
Copy Markdown
Member

Move the Python eval fixture to Python 3.14 and refresh its uv lockfile with newer autoevals and braintrust releases. Remove the unused root uv lockfile so Python fixture dependency state stays scoped to test-eval-py.

@github-actions

github-actions Bot commented Jul 2, 2026

Copy link
Copy Markdown

Braintrust eval report

Say Hi Bot Python (abhi-chore-upgrade-python-eval-fixtures-1783018410-af6a960a)

Score Average Improvements Regressions
Levenshtein 77.8% (+0pp) - -
Llm_calls 0 (+0) - -
Tool_calls 0 (+0) - -
Errors 0 (+0) - -
Llm_errors 0 (+0) - -
Tool_errors 0 (+0) - -
Prompt_tokens 0tok (+0tok) - -
Prompt_cached_tokens 0tok (+0tok) - -
Prompt_cache_creation_tokens 0tok (+0tok) - -
Prompt_cache_creation_5m_tokens 0tok (+0tok) - -
Prompt_cache_creation_1h_tokens 0tok (+0tok) - -
Completion_tokens 0tok (+0tok) - -
Completion_reasoning_tokens 0tok (+0tok) - -
Total_tokens 0tok (+0tok) - -
Duration 0s (0s) 2 🟢 -

@github-actions

github-actions Bot commented Jul 2, 2026

Copy link
Copy Markdown

Braintrust eval report

Say Hi Bot Python (abhi-chore-upgrade-python-eval-fixtures-1783018289)

Score Average Improvements Regressions
Levenshtein 77.8% (+0pp) - -
Llm_calls 0 (+0) - -
Tool_calls 0 (+0) - -
Errors 0 (+0) - -
Llm_errors 0 (+0) - -
Tool_errors 0 (+0) - -
Prompt_tokens 0tok (+0tok) - -
Prompt_cached_tokens 0tok (+0tok) - -
Prompt_cache_creation_tokens 0tok (+0tok) - -
Prompt_cache_creation_5m_tokens 0tok (+0tok) - -
Prompt_cache_creation_1h_tokens 0tok (+0tok) - -
Completion_tokens 0tok (+0tok) - -
Completion_reasoning_tokens 0tok (+0tok) - -
Total_tokens 0tok (+0tok) - -
Duration 0s (+0s) - 2 🔴

@github-actions

github-actions Bot commented Jul 2, 2026

Copy link
Copy Markdown

Braintrust eval report

Say Hi Bot Python (abhi-chore-upgrade-python-eval-fixtures-1783018410)

Score Average Improvements Regressions
Levenshtein 77.8% (+0pp) - -
Llm_calls 0 (+0) - -
Tool_calls 0 (+0) - -
Errors 0 (+0) - -
Llm_errors 0 (+0) - -
Tool_errors 0 (+0) - -
Prompt_tokens 0tok (+0tok) - -
Prompt_cached_tokens 0tok (+0tok) - -
Prompt_cache_creation_tokens 0tok (+0tok) - -
Prompt_cache_creation_5m_tokens 0tok (+0tok) - -
Prompt_cache_creation_1h_tokens 0tok (+0tok) - -
Completion_tokens 0tok (+0tok) - -
Completion_reasoning_tokens 0tok (+0tok) - -
Total_tokens 0tok (+0tok) - -
Duration 0s (0s) 2 🟢 -

@github-actions

github-actions Bot commented Jul 2, 2026

Copy link
Copy Markdown

Braintrust eval report

Say Hi Bot Python (abhi-chore-upgrade-python-eval-fixtures-1783018289-05b44c5c)

Score Average Improvements Regressions
Levenshtein 77.8% (+0pp) - -
Llm_calls 0 (+0) - -
Tool_calls 0 (+0) - -
Errors 0 (+0) - -
Llm_errors 0 (+0) - -
Tool_errors 0 (+0) - -
Prompt_tokens 0tok (+0tok) - -
Prompt_cached_tokens 0tok (+0tok) - -
Prompt_cache_creation_tokens 0tok (+0tok) - -
Prompt_cache_creation_5m_tokens 0tok (+0tok) - -
Prompt_cache_creation_1h_tokens 0tok (+0tok) - -
Completion_tokens 0tok (+0tok) - -
Completion_reasoning_tokens 0tok (+0tok) - -
Total_tokens 0tok (+0tok) - -
Duration 0s (0s) 2 🟢 -

@github-actions

github-actions Bot commented Jul 2, 2026

Copy link
Copy Markdown

Braintrust eval report

Say Hi Bot (abhi-chore-upgrade-python-eval-fixtures-1783018419)

Score Average Improvements Regressions
Levenshtein 100% (+0pp) - -
Llm_calls 0 (+0) - -
Tool_calls 0 (+0) - -
Errors 0 (+0) - -
Llm_errors 0 (+0) - -
Tool_errors 0 (+0) - -
Prompt_tokens 0tok (+0tok) - -
Prompt_cached_tokens 0tok (+0tok) - -
Prompt_cache_creation_tokens 0tok (+0tok) - -
Prompt_cache_creation_5m_tokens 0tok (+0tok) - -
Prompt_cache_creation_1h_tokens 0tok (+0tok) - -
Completion_tokens 0tok (+0tok) - -
Completion_reasoning_tokens 0tok (+0tok) - -
Total_tokens 0tok (+0tok) - -
Duration 1s (+0s) 1 🟢 17 🔴

@github-actions

github-actions Bot commented Jul 2, 2026

Copy link
Copy Markdown

Braintrust eval report

Console logging (abhi-chore-upgrade-python-eval-fixtures-1783018416)

Score Average Improvements Regressions
Levenshtein 100% (+0pp) - -
Llm_calls 0 (+0) - -
Tool_calls 0 (+0) - -
Errors 0 (+0) - -
Llm_errors 0 (+0) - -
Tool_errors 0 (+0) - -
Prompt_tokens 0tok (+0tok) - -
Prompt_cached_tokens 0tok (+0tok) - -
Prompt_cache_creation_tokens 0tok (+0tok) - -
Prompt_cache_creation_5m_tokens 0tok (+0tok) - -
Prompt_cache_creation_1h_tokens 0tok (+0tok) - -
Completion_tokens 0tok (+0tok) - -
Completion_reasoning_tokens 0tok (+0tok) - -
Total_tokens 0tok (+0tok) - -
Duration 0s (+0s) - 20 🔴

My Evaluation (abhi-chore-upgrade-python-eval-fixtures-1783018416)

Score Average Improvements Regressions
Exact match 100% (+0pp) - -
Llm_calls 0 (+0) - -
Tool_calls 0 (+0) - -
Errors 0 (+0) - -
Llm_errors 0 (+0) - -
Tool_errors 0 (+0) - -
Prompt_tokens 10tok (+0tok) - -
Prompt_cached_tokens 0tok (+0tok) - -
Prompt_cache_creation_tokens 0tok (+0tok) - -
Prompt_cache_creation_5m_tokens 0tok (+0tok) - -
Prompt_cache_creation_1h_tokens 0tok (+0tok) - -
Completion_tokens 2tok (+0tok) - -
Completion_reasoning_tokens 0tok (+0tok) - -
Total_tokens 12tok (+0tok) - -
Duration 0.15s (-0.2s) 1 🟢 -

Say Hi Bot (abhi-chore-upgrade-python-eval-fixtures-1783018416)

Score Average Improvements Regressions
Levenshtein 100% (+0pp) - -
Llm_calls 0 (+0) - -
Tool_calls 0 (+0) - -
Errors 0 (+0) - -
Llm_errors 0 (+0) - -
Tool_errors 0 (+0) - -
Prompt_tokens 0tok (+0tok) - -
Prompt_cached_tokens 0tok (+0tok) - -
Prompt_cache_creation_tokens 0tok (+0tok) - -
Prompt_cache_creation_5m_tokens 0tok (+0tok) - -
Prompt_cache_creation_1h_tokens 0tok (+0tok) - -
Completion_tokens 0tok (+0tok) - -
Completion_reasoning_tokens 0tok (+0tok) - -
Total_tokens 0tok (+0tok) - -
Duration 1s (0s) 17 🟢 1 🔴

@github-actions

github-actions Bot commented Jul 2, 2026

Copy link
Copy Markdown

Braintrust eval report

Say Hi Bot (abhi-chore-upgrade-python-eval-fixtures-1783018427)

Score Average Improvements Regressions
Levenshtein 100% (+0pp) - -
Llm_calls 0 (+0) - -
Tool_calls 0 (+0) - -
Errors 0 (+0) - -
Llm_errors 0 (+0) - -
Tool_errors 0 (+0) - -
Prompt_tokens 0tok (+0tok) - -
Prompt_cached_tokens 0tok (+0tok) - -
Prompt_cache_creation_tokens 0tok (+0tok) - -
Prompt_cache_creation_5m_tokens 0tok (+0tok) - -
Prompt_cache_creation_1h_tokens 0tok (+0tok) - -
Completion_tokens 0tok (+0tok) - -
Completion_reasoning_tokens 0tok (+0tok) - -
Total_tokens 0tok (+0tok) - -
Duration 1s (0s) 9 🟢 -

Move the Python eval fixture to Python 3.14 and refresh its uv lockfile
with newer autoevals and braintrust releases. Remove the unused root uv
lockfile so Python fixture dependency state stays scoped to test-eval-py.
@AbhiPrasad Abhijeet Prasad (AbhiPrasad) force-pushed the abhi-chore-upgrade-python-eval-fixtures branch from 08d2ae4 to 823eb8b Compare July 2, 2026 18:53
@AbhiPrasad Abhijeet Prasad (AbhiPrasad) merged commit a49be6e into main Jul 2, 2026
9 checks passed
@github-actions

github-actions Bot commented Jul 2, 2026

Copy link
Copy Markdown

Braintrust eval report

Say Hi Bot Python (main-1783019069)

Score Average Improvements Regressions
Levenshtein 77.8% (+0pp) - -
Llm_calls 0 (+0) - -
Tool_calls 0 (+0) - -
Errors 0 (+0) - -
Llm_errors 0 (+0) - -
Tool_errors 0 (+0) - -
Prompt_tokens 0tok (+0tok) - -
Prompt_cached_tokens 0tok (+0tok) - -
Prompt_cache_creation_tokens 0tok (+0tok) - -
Prompt_cache_creation_5m_tokens 0tok (+0tok) - -
Prompt_cache_creation_1h_tokens 0tok (+0tok) - -
Completion_tokens 0tok (+0tok) - -
Completion_reasoning_tokens 0tok (+0tok) - -
Total_tokens 0tok (+0tok) - -
Duration 0s (0s) 2 🟢 -

@github-actions

github-actions Bot commented Jul 2, 2026

Copy link
Copy Markdown

Braintrust eval report

Say Hi Bot Python (main-1783019071)

Score Average Improvements Regressions
Levenshtein 77.8% (+0pp) - -
Llm_calls 0 (+0) - -
Tool_calls 0 (+0) - -
Errors 0 (+0) - -
Llm_errors 0 (+0) - -
Tool_errors 0 (+0) - -
Prompt_tokens 0tok (+0tok) - -
Prompt_cached_tokens 0tok (+0tok) - -
Prompt_cache_creation_tokens 0tok (+0tok) - -
Prompt_cache_creation_5m_tokens 0tok (+0tok) - -
Prompt_cache_creation_1h_tokens 0tok (+0tok) - -
Completion_tokens 0tok (+0tok) - -
Completion_reasoning_tokens 0tok (+0tok) - -
Total_tokens 0tok (+0tok) - -
Duration 0s (0s) 1 🟢 1 🔴

@github-actions

github-actions Bot commented Jul 2, 2026

Copy link
Copy Markdown

Braintrust eval report

Say Hi Bot (main-1783019073)

Score Average Improvements Regressions
Levenshtein 100% (+0pp) - -
Llm_calls 0 (+0) - -
Tool_calls 0 (+0) - -
Errors 0 (+0) - -
Llm_errors 0 (+0) - -
Tool_errors 0 (+0) - -
Prompt_tokens 0tok (+0tok) - -
Prompt_cached_tokens 0tok (+0tok) - -
Prompt_cache_creation_tokens 0tok (+0tok) - -
Prompt_cache_creation_5m_tokens 0tok (+0tok) - -
Prompt_cache_creation_1h_tokens 0tok (+0tok) - -
Completion_tokens 0tok (+0tok) - -
Completion_reasoning_tokens 0tok (+0tok) - -
Total_tokens 0tok (+0tok) - -
Duration 1s (+0s) - 20 🔴

@github-actions

github-actions Bot commented Jul 2, 2026

Copy link
Copy Markdown

Braintrust eval report

Console logging (main-1783019074)

Score Average Improvements Regressions
Levenshtein 100% (+0pp) - -
Llm_calls 0 (+0) - -
Tool_calls 0 (+0) - -
Errors 0 (+0) - -
Llm_errors 0 (+0) - -
Tool_errors 0 (+0) - -
Prompt_tokens 0tok (+0tok) - -
Prompt_cached_tokens 0tok (+0tok) - -
Prompt_cache_creation_tokens 0tok (+0tok) - -
Prompt_cache_creation_5m_tokens 0tok (+0tok) - -
Prompt_cache_creation_1h_tokens 0tok (+0tok) - -
Completion_tokens 0tok (+0tok) - -
Completion_reasoning_tokens 0tok (+0tok) - -
Total_tokens 0tok (+0tok) - -
Duration 0s (+0s) 1 🟢 2 🔴

My Evaluation (main-1783019074)

Score Average Improvements Regressions
Exact match 100% (+0pp) - -
Llm_calls 0 (+0) - -
Tool_calls 0 (+0) - -
Errors 0 (+0) - -
Llm_errors 0 (+0) - -
Tool_errors 0 (+0) - -
Prompt_tokens 10tok (+0tok) - -
Prompt_cached_tokens 0tok (+0tok) - -
Prompt_cache_creation_tokens 0tok (+0tok) - -
Prompt_cache_creation_5m_tokens 0tok (+0tok) - -
Prompt_cache_creation_1h_tokens 0tok (+0tok) - -
Completion_tokens 2tok (+0tok) - -
Completion_reasoning_tokens 0tok (+0tok) - -
Total_tokens 12tok (+0tok) - -
Duration 0.23s (+0.08s) - 1 🔴

Say Hi Bot (main-1783019074)

Score Average Improvements Regressions
Levenshtein 100% (+0pp) - -
Llm_calls 0 (+0) - -
Tool_calls 0 (+0) - -
Errors 0 (+0) - -
Llm_errors 0 (+0) - -
Tool_errors 0 (+0) - -
Prompt_tokens 0tok (+0tok) - -
Prompt_cached_tokens 0tok (+0tok) - -
Prompt_cache_creation_tokens 0tok (+0tok) - -
Prompt_cache_creation_5m_tokens 0tok (+0tok) - -
Prompt_cache_creation_1h_tokens 0tok (+0tok) - -
Completion_tokens 0tok (+0tok) - -
Completion_reasoning_tokens 0tok (+0tok) - -
Total_tokens 0tok (+0tok) - -
Duration 1s (0s) 20 🟢 -

@github-actions

github-actions Bot commented Jul 2, 2026

Copy link
Copy Markdown

Braintrust eval report

Say Hi Bot (main-1783019082)

Score Average Improvements Regressions
Levenshtein 100% (+0pp) - -
Llm_calls 0 (+0) - -
Tool_calls 0 (+0) - -
Errors 0 (+0) - -
Llm_errors 0 (+0) - -
Tool_errors 0 (+0) - -
Prompt_tokens 0tok (+0tok) - -
Prompt_cached_tokens 0tok (+0tok) - -
Prompt_cache_creation_tokens 0tok (+0tok) - -
Prompt_cache_creation_5m_tokens 0tok (+0tok) - -
Prompt_cache_creation_1h_tokens 0tok (+0tok) - -
Completion_tokens 0tok (+0tok) - -
Completion_reasoning_tokens 0tok (+0tok) - -
Total_tokens 0tok (+0tok) - -
Duration 1s (+0s) 1 🟢 19 🔴

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant