Fixes ENG-1430: Add eval run utility in TS#8

Merged

andreibratu merged 42 commits intomasterfrom

ENG-1430-eval-run

Dec 20, 2024

Commits on Dec 6, 2024

Early draft
Andrei Bratu
committed

Commits on Dec 7, 2024

Fixed exporter
Andrei Bratu
committed
processor correctly implements interface
Andrei Bratu
committed
Update yarn lock
Andrei Bratu
committed
Added back prompt test
Andrei Bratu
committed

Commits on Dec 8, 2024

Added provider tests
Andrei Bratu
committed

Commits on Dec 9, 2024

Yarn lock change
Andrei Bratu
committed
Added API keys to CI action
Andrei Bratu
committed
debugging api keys
Andrei Bratu
committed
whitespace fix attempt
Andrei Bratu
committed
Fixed wrong names on CI secrets
Andrei Bratu
committed
Added instrumentation function
Andrei Bratu
committed

Commits on Dec 10, 2024

QA Changes
Andrei Bratu
committed
Debugging webpack test
Andrei Bratu
committed
Leftover console log
Andrei Bratu
committed
More QA
Andrei Bratu
committed
QA changes
Andrei Bratu
committed
Fixed tool test
Andrei Bratu
committed
Sorted imports
Andrei Bratu
committed

Commits on Dec 11, 2024

Import sort + code cleanup
Andrei Bratu
committed
Added scripts for local testing
Andrei Bratu
committed
amend prepack script
Andrei Bratu
committed
nit in .fernignore
Andrei Bratu
committed

Commits on Dec 12, 2024

Reorganised imports
Andrei Bratu
committed
Init draft eval_utils
Andrei Bratu
committed
Ported eval_utils
Andrei Bratu
committed
Reordered imports
Andrei Bratu
committed
package json tweaks
Andrei Bratu
committed
Type errors
Andrei Bratu
committed
Prompt instrumentation works in all downstream examples
Andrei Bratu
committed

Commits on Dec 14, 2024

Eval works with simple callables
Andrei Bratu
committed

Commits on Dec 17, 2024

QA pass over eval run
Andrei Bratu
committed
Fixed webpack test
Andrei Bratu
committed
use lodash for safe object comparison
Andrei Bratu
committed

Commits on Dec 19, 2024

change to use smaller models
harry-humanloop
committed
Harry feedback
Andrei Bratu
committed

Commits on Dec 20, 2024

Support streamable mode + added doc to utilities
Andrei Bratu
committed
Docstring for run eval in TS
Andrei Bratu
committed
QA for processor changes
Andrei Bratu
committed
QA pass + PR review feedback
Andrei Bratu
committed
Added catch blocks detailing the upsert error
Andrei Bratu
committed
Better messages around instrumented provider
Andrei Bratu
committed