Skip to content

Agent#18

Open
AymanBx wants to merge 84 commits intomainfrom
Agent
Open

Agent#18
AymanBx wants to merge 84 commits intomainfrom
Agent

Conversation

@AymanBx
Copy link
Contributor

@AymanBx AymanBx commented Mar 7, 2025

closing #2

Falvespereira and others added 15 commits March 4, 2025 18:04
@Falvespereira
first steps to making reading csv
0917ea9 
creating csv file
@Falvespereira
csv runner update (untested)
00b05e8 
should return the prompts as a array of strings
@bendahrooge
Ollama and OpenAI API support complete
8e94e0a
@emorysalaun
From_Yaml and minor fixes.
d19bb60 
From_Yaml has been implemented and tested on the test yml file. Minor changes to the imports in runner.py. Initialization of the tasks list for Bench() has been fixed as well.
@rittaneg86
log file.
8b92d09
@AymanBx
Delete .vscode directory
90083a6
@victoriaryan2
Update runner.py
5b50d5b 
yaml
@AymanBx
initialze bench object
84ab6de
@Falvespereira
update to include the answers
36d5230
@bendahrooge
Runner returns prompt tuple, response
19bb281
@AymanBx
Planning out the run function and the task object creation
fb98657
@AymanBx
fixed CSV dependancies merge conflict
40d391b
@AymanBx
Merge branch 'ben-mar4' into Agent
5eafec8
@AymanBx
Merge branch 'log_file' into Agent
baf9a1c
@AymanBx
Merge branch 'yaml-1' into Agent
6afd6f5
@AymanBx AymanBx linked an issue Mar 7, 2025 that may be closed by this pull request
Agent evaluation #2
Open
AymanBx and others added 14 commits July 1, 2025 19:40
@AymanBx
Merge branch 'Agent' into from_yaml
a37b38d
@AymanBx
Merge pull request #11 from evalexplainAI/from_yaml
59a8f47 
From_Yaml and minor fixes.
@AymanBx
Added .gitignore with pyhon template
5b17a73
@AymanBx
Added necessary files to make the project into an installable python …
d6fa598 
…package
@AymanBx
Modifications to metadata in project.toml + fixing the cli to call th…
0b0c068 
…e correct cli function
@AymanBx
cli.py: Adding init function and subfunctions for it
8d9aaaf
@AymanBx
add git init function
62d3b42
@AymanBx
small modifications to CLI functions to test
097e08e
@AymanBx
CLI: Create .gitignore in init_repo function
b9f07cc
@AymanBx
README: Adding some usage info for CLI
dad2eae
@abdel-elj @AymanBx
Adding CLI group to cli.py
65c219e
@abdel-elj @AymanBx
Initialize class for adding new task in cli.py
73cea67
@emorysalaun @abdel-elj
@nadia-sousa @AymanBx
Generates Folder Structure For Benchmark
7630e35 
Generates main folder and two subfolders (reports and tasks) for further filling.

Co-Authored-By: abdel-elj <196731936+abdel-elj@users.noreply.github.com>
Co-Authored-By: nadia-sousa <98436401+nadia-sousa@users.noreply.github.com>
@AymanBx
Small modifications and comments
1b62f83
@brownsarahm
Copy link
Contributor

brownsarahm commented Jan 29, 2026

docs should go into main at the same time as the functionality....

@brownsarahm
Copy link
Contributor

brownsarahm commented Feb 12, 2026

Urgent:

  • datasets module is imported and not clear where it is, so i cannot

Notes: (I can help fix, if i get the missing module so that i can run any of this)

  • everything other than CLI needs compelte docstrings
  • all CLI command options need helps
  • tasks in init should not require 2 params, default should be that name and relative path match
  • build and add task are not clear
  • print will not work all the itm in CLI, there needs to be better handling, logs are returned and cli can click.echo
  • tasks are underdeveloped
  • scorerer are underdeveloped
  • betterbench seems missing

brownsarahm and others added 22 commits February 12, 2026 10:25
@brownsarahm
begin documentation and more dependencies
cd20fb9
@brownsarahm
Merge branch 'Agent' into doc
5f4a280
@brownsarahm
Merge pull request #20 from ml4sts/doc
c805ec0 
Doc
@brownsarahm
get it to run enough to give help function and some cleanup
d26ab3a
@AymanBx
Some bug fixes and cleaning up
53acd88
@AymanBx
Some documenting
134bef0
@AymanBx
BetterBench: Introducing betterbench interactive session
6b5a5ec
@AymanBx
Logger: Now task objects have their separate loggers.
9443fa4
@brownsarahm
fix typo
4703271
@brownsarahm
remove unnecessary
7a006b2
@brownsarahm
demo running form cwd, not up one, very conufsing
65c86f5
@brownsarahm
start as notebook to have a demo
2249d30
@brownsarahm
doc tone fixes
f010754
@brownsarahm
"correct mindset" is dimunitiive
d5dda91
@brownsarahm
start refactor
603b665
@brownsarahm
rename, start class methods, and move runner all to run method to mak…
b8d7f62 
…e the task objects more usable in design
@brownsarahm
continue refactor and start test case
d0910ea
@brownsarahm
more demo use
cc2b61d
@AymanBx
Bug fixes to get the CLI to reach . Tasks still need to be initialize…
6581b95 
…d in the Bench object
@AymanBx
Benchmark: run(): after reading the docs and the issues, I think this…
1241c2f 
… is how this should look like
@AymanBx
CLI: got rid of absolute paths using the Bench.load logic. Applying T…
a77166b 
…ask logic in pylib.md to loading tasks into the Bench
@AymanBx
bug fix
7a4804c
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@brownsarahm brownsarahm Awaiting requested review from brownsarahm

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

Agent evaluation

8 participants

@AymanBx @brownsarahm @Falvespereira @bendahrooge @emorysalaun @rittaneg86 @victoriaryan2 @abdel-elj