Conversation
Added variables from run_experiments.sh and eval.sh, i added short descriptions for the ones I know
AymanBx
commented
Jul 22, 2025
README.md
Outdated
| - all_tasks | ||
| - list of tasks to be evaluated | ||
| - models | ||
| - Models being used to evaluate results |
Collaborator
Author
There was a problem hiding this comment.
This actually describes the eval model
README.md
Outdated
| - | ||
| - all_tasks | ||
| - list of tasks to be evaluated | ||
| - models |
Collaborator
Author
There was a problem hiding this comment.
The models that we are evaluating on above tasks
README.md
Outdated
| - log_dir | ||
| - directory that the llm placed the experiment logs | ||
| - json_folder | ||
| - |
Collaborator
Author
There was a problem hiding this comment.
The path in which the evaluation results will be placed
README.md
Outdated
Comment on lines
47
to
49
| - edit_script_model | ||
|
|
||
| - fast_llm |
Collaborator
Author
There was a problem hiding this comment.
Using a different model to do smaller tasks as in editing scripts and understanding file contents rather than using the same main agent model is optional.
For our experiment we are using the same model to do all the work
AymanBx
commented
Jul 22, 2025
README.md
Outdated
| How well can an LLM agent improve the training script to achieve high fairness metrics. | ||
|
|
||
| ## Fairness Metrics: | ||
|
|
Collaborator
Author
There was a problem hiding this comment.
@surbhir08 could you please just put the names of the metrics here?
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
No description provided.