feat(skill): evidence-backed eval config generation for run-assert-eval - #337

Merged

alex ngo (ango10) merged 32 commits into

mainfrom

ango10/integrate-harm-eval-research

Sep 2, 2026

Commits on Aug 20, 2026

feat(skill): Integrate evidence-backed eval config generation into run-assert-eval.
Alex Ngo
committed
c942ff4
fix(skill): enforce anti-shadowing gate on the written config.
Alex Ngo
committed
9e0b2e9

Commits on Aug 21, 2026

chore(examples): clean stale generated eval configs before e2e workflow
Alex Ngo
committed
d9d5176

Commits on Aug 24, 2026

feat(init): port web search and harm-template mode into assert-ai init.
Alex Ngo
committed
f4d0908
refactor(skill): domain-scope generated eval configs and prune the unused system-eval path.
Alex Ngo
committed
32a10de
feat(eval-config): fix behavior_category_count at 25, sample_size floor, and max_turns at 6.
Alex Ngo
committed
5707532
Merge remote-tracking branch 'origin/main' into ango10/integrate-harm-eval-research
Alex Ngo
committed
1af6dee

Commits on Aug 25, 2026

feat(examples): regenerate azure_doc_qa eval configs via the research workflow.
Alex Ngo
committed
2eeb8f7
docs(skill): clean up leftover policy_violation and overrefusal in Skill.
Alex Ngo
committed
e3c366f
docs(gitignore): gitignore Clarity Protocol in .examples/.
Alex Ngo
committed
93960ba
feat(skill): annotate generated sample_size as a review me coverage floor.
Alex Ngo
committed
393460d
fix(examples): delete eval_config.yaml for azure_doc_qa to regenerate.
Alex Ngo
committed
1ce5b6a
feat(examples): regenerate azure_doc_qa evals with inline sizing comment.
Alex Ngo
committed
493b674

Commits on Aug 26, 2026

feat(examples): generate change_control_agent .yaml through the Skill workflow.
Alex Ngo
committed
09362b8
feat(examples): generate clinical prompt agents .yaml through the Skill workflow.
Alex Ngo
committed
bc7ce84
feat(examples): generate science_research_agent .yaml through the Skill workflow.
Alex Ngo
committed
010a6cb
feat(examples): generate travel_planner_langgraph .yaml through the Skill workflow.
Alex Ngo
committed
d08b1a3
feat(examples): generate travel_planner_neurosan .yaml through the Skill workflow.
Alex Ngo
committed
4364ab3
Merge remote-tracking branch 'origin/main' into ango10/integrate-harm-eval-research
Alex Ngo
committed
a7ae0e7
feat(examples): finish generation of prompt agents .yaml and update paths and readme.
Alex Ngo
committed
8780864
docs(examples): updated path to align with new azure_doc_qa.
Alex Ngo
committed
8f44b7a
docs(examples): updated path to align with new change_control_agent.
Alex Ngo
committed
64c2f5f

Commits on Aug 27, 2026

docs(examples): updated path to align with new science_research_agent.
Alex Ngo
committed
3f47947
docs(examples): updated path to align with new travel_planner_langgraph.
Alex Ngo
committed
f8ba354
docs(examples): updated path to align with new travel_planner_neurosan.
Alex Ngo
committed
7493b49

Commits on Sep 1, 2026

fix: Chang's requested changes #1, 2, 3, 4, 7
Alex Ngo
committed
7ae3c96
fix(skill): compare effective judge dimensions in the post-write gate.
Alex Ngo
committed
dbda816
fix(init): disclose web search and bound retrieved content as untrusted.
Alex Ngo
committed
ed6ce3f
fix(config): keep the omitted max_turns value and state 6 in new config.
Alex Ngo
committed
3bb4813
Merge branch 'main' into ango10/integrate-harm-eval-research
alex ngo (ango10)
authored
71be7cf
revert: restore 6 as the resolved default for an omitted max_turns.
Alex Ngo
committed
3ee74a4
docs(changelog): record the max_turns default change as breaking for 0.3.0.
Alex Ngo
committed
a97ed3e