Skip to content

Add SWE-bench-verified benchmark with GLM 5.3 as optimizer and agent - #673

Closed
yyualvin wants to merge 1 commit into
skillberry-ai:mainfrom
yyualvin:glm-5-3-run
Closed

yyualvin wants to merge 1 commit into
skillberry-ai:mainfrom
yyualvin:glm-5-3-run

Conversation

@yyualvin

@yyualvin yyualvin commented Oct 6, 2026

Copy link
Copy Markdown

Adds a complete worked example for optimizing a skill-package capability
against SWE-bench Verified using the Harbor adapter and GLM-5.3:

  • capevolve.yaml: cap-evolve spec (hill-climb, skill-package/system-prompt,
    Harbor dataset adapter, paired gate)
  • run.sh: reproducible launcher for Harbor on OpenShift via an AI gateway,
    with env wiring for agent/optimizer and headless claude onboarding
  • run_full/: full 500-task run artifacts — baseline/final JSON scores,
    events/graph/history/rejected JSONL, and optimizer JOURNAL/INSIGHTS

The run reached 0.976 val and 0.959 test reward after 10 candidates.

…h GLM-5.3

Adds a complete worked example for optimizing a skill-package capability
against SWE-bench Verified using the Harbor adapter and GLM-5.3:

- capevolve.yaml: cap-evolve spec (hill-climb, skill-package/system-prompt,
  Harbor dataset adapter, paired gate)
- run.sh: reproducible launcher for Harbor on OpenShift via an AI gateway,
  with env wiring for agent/optimizer and headless claude onboarding
- run_full/: full 500-task run artifacts — baseline/final JSON scores,
  events/graph/history/rejected JSONL, and optimizer JOURNAL/INSIGHTS

The run reached 0.976 val and 0.959 test reward after 10 candidates.
@OsherElhadad

OsherElhadad commented Oct 13, 2026 •

Copy link
Copy Markdown
Collaborator

@yyualvin Can you pls push it to this branch: benchmark_history ?

@yyualvin

Copy link
Copy Markdown
Author

Hi @OsherElhadad sorry this is a residual PR, the actual one is merged #699

@yyualvin yyualvin closed this Oct 14, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants