Skip to content

Skill report generator crashes when history is empty #417

Description

@MicroMilo

Summary

skills/skill-creator/scripts/generate_report.py::generate_html() assumes that history contains at least one iteration. Passing an empty history raises ValueError: max() iterable argument is empty instead of producing an empty or "no iterations yet" report. This makes the report generator brittle for early live-report states, partial results, or callers that want to render a valid report before the first evaluation result exists.

Code path

  • skills/skill-creator/scripts/generate_report.py:16-20 initializes history = data.get("history", []).
  • skills/skill-creator/scripts/generate_report.py:25-30 handles query discovery only when history is non-empty.
  • skills/skill-creator/scripts/generate_report.py:205-209 always calls max(history, ...) when choosing the best iteration, even if history is empty.
  • skills/skill-creator/scripts/run_loop.py uses generate_html() for live report generation during the optimization loop, so this helper is part of the skill-creator reporting path.

Steps to reproduce

Validation level: dynamic reproduction plus source-control-flow inspection.

import importlib.util
from pathlib import Path

path = Path("skills/skill-creator/scripts/generate_report.py")
spec = importlib.util.spec_from_file_location("generate_report", path)
generate_report = importlib.util.module_from_spec(spec)
spec.loader.exec_module(generate_report)

generate_report.generate_html({"history": []})

Observed output:

ValueError: max() iterable argument is empty

Expected behavior

generate_html({"history": []}) should return a valid HTML report with summary information and an empty-state message, or at least a valid report shell that says no iterations have been recorded yet.

Actual behavior

The function raises ValueError at the best-iteration selection step:

max() iterable argument is empty

Existing coverage

I searched current issues and PRs for generate_report.py, generate_html, history, empty, ValueError, max() iterable, and Skill Description Optimization. I did not find an existing issue or PR that covers this root cause.

Suggested fix

Add an early empty-state branch before computing best_iter. For example:

  • if history is empty, render the summary, legend, and a one-row/paragraph empty-state message;
  • skip best-row highlighting when there is no iteration;
  • keep the existing behavior unchanged for non-empty history.

Suggested tests

  • Add a unit test for generate_html({"history": []}) that asserts it returns valid HTML and contains an empty-state message.
  • Add a test for the existing non-empty path to ensure best-row highlighting still works.
  • Add a live-report smoke test that renders before the first optimization iteration completes.

Submitted with Codex.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions