[ArjunS07]

Purpose

formula-code/formulacode-verified-rl#11 (merged) changes the meaning of difficulty in rl_task_records.jsonl. When difficulty_source is set, difficulty is a curriculum bucket (very_hard, hard, balanced, easy), and the datasmith label (easy, medium, hard, unknown) moves to difficulty_datasmith. Without this PR, regen render writes the curriculum bucket into task.toml and tests/config.json. This PR makes regen keep the datasmith label.

How it works

flowchart TD
  A["records row"] --> B{"difficulty_datasmith set?"}
  B -- yes --> C["use difficulty_datasmith"]
  B -- no --> D{"difficulty_source set?"}
  D -- yes --> E["drop difficulty, so the record default applies"]
  D -- no --> F["use difficulty (old schema)"]

Loading

src/datasmith/harbor_adapter/regen.py      record_from_config picks the datasmith label
tests/docker/test_regen_difficulty.py      new: 3 test cases for old and new schema rows

Changes

1. record_from_config uses the datasmith label.
   • It uses difficulty_datasmith when the row has it.
   • If the row has only a curriculum difficulty (with difficulty_source), it drops that value, so the record default applies.
   • Effect: tasks that you render again keep the datasmith label for both schemas.
   • Before: a new schema row wrote difficulty = "balanced" (or another curriculum bucket) into task.toml and config.json.
2. New tests.
   • tests/docker/test_regen_difficulty.py has one parametrized test with 3 cases: an old schema row, a new schema row with difficulty_datasmith, and a new schema row without it.
   • Effect: a test fails if the curriculum bucket gets into rendered tasks again.
   • Before: no test covered the difficulty from the records file.

Verification

These results are from when this PR was based on #30:

• pytest -m "not slow" tests: 1682 passed, 10 skipped. The new tests are the difference from Make harbor_adapter the only FormulaCode task template (nested layout, baked image, baseline fingerprint, harbor imports it) #30.
• A real row (TileDB-Inc/TileDB-Py#1005, difficulty easy) from verified-rl's rl_task_records.jsonl was rendered with the old and the new code.
  • task.toml is identical byte for byte.
  • The full task directory is the same, except render_stamp.json (timestamp and dirty flag).
• The same row was converted to the new schema (difficulty=balanced, difficulty_source=curriculum_spec, difficulty_datasmith=easy) and rendered.
  • The task directory is the same as the old schema render, except render_stamp.json.

Notes

• The base branch formulacode/canonical-task-template is stale. This PR should be retargeted to main.
  • Make harbor_adapter the only FormulaCode task template (nested layout, baked image, baseline fingerprint, harbor imports it) #30 is closed. Its commits are in main, except one parser.py note that is in task template v1: the Phase 1 template and its fixes in one branch, for one merge into main #65.
  • The diff of this PR against main has only the 2 files above.
• Companion to formula-code/formulacode-verified-rl#11, which is merged.
• records.to_record (Supabase pull_requests rows) does not change, because those rows never have the curriculum fields.