Skip to content

prompt: tell the agent to leave the measurement setup alone - #58

Open
ArjunS07 wants to merge 2 commits into
survey/phase1-template-20261008from
feat/prompt-leave-measurement-setup
Open

ArjunS07 wants to merge 2 commits into
survey/phase1-template-20261008from
feat/prompt-leave-measurement-setup

Conversation

@ArjunS07

@ArjunS07 ArjunS07 commented Oct 9, 2026 •

Copy link
Copy Markdown
Contributor

Purpose

Every Codex patch for bottleneck#309 was rejected because it fixed a bug in an ASV benchmark file. The prompt did not say that these files are off limits. This change tells the agent that these files are restored before measuring (datasmith#59), so edits there are thrown away.

How it works

final.md.j2 renders instruction.md for each task. One line is added to Operating Principles, after "Correctness first":

* **Leave the measurement setup alone**: the ASV benchmark files, `asv.conf*`, `conftest.py` and `.github/` are restored to their original state before your change is measured, so edits there are thrown away. Speed up the library code instead.

Changes

File Effect Before
src/datasmith/github/templates/final.md.j2 The prompt names the files that are restored before measuring The prompt did not mention them

Verification

  • pytest tests -k "render or template or prompt" --ignore=tests/docker: 67 passed.

Notes

  • Only newly rendered tasks get the line. Rendered tasks keep their instruction.md until they are rendered again or edited.

@atharvas

atharvas commented Oct 9, 2026

Copy link
Copy Markdown
Member

Looks good to me.

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants