Repository navigation
fix(bundle): a bundle rewrites the ids a step's context paths name, as it does every other id - #653
Merged
Merged
Conversation
…s it does every other id A step type declares the fields that hold paths into a run's context (TaskStep.context_paths), and the bundle resolver renames each id such a path names to the one the bundle gives that entity, before the run. A renamed id that a dotted path can't hold is bracketed: env_trigger_state[@local/<root>/<env>].statuses.wf-1. - rubrics_verifier and always_on_agent_verifier declare context_facts_for_judge, whose lookup is plain again. - bundle register renames such ids to registry ids, and refuses an @Local id left inside a path. - A path naming a renamed id in a field its step doesn't declare is warned of when the bundle is resolved. - Run.firings(env=...) and Run.firing(..., env=...) take an env as the run's task names it. - A task written before this is written again: its resolved config, which the ledger hashes, changed. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…r names it A task's import rewrites each id a field of its steps names to the name the folder gives that entity, numbered when the name is taken. A declared path into the run's context now takes the same name, so a fact naming an env the folder writes as crm-env-2 names it so. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
… env fields, and nothing else in it changes A path is rebuilt from the one the task wrote, after each place that rewrites a task's env fields: resolving, pointing an installed alias, repointing a variant, and importing. Only the segment that names an env changes: the one after a key whose entries are keyed by env id (ENV_KEYED_METADATA). A key that only shares an entity's name, such as a verifier's id, is kept, and an env named by an installed alias is named by what the alias names. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…ds a path back from before masking - ENV_KEYED_METADATA records where each key's entries name an env: the segment after env_trigger_state, env_changes, env_trajectory and live_websites, and env_skills' third segment, after the agent's name. - A path is written only when its env is renamed. - An import leaves out scoped fields and masks credentials before it keeps the steps a path is rebuilt from, so a credential masked in a path stays masked when the path's env is renamed. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What and why
#641 records the ids a bundle gives the entities a task names (
Task.entity_ids). Context facts then looked a missed key up through that map at run time. Only code that calls the helper got that translation, so every other reader of a task author's ids would have needed the same call.The framework already has a general mechanism for this. A step type declares the fields that name entities (
entity_refs), and the bundle resolver rewrites them before the run; nothing at run time reads the declarations. Context-fact paths escaped it only because a path isn't an id field. This change brings them in.TaskStep.context_pathsnames the fields that hold paths into a run's context. Likeentity_refs, it uses[]/{}for each element or value and isn't inherited.rubrics_verifierandalways_on_agent_verifierdeclarecontext_facts_for_judge{}.sync_context_pathsrebuilds each declared path from the one the task wrote, giving its env the id the task's env fields hold now. It runs after each place that rewrites those fields: resolving, pointing an installed alias, repointing a variant, and importing. Running it again changes nothing.ENV_KEYED_METADATArecords where each metadata map names an env: the segment afterenv_trigger_state,env_changes,env_trajectoryandlive_websites, and the third segment ofenv_skills, after the agent's name. Only a path whose env is renamed is written. A key that only shares an entity's name is kept, such assolverinverifications.solver.scorewhen an agent is calledsolver, or an agent named like an env inenv_skills.@,/and~, so a segment may be an id in brackets, for exampleenv_trigger_state[@local/~/runs/pn/world].statuses.wf-1. A plain dotted path reads as before.context_path_segmentsandrenamed_context_pathinagent_env.task_step.contextparse and rewrite paths.entity_idsis gone. The judge sees each fact's label and value, never its path, so its prompt doesn't change.bundle registerrenames such ids to registry ids too, and refuses an@localid left inside a path.@localids in its paths, and a fact whose env was renamedcrm-env-2still namedcrm-env. The import leaves out scoped fields and masks credentials before it keeps the steps a path is rebuilt from, so a masked credential stays masked.tickets.jsonisn't reported.Run.firings(env=…)andRun.firing(…, env=…)take an env as the run's task names it.Run data keeps the ids the run used, so one id names one entity everywhere.
Task.entity_idsstays, for tools that read a stored run.How it was tested
Unit:
tst/unit/task_step/context_path_test.py:tst/unit/bundle/resolve_test.py:env_skillsat its third segment;env_skillsagent named like the env;tst/unit/bundle/installed_entities_test.py: a fact naming an env by an installed alias names the env the alias names. The test fails without the change to planning.tst/unit/bundle/variants_test.py: a fact naming the env by id is repointed at the variant.tst/unit/bundle/register_test.py: a bracketed@localid in a path takes its registry id, and an@localid left in a path is refused.tst/unit/bundle/importer_test.py:crm-env-2, each fact names its own env's folder;Each test fails without its importer change.
tst/unit/bundle/ledger_test.py: a task written by a build that didn't rename its paths is written again ("config changed"), then reused.tst/unit/task_step/verifiers/test_rubrics_verifier_context_facts_bundle_ids.py:tst/unit/task_step/verifiers/test_run_context.py:firings(env=…)finds a renamed env by the id the task names it by.tst/unit/task_step/test_entity_refs.py:context_pathsname real fields;field_sitesreaches each value or element of a last{}or[].tst/unit/bundle,tst/unit/taskandtst/unit/task_steppass: 4,565 passed, 14 skipped.End to end. A benchmark task with a scripted world (timed events and colleague agents) was run as a bundle with
agent-env runon a Linux VM. The build combined this branch with #640, so a validity verifier can fail the task.A dry run of a folder an earlier revision of #641 wrote planned only its task anew:
v2 (its stored version doesn't record the ids the bundle gave what it names; config changed). The first reason is there because an earlier revision of #641 wrote that folder. Its envs and artifacts wereunchanged.env_trigger_state[@local/…/<env>]v-worldfailed, and the run failed at validity withVerifierNotPassingErrorIn both runs no fact rendered "(not available)", and the facts match the trigger statuses the run stored under the namespaced env id.
Import round trips and adversarial cases, on the same VM against its real local store. Each case checks that every
@localid in a resolved fact path is an env id the resolved task deploys:-2verifications.coopbench-star-pm-v2.score, a verifier key sharing the env's namechaos.dot.v2)coopbench-star-pm-v2.json🤖 Generated with Claude Code


Confidence Score: 5/5
The PR appears safe to merge; no new blocking issues were found.Summary
The PR rewrites declared context paths when bundles rename environments, including during import, registration, alias lookup, and variant creation.
Diagram
%%{init: {'theme': 'neutral'}}%% flowchart LR A[Authored task paths] --> B[Rewrite declared env fields] B --> C[Find each env's new ID] C --> D[Rebuild declared context paths] D --> E[Store task] E --> F[Read facts exactly as written]Reviews (8) · Last reviewed commit: "fix(bundle): env_skills paths name their..." · Reviewed by Greptile