Repository navigation
Read narrated workflow results without repeating completed work #1014
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Merged
Merged
Changes from all commits
Commits
Show all changes
2 commits
Select commit
Hold shift + click to select a range
File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
69 changes: 69 additions & 0 deletions
69
examples/adversarial-implementation/components/Adversarial/Response.md
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,69 @@ | ||
| --- | ||
| description: 'Validate a saved foundation reply and bound response-only corrections in its original session. Use <Adversarial.Response assignment={assignment} raw={raw} as="accepted" />.' | ||
| props: | ||
| type: object | ||
| properties: | ||
| assignment: {type: object} | ||
| raw: {type: string} | ||
| required: [assignment, raw] | ||
| additionalProperties: false | ||
| returns: {type: object} | ||
| --- | ||
|
|
||
| <Adversarial.Schema as="schemas" /> | ||
| <Let as="raw" value={props.raw} /> | ||
| <Let as="rawPath" value={props.assignment.attempt + "/raw.txt"} /> | ||
| <Let as="round" value={0} /> | ||
| <Phase.Extract text={raw} as="extracted" /> | ||
| <If condition={!extracted.ok}> | ||
| <Dir path={props.assignment.attempt}><File path="response-rejection-0000.json"><Json value={extracted} /></File></Dir> | ||
| <Fail message="No unambiguous substantive foundation result is established. Preserve raw.txt and response-rejection-0000.json; no productive prompt is repeated." /> | ||
| </If> | ||
| <SafeParse schema={schemas.foundation} as="checked">{extracted.json}</SafeParse> | ||
| <If condition={!checked.ok}> | ||
| <Dir path={props.assignment.attempt}><File path="response-rejection-0000.json"><Json value={checked} /></File></Dir> | ||
| <Let as="relaxed" value={{...schemas.foundation,properties:{...schemas.foundation.properties,summary:{type:"string",minLength:1},handoff:{type:"string",minLength:1}}}} /> | ||
| <SafeParse schema={relaxed} as="established">{extracted.json}</SafeParse> | ||
| <If condition={!established.ok}><Fail message="No unambiguous substantive foundation result is established. Preserve raw.txt and response-rejection-0000.json; no productive prompt is repeated." /></If> | ||
| <Let as="frozen" value={{...schemas.foundation,properties:Object.fromEntries(Object.entries(schemas.foundation.properties).map(([key,schema]) => [key,(key === "summary" || key === "handoff") && established.value[key].length > schema.maxLength ? schema : {const:established.value[key]}]))}} /> | ||
| <Loop max={3}> | ||
| <If condition={checked.ok}><Break /></If> | ||
| <Let as="round" value={round + 1} /> | ||
| <Let as="directory" value={props.assignment.attempt + "/response-corrections/" + round} /> | ||
| <Dir path={props.assignment.attempt + "/response-corrections"}><Let as="parentPrepared" value={true} /></Dir> | ||
| <Let as="quoted" value={"'" + directory.replaceAll("'","'\"'\"'") + "'"} /> | ||
| ```bash exec as="reserved" | ||
| mkdir -- {quoted} | ||
| ``` | ||
| <If condition={reserved.exitCode !== 0}><Fail message="Preserve the occupied response correction; do not repeat its prompt." /></If> | ||
| <Let as="packet" value={{ordinal:round,remaining:3-round,raw:rawPath,rejection:checked.errors,established:established.value,resultSchema:frozen}} /> | ||
| <If condition={JSON.stringify(packet).length > 64000}><Fail message="The response correction packet exceeds its input bound; preserve it without dispatch." /></If> | ||
| <Dir path={directory}><File path="request.json"><Json value={packet} /></File></Dir> | ||
| <Ticket.Checkout repository={props.assignment.repository} as="before" /> | ||
| <If condition={!before.clean || before.head !== established.value.revision}><Fail message="Reconcile source before response-only correction." /></If> | ||
| <Dir path={props.assignment.repository}> | ||
| ```bash exec as="statusBefore" | ||
| git status --porcelain=v1 --untracked-files=all | ||
| ``` | ||
| </Dir> | ||
| <If condition={statusBefore.exitCode !== 0 || statusBefore.stdout !== ""}><Fail message="Reconcile tracked and untracked source before response-only correction." /></If> | ||
| <Dir path={directory}><File path="source-before.json"><Json value={statusBefore} /></File></Dir> | ||
| <Dir path={props.assignment.repository}><Adversarial.PlannerRole assignment={{...props.assignment,initialize:false}} guide="" correction={packet} as="raw" /></Dir> | ||
| <Let as="rawPath" value={directory + "/raw.txt"} /> | ||
| <Dir path={directory}><File path="raw.txt">{raw}</File></Dir> | ||
| <Ticket.Checkout repository={props.assignment.repository} as="after" /> | ||
| <Dir path={props.assignment.repository}> | ||
| ```bash exec as="statusAfter" | ||
| git status --porcelain=v1 --untracked-files=all | ||
| ``` | ||
| </Dir> | ||
| <Dir path={directory}><File path="source-after.json"><Json value={statusAfter} /></File></Dir> | ||
| <If condition={!after.clean || after.head !== before.head || statusAfter.exitCode !== 0 || statusAfter.stdout !== statusBefore.stdout}><Fail message="Response-only correction changed source. Preserve its effects and hold." /></If> | ||
| <Phase.Extract text={raw} as="extracted" /> | ||
| <Let as="checked" value={{ok:false,input:raw,errors:extracted.ok ? [] : [extracted.rejection]}} /> | ||
| <If condition={extracted.ok}><SafeParse schema={frozen} as="checked">{extracted.json}</SafeParse></If> | ||
| <Dir path={directory}><File path="disposition.json"><Json value={{round,accepted:checked.ok,raw:rawPath,errors:checked.ok ? [] : checked.errors}} /></File></Dir> | ||
| </Loop> | ||
| <If condition={!checked.ok}><Fail message="Three response-only corrections are exhausted. Preserve their receipts; no productive prompt is repeated." /></If> | ||
| </If> | ||
| <Return value={{result:checked.value,round,rawPath}} /> |
25 changes: 0 additions & 25 deletions
25
examples/adversarial-implementation/components/Phase/Extract.md
This file was deleted.
Oops, something went wrong.
157 changes: 157 additions & 0 deletions
157
examples/adversarial-implementation/components/Phase/Extract.ts
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,157 @@ | ||
| import { parseJson } from "@executablemd/core"; | ||
| import type { Json } from "@executablemd/core"; | ||
| import { Err, Ok } from "effection"; | ||
| import type { Operation, Result } from "effection"; | ||
|
|
||
| export const props = { | ||
| type: "object", | ||
| properties: { text: { type: "string" } }, | ||
| required: ["text"], | ||
| additionalProperties: false, | ||
| }; | ||
| export const returns = { type: "object" }; | ||
|
|
||
| export default function* Extract(props: Record<string, Json>): Operation<Json> { | ||
| const input = props.text; | ||
| if (typeof input !== "string") { | ||
| throw new TypeError("Phase.Extract requires text."); | ||
| } | ||
| const parsed = readResult(input); | ||
| if (!parsed.ok) { | ||
| return { | ||
| ok: false, | ||
| rejection: { | ||
| kind: "response_validation", | ||
| code: "extraction", | ||
| message: parsed.error.message, | ||
| issues: [], | ||
| }, | ||
| }; | ||
| } | ||
| return { ok: true, json: JSON.stringify(parsed.value) }; | ||
| } | ||
|
|
||
| function parseText(text: string): Result<Json> { | ||
| try { | ||
| return Ok(parseJson(JSON.parse(text))); | ||
| } catch (error) { | ||
| return Err(error); | ||
| } | ||
| } | ||
|
|
||
| function opensRecord(text: string): boolean { | ||
| return ( | ||
| /^\s*\{\s*(?:"|}|$)/.test(text) || | ||
| /^\s*\[\s*(?:\{|\[|"|\]|(?:-?(?:0|[1-9]\d*)(?:\.\d+)?(?:[eE][+-]?\d+)?|true|false|null)(?=\s*(?:,|\]|$))|$)/.test( | ||
| text, | ||
| ) | ||
| ); | ||
| } | ||
|
|
||
| function readResult(input: string): Result<Json> { | ||
| const whole = parseText(input.trim()); | ||
| if (whole.ok) { | ||
| return whole; | ||
| } | ||
| const candidates: Json[] = []; | ||
| let outside = ""; | ||
| let fence: { marker: string; json: boolean; unlabeled: boolean; body: string } | undefined; | ||
| for (const line of input.split(/(?<=\n)/)) { | ||
| const delimiter = | ||
| fence === undefined | ||
| ? /(`{3,}|~{3,})([^\r\n]*)/.exec(line) | ||
| : /^ {0,3}(`{3,}|~{3,})([^\r\n]*)/.exec(line); | ||
| if (fence !== undefined) { | ||
| if ( | ||
| delimiter && | ||
| delimiter[1][0] === fence.marker[0] && | ||
| delimiter[1].length >= fence.marker.length && | ||
| delimiter[2].trim() === "" | ||
| ) { | ||
| if (fence.json || (fence.unlabeled && opensRecord(fence.body))) { | ||
| const parsed = parseText(fence.body); | ||
| if (!parsed.ok) { | ||
| return parsed; | ||
| } | ||
| candidates.push(parsed.value); | ||
| } | ||
| fence = undefined; | ||
| } else { | ||
| fence.body += line; | ||
| } | ||
| } else if (delimiter) { | ||
| if (delimiter[2].includes(delimiter[1])) { | ||
| outside += line; | ||
| continue; | ||
| } | ||
| const language = delimiter[2].trim().toLowerCase(); | ||
| fence = { | ||
| marker: delimiter[1], | ||
| json: language === "json", | ||
| unlabeled: language === "", | ||
| body: "", | ||
| }; | ||
| outside += line.slice(0, delimiter.index) + "\n"; | ||
| } else { | ||
| outside += line; | ||
| } | ||
| } | ||
| if (fence !== undefined) { | ||
| return Err(new Error("The agent reply contains an unclosed code fence.")); | ||
| } | ||
| for (let index = 0; index < outside.length; index += 1) { | ||
| if ( | ||
| (outside[index] !== "{" && outside[index] !== "[") || | ||
| /[A-Za-z0-9_$]/.test(outside[index - 1] ?? "") | ||
| ) { | ||
| continue; | ||
| } | ||
| if (!opensRecord(outside.slice(index))) { | ||
| continue; | ||
| } | ||
| const end = recordEnd(outside, index); | ||
| if (!end.ok) { | ||
| return end; | ||
| } | ||
| const parsed = parseText(outside.slice(index, end.value)); | ||
| if (!parsed.ok) { | ||
| return parsed; | ||
| } | ||
| candidates.push(parsed.value); | ||
| index = end.value - 1; | ||
| } | ||
| if (candidates.length !== 1) { | ||
| return Err(new Error("The agent reply must contain exactly one unambiguous JSON result.")); | ||
| } | ||
| return Ok(candidates[0]); | ||
| } | ||
|
|
||
| function recordEnd(text: string, start: number): Result<number> { | ||
| const closings: string[] = []; | ||
| let quoted = false; | ||
| let escaped = false; | ||
| for (let index = start; index < text.length; index += 1) { | ||
| const character = text[index]; | ||
| if (quoted) { | ||
| if (escaped) { | ||
| escaped = false; | ||
| } else if (character === "\\") { | ||
| escaped = true; | ||
| } else if (character === '"') { | ||
| quoted = false; | ||
| } | ||
| } else if (character === '"') { | ||
| quoted = true; | ||
| } else if (character === "{" || character === "[") { | ||
| closings.push(character === "{" ? "}" : "]"); | ||
| } else if (character === "}" || character === "]") { | ||
| if (character !== closings.pop()) { | ||
| return Err(new Error("The agent reply contains an incomplete JSON result.")); | ||
| } | ||
| if (closings.length === 0) { | ||
| return Ok(index + 1); | ||
| } | ||
| } | ||
| } | ||
| return Err(new Error("The agent reply contains an incomplete JSON result.")); | ||
| } | ||
Oops, something went wrong.
Oops, something went wrong.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Blocking: unfenced
{/[runs that are not JSON reject the whole reply, so ordinary narration around one valid result holds instead of being accepted.The scan at line 84 treats every
{or[not glued to an identifier character as a result candidate, and a parse failure here returns the error for the entire reply. Prose that is not JSON therefore vetoes a reply that contains exactly one well-formed record. Running this file against twelve reply shapes (imports stubbed, function unchanged) gave:{"passed":true}Saved the receipt under [receipt](receipt.json)."[receipt]" is not valid JSON- [x] focused tests passed"[x]" is not valid JSONChecks [1/3] done.Expected ',' or ']' after array elementKept the {quoted} placeholder.Expected property name or '}'Ran it (see ```log``` above).Done./ fencedjson/ trailing narration /phase-result.v2inlineConsequence: the README promise on this PR, "One narrated or fenced JSON result is accepted without another agent call", is not met for Markdown links, task-list checkboxes, bracket counters or brace placeholders, all routine in agent narration. Because
Phase.ResponseandPhase.Correctshare this component, the hold also reaches the phase pipeline, not onlyAdversarial.Foundation. The replaced.mdextractor keyed on the single line-start{and never inspected prose, so these four shapes were accepted before this change; now each costs a completed productive session a manual hold.Recommendation: only treat a run as a candidate when its opening looks like a JSON record, for example
{followed by optional whitespace then"or}, or[followed by optional whitespace then{,[,",], a digit,-,t,forn; otherwise continue scanning. Keep the existing ambiguity and incomplete-record errors for real candidates. Every rejection case in the new tests still rejects under that rule, because each begins with{"or is a duplicate record. Add the link and checkbox shapes to the acceptance test so the contract is held.