A follow-up that reworks the previous answer skips the search - #193
Merged
Merged
Conversation
"summarize" after an answered question ran the whole pipeline and wrote a second, different answer: the answer prompt never saw the previous text and was told not to restate it, so it answered "summarize" as a question. The understanding step now names a rework (summarize, shorter, as a table, in any of the four languages). The pipeline leaves before the search and answers from the previous answer and its own persisted sources, markers stripped from the old text so the model cites again from the list. The handler reads those sources for every follow-up. A basis a re-index took a chunk of is refused with its own line, never summarised from survivors and never searched afresh. A first turn, or an antecedent without a basis, runs the ordinary path and records no rework intent. The re-explain of a rework row reworks the same antecedent again instead of re-answering the instruction over the sources.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
"summarize" after an answered question ran the whole pipeline and wrote a second, different answer. The answer prompt never saw the previous text (AGENTS.md:58, by design) and was told not to restate it, so it answered "summarize" as a fresh question over re-retrieved sources.
What changed:
Threadcarries the previous answer's sources;handleAskreads them for every follow-up.Runexits before the search on a rework and answers from the previous text plus those sources, markers stripped outside fences so the model cites again from the numbered list.ErrBasisGone), never summarised from survivors, never searched afresh. A first turn or a "nothing found" antecedent runs the ordinary path and stores no rework intent.Checked live on the sock-shop flow corpus with the real gate and answer models: "summarize" and "as a table" take the lane (understanding, answering, writing; 94 sources in the prompt), a "why" follow-up still searches, "kuerzer" cuts a 3425-char German answer to 1209 with every claim cited, the audience toggle on the summary row reworks again.
Deferred: the previous answer's prose in every ordinary follow-up's answer prompt reverses AGENTS.md:58 and cannot be measured, the eval has no two-turn cases yet.