Repository navigation
Conversation
- Claim eligibility: check v37 asks each substantive claim for its own eligibility; a closed class of future/intention/commitment markers (PLAN_MARKERS) obliges focus-level claim_checks. Code binds the verdicts: any ineligible claim or any plan rejects the sentence through the existing repair/delete path; missing or malformed verdicts count as ineligible. - Frequency strength: FREQUENCY_STRENGTH ranks a closed class of frequency and universal words (偶尔<有时<经常<通常/习惯<总是/都/每次/从不) inside the existing quantifier_checks contract, plus source_instances. A candidate may not outrank its source word; instances alone support only a bounded experience (那两次……). Hedges never lower a rank. - Output limits: a truncated one-sentence window is split by focus with the whole sentence and sources kept; an unsplittable request whose truncated reply is an unclosed JSON object or blank padding is retried once while the stage can still hold it. Truncation records note their handling; new focus_splits/output_retries counts reach the evaluation report. - Prompts: persona_check_v37, persona_generate_v24; older files kept.
- Retry: an empty, blank or unclosed truncated reply of an unsplittable request is asked once more, without the timeout allowance and with a deadline that leaves a 60 s stage reserve. Any failure of the retry restores the original truncation (terminal / current kept), never a stage failure; degree and handwritten review included. A focus split needs the same room in the stage, else the truncation is terminal. - Counting: truncations of a window that ends complete (retry, structure reask or split) are recovered; only an incomplete window counts terminal. Handwritten review gets its own diagnostics, summed in the report. - Frequency: candidate and source share one reading; negated words and a distributive 都 (adjacent quantifier, plural pronoun, bounded set) are no claim or evidence. Evidence comes from utterances only. A bounded experience needs 这/那 + count + 次/回/场/遍/天 before the word, not opening a period, within source_instances. Adds 每天 天天 每晚 每周 每月 每年 全都. - Claims: PLAN_MARKERS drop 会 要 想要 希望 但愿 愿意; only marked requested foci owe claim_checks; a missing or incomplete contract is asked again once, then rejected. - Prompts v37/v24 edited in place (unmerged, never run formally).
- Recovery budget: a retry or focus split runs only if its estimate (one request, or one per focus, as long as the truncated one) plus the windows still to run (each as the longest seen in the stage, degree included; the handwritten review also counts its remaining pages and the check after it) plus the 60 s reserve fits. Halves end before the reserve; any failed half (timeouts after their retry, 429, transport, truncation) restores the original truncation as terminal instead of raising. - 都: read from everything before it in its clause. Sets of times or occasions keep it a frequency claim; a universal phrase (每次排练, 每个人, 任何人) takes it as its correlative; bounded sets of people or things, including a coordination with names or pronouns right before it, make it distributive; 一样 is no numeral and classifier. - Bounded counts: 那次 counts one; any period opener between the count and the word unbounds it. - source_quantifier must be one closed-class word: otherwise asked again once, then read as no frequency evidence. - Evaluation: a failed output retry's own timeout is tolerated (not recovered) at a retained checkpoint, only as the next same-purpose call after the truncation and one per failed retry; retry errors are reported. - Prompt check v37: source_quantifier names one word.
- 都/全都 is a frequency or universal claim except in three closed forms: a plural pronoun or group word right before it; a numeral (or 这/那 + numeral) with a closed person-or-thing classifier before it in its clause; X 和/跟/与/同 Y right before it, X and Y pronouns or names/aliases of subjects in this material (code-only, not sent to the checker). The occasion, set and text-coordination heuristics are removed. - Only 都 can be bounded: 这/那 + count + 次/回/场/遍/趟 right before it, only a subject pronoun or adverb between, no period word anywhere in the clause. 总是/每次/一直 and the rest always need a source frequency word. - A recovery leaves room for unrun sibling sub-batches of a split; a failed sentence-split child restores the original truncation as terminal. - A reask for one bad source word keeps the other items' adverse verdicts. - Evaluation: truncation records carry their call order; only the call right after the truncation whose record says its retry failed by timeout is tolerated, records deduplicated across phases, none if counts disagree. - Prompts v37/v24 state the narrower bounded rule.
…退、拆分失败超时按记录容忍 Tests follow in the next commit.
Counted groups need two or more and no each-clause; each 都 needs its own count; a bare source 都 is no evidence; pauses are never absorbed by a fallback; a failed focus split's own timeouts are tolerated by record; subject names never reach a model request.
- A counted group is one fixed shape right before 都: 这/那? + two or more (or 几) + 个人, or a person/thing classifier with a noun of at most two characters (one adjective may precede it, no time, place or action character), then at most one pronoun and one adverb. Anything else between the count and 都 makes it a frequency claim. - item_error and content_rejection of the one request are contained by a retry or split fallback; configuration, key, account, pause, usage limit and call budget errors keep their own path. - Check prompt v37: a bare 都 of the source is no frequency evidence.
- COUNTED_GROUP: a count right after 第 or another numeral is no group, so an ordinal (第二本书我都) stays a frequency claim. - A bare 每 of the source with a later 都 in its clause (每遇到新朋友我都先问) is explicit universal evidence; 每 is a legal source_quantifier and asks for no reask. A lone 都, a 每 without its 都 or a negated 每 is still none.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
PE-23:persona 生成与检查的三类通用修复。只改提示词和确定性检查,没有调用真实模型;真实评测和判分不在本 PR 内。之后的提交按规划者五轮审查意见修改。
提示词
persona_check_v37(SHA-2560e33db0b7676d971ef3d4a2940edc1e13a3a9aca26e01b89929d030f349091b4,18699 字节):逐命题收录资格与claim_checks;频率强度并入quantifier_checks(source_instances、强度序、只认原话里被肯定的频率词、只有实例时写成有计数的有界经历)。比 v36 多约 2.6 KB;source_quantifier要求只填一个词,长依据句的材料字节余量相应减少(按决定不改,评测时观察)。persona_generate_v24(SHA-256eab1d23d1b9c283da2b1d6a2a20037c36814fe343442ac53f7cf4dc9510ffbcc):保留当次认识时省去随附的未来打算;不把有限次数写成“都/总是/每次”。1. 逐命题收录资格
PLAN_MARKERS(将来、打算、承诺的时间词和动词,封闭语法类,同 整理:保留限定词并精确校验结构化主体引用(含 GLM 评测) #107 先例):以后 今后 往后 日后 将来 下次 下回 下一次 下一回 改天 届时 到时 到时候 将要 将会 打算 准备 计划 预备 决定 承诺 保证 答应 发誓 立志 争取 尽量 力争 务必。单字“会、要”和希望、愿意这类词不作标记。正反问句(打不打算)不算标记。plan_markers的焦点必须返回claim_checks;模型多返回的不带标记的分句不要求。缺失、格式不符或漏项先重问一次,仍缺按不合格处理。plan一律不合格;任一项不合格,整句走既有修复/删句路径,不被其他命题覆盖。负结论也进_adverse_field_verdicts。2. 频率与断言强度
FREQUENCY_STRENGTH:偶尔<有时<经常<通常/习惯<总是/都/每次/从不,五级封闭词表,含每天、天天、每晚、每周、每月、每年、全都;单字“常、总”和“一般”有歧义,不加。与SCOPED_QUANTIFIERS共用quantifiers/quantifier_checks,频率项另含source_instances;非频率量词多给的这个字段忽略。_focus_quantifiers:被否定的频率词(不经常、不是每次、从没……每次)不算。source_quantifier只能是词表中的一个明确频率词,来源里单独的“都”(这些歌我都听过、每个人我都问了)不算依据;填整段或多个词时按结构错误重问一次,仍不合格按无依据处理。频率依据只取原话;当前正文只能收窄,没有摘录才用正文。修复替换句复查时同样核对。source_instances。总是、每次、一直等其他频率词没有来源频率词时一律不通过,任何计数都不能代替;“那次直播里我一直耐心解释”也因此被拦(已接受的保守取舍)。source_quantifier不合格触发重问时,同一焦点其他量词项已算出的不利结论随重问保留。3. 截断后的恢复
_check_focus_split:只剩一句的窗口截断时按焦点二分,子请求带完整句子与全部绑定来源,直到单焦点;合并后缺焦点结论不能通过。子请求截止时间扣掉 60 秒保留时间;任一子请求失败(超时及其超时重试、429、传输错误、仍截断)都退回原来的截断,按终端截断/保留当前版处理,不抛子请求的错误。truncations、splits、split_results、terminal_truncations和retained_incomplete_checkpoints_v2的口径。recovered_truncations、focus_splits、focus_split_results、output_retries、output_retry_results。output_limit_diagnostics,并计入评测汇总。评测代码(报告计数部分)
persona_evaluation.py只改计数:_check_output_limit_metrics汇总新字段(含手写审查和output_retry_errors)。_model_call_outcomes按窗口是否最终完整判断截断调用是否已恢复(旧记录仍认按句拆分的恢复)。truncation_events,跨阶段按顺序号去重),与检查点里按顺序排列的截断调用一一对应;只有记录写明重试因超时失败的那条截断,其紧随的同用途超时调用才被容忍,每个失败重试至多一条,按焦点拆分因超时失败时,只容忍记录在该次拆分失败子请求名下的那几次超时调用(含它自己的超时重试;超时与截断共用调用顺序号),只在它与下一条截断调用之间找;按句拆分的兄弟子批超时不容忍。数目对不上时一条都不容忍,仍占每轮不完整额度;calls新增tolerated_timed_out_calls。其他超时口径不变(例如拆分子请求的普通超时即使靠超时重试恢复,在不完整检查点里仍判无效)。已知限制
source_instances仍采信模型填写。tolerated_as标注按顺序关联,可能对不上具体是哪一次调用(容忍的条数是对的)。persona_check_v37已 18699 字节,长依据句的单次请求离 32768 字节上限更近。tests/test_persona_streaming.py::test_future_deadline_closes_stream_worker_and_preserves_shared_client时间敏感:在未改动的 origin/main 上也会失败,本机时过时不过;与本 PR 无关,由规划者另开任务。不变
手写审查规则、发布流程、
DEFAULT_PUBLISH_MODE=all_manual、评分说明和评测语料都没有改;没有新增模型用途。测试
新增
test_persona_claim_eligibility.py、test_persona_frequency_strength.py、test_persona_focus_split.py,每条修改都有通过和拦截两个方向。persona_fakes的协议假模型补齐claim_checks与source_instances(仍不判断资格或强度);几处既有测试按新版本号、按焦点拆分的调用次数与新计数字段更新。本次(
0025f86)结果:3.13 6118 passed、1 failed;3.12(--locked --isolated)6118 passed、1 failed;两边失败的都是上面那个 streaming 测试。uv build通过。