Skip to content

fix(validate): stop reading capability words as description-no-when cues - #743

Open
xaleryb wants to merge 6 commits into
intel:mainfrom
xaleryb:fix/validate-when-cue-followups
Open

xaleryb wants to merge 6 commits into
intel:mainfrom
xaleryb:fix/validate-when-cue-followups

Conversation

@xaleryb

@xaleryb xaleryb commented Nov 14, 2026 •

Copy link
Copy Markdown
Contributor

Three cues from #454 matched words that state a capability, so a description with no "when" clause lost its description-no-when warn:

  • ja: いつ matched inside 思いつく, 追いつく, 食いつく and こいつ; it now counts only before a use verb (いつ使う).
  • zh: 耗时/超时/延时 before 调用/执行 matched the 时+verb cue, and 当事人 opened a 当…时 span; both are excluded, 当请求超时 still counts.
  • de: bare beim matched "Hilft beim ..."; it now needs verwenden/nutzen/benutzen/einsetzen/aufrufen in the same sentence.

Tests: new NO_WHEN and HAS_WHEN cases in tests/test_description_when_cues.py; reverting each cue change fails its own cases (4, 4, 2).

docs/external-content.md now says which capability words are not when cues.

A bare いつ also matched inside 思いつく, 追いつく, 食いつく and こいつ, so a
Japanese capability-only description lost the description-no-when warn.
Accept いつ only when a use verb follows (いつ使う, いつ呼び出す, ...).

Signed-off-by: Alexander Rybkin <alexander.rybkin@intel.com>
时 after 耗/超/延 (time-consuming, timeout, delay) before 调用/执行 matched the
时+verb cue, and 当事人 opened a 当…时 span. Exclude them there; the closing
时 of a 当… span keeps its class, since 当请求超时 is a when clause.

Signed-off-by: Alexander Rybkin <alexander.rybkin@intel.com>
A bare beim matched "Hilft beim ..." and "Unterstützt beim ...", which state
a capability, so those descriptions lost the description-no-when warn.
Accept beim only with verwenden, nutzen, benutzen, einsetzen or aufrufen
in the same sentence.

Signed-off-by: Alexander Rybkin <alexander.rybkin@intel.com>
@xaleryb
xaleryb requested a review from a team as a code owner November 14, 2026 03:14
@exai-bot

exai-bot commented Nov 14, 2026 •

Copy link
Copy Markdown

🔍 OpenCodeReview found 2 issue(s) in this PR.

CI at 927c4ec: 25 passing, 3 skipped.

By severity: 2 medium

Severity Category Location Finding
medium bug when_cues.py:16 Must-fix — German sentence-boundary detection produces incorrect validation verdicts.
medium bug when_cues.py:37 Must-fix — the new 当事人 exclusion stops working when a sentence contains two duration words.
Review details
  • Commit reviewed: 927c4ec against merge base 37ba217
  • Coverage: 4 files in 1 groups
  • Engine: OpenCodeReview v1.13.5-exai.4.8f73eada, us.openai.gpt-6.1-sol
  • Usage: 248k tokens (147k from cache), 1m23s
  • Tool calls: 20 (13 file_read, 2 code_comment, 2 code_search, 2 local_check_results, 1 ci_status)
  • When-cue follow-up handling, tests, and documentation: 4 files

Comment thread dokimi/dokimi/external/when_cues.py Outdated
"es": r"(?<!vez en )\bcuando\b|\bcuándo\b|\bsiempre que\b|\bcada vez que\b|\ben caso de\b|\bal (?:usar|utilizar|invocar|ejecutar)\b",
"fr": r"\bquand\b(?!\s+même)|\blorsqu\w*|\bdès que\b|\bchaque fois que\b|\ben cas de\b|\blors (?:de|d')",
"de": r"\bwenn\b|\bbeim\b|\bsobald\b|\bimmer dann\b",
"de": r"\bwenn\b|\b(?:verwende|nutze|benutze|verwenden|nutzen|benutzen|einsetzen|aufrufen)\b[^.;!?]{0,60}?\bbeim\b|"

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

bug · medium
Must-fix — do not impose a 60-character limit on a same-sentence use instruction. Trigger: Verwenden Sie diesen Skill zur Prüfung und Aktualisierung der Dokumentation beim Erstellen eines Release. Evidence: the text between Verwenden and beim exceeds 60 characters, so neither new German branch matches and when_clause_found returns False; the previous bare beim matched. Impact: catalog check incorrectly emits description-no-when despite an explicit release-time use instruction. Both new German branches have this distance-limit defect. Keep the sentence-boundary restriction without this short distance cap, and add a regression case. Workaround: shorten the instruction or rewrite it with wenn. Severity: medium because this is a concrete, bounded validation-verdict regression, not a routing or data-loss issue.

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fixed in 927c4ec: beim now takes any distance to a use verb in the same sentence, with the conjugated and separable forms. tests/test_description_when_cues.py::test_a_native_when_cue_is_accepted[de-beim-long-gap] pins it and fails without the change.

Comment thread dokimi/dokimi/external/when_cues.py Outdated
"fr": r"\bquand\b(?!\s+même)|\blorsqu\w*|\bdès que\b|\bchaque fois que\b|\ben cas de\b|\blors (?:de|d')",
"de": r"\bwenn\b|\bbeim\b|\bsobald\b|\bimmer dann\b",
"de": r"\bwenn\b|\b(?:verwende|nutze|benutze|verwenden|nutzen|benutzen|einsetzen|aufrufen)\b[^.;!?]{0,60}?\bbeim\b|"
r"\bbeim\b[^.;!?]{0,60}?\b(?:verwenden|nutzen|benutzen|einsetzen|aufrufen)\b|\bsobald\b|\bimmer dann\b",

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

bug · medium
Must-fix — preserve imperative use verbs after beim. Trigger: Beim Erstellen eines Release verwende diesen Skill. Evidence: the new beim-first branch accepts only infinitives, so when_clause_found returns False, whereas Verwende diesen Skill beim Erstellen eines Release. matches the preceding branch. The same asymmetry affects nutze and benutze. Impact: catalog check now incorrectly emits description-no-when for these valid German usage instructions. Workaround: move the imperative before beim or rewrite it as an infinitive. Include the imperative forms in both branches and add reversed-order positive tests. Severity: medium because this is a concrete, bounded regression in supported-language validation, not a routing or data-loss issue.

Check run: the covering steps pass with this change applied.

Suggestion:

Suggested change
r"\bbeim\b[^.;!?]{0,60}?\b(?:verwenden|nutzen|benutzen|einsetzen|aufrufen)\b|\bsobald\b|\bimmer dann\b",
r"\bbeim\b[^.;!?]{0,60}?\b(?:verwende|nutze|benutze|verwenden|nutzen|benutzen|einsetzen|aufrufen)\b|\bsobald\b|\bimmer dann\b",

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fixed in 927c4ec: The beim-first branch takes verwende, nutze, benutze and the separable forms. tests/test_description_when_cues.py::test_a_native_when_cue_is_accepted[de-beim-first-verwende] pins it and fails without the change.

Comment thread dokimi/dokimi/external/when_cues.py Outdated
r"\bпри (?:работе|использовании|необходимости|запросе|наличии|вызове|запуске|получении)\b",
"zh": r"(?<!任)何时|(?<!任)何時|什么时候|什麼時候|的时候|的時候|(?<![相适適恰应應妥得充担擔])[当當](?![前然即作做成地天日晚年初今中])[^。;!?;!?]{1,80}?(?<![实實同隨随及定按即暂暫不准準])(?<!(?<![水])平)(?<!(?<![没沒])有)(?<!(?<![合来來])[适適临臨])(?<!(?<![较較很过過太更最])小)[时時](?![间間区區])|(?<![实實同隨随及定按即暂暫不准準])(?<!(?<![水])平)(?<!(?<![没沒])有)(?<!(?<![合来來])[适適临臨])(?<!(?<![较較很过過太更最])小)[时時][,,]?(?:请|請)?(?:使用|调用|調用|触发|觸發|启用|啟用|激活|运行|運行|执行|執行)|"
r"适用于[^。]{0,20}?(?:场景|场合|情况|(?<![实實同隨随及定按即暂暫不准準])(?<!(?<![水])平)(?<!(?<![没沒])有)(?<!(?<![合来來])[适適临臨])(?<!(?<![较較很过過太更最])小)时(?![间間段区區序代]))|適用於[^。]{0,20}?(?:場景|場合|情況|(?<![實実同隨随及定按即暫暂不準准])(?<!(?<![水])平)(?<!(?<![没沒])有)(?<!(?<![合来來])[适適临臨])(?<!(?<![较較很过過太更最])小)時(?![間间段區区序代]))|(?:适用|適用|使用)(?:场景|場景)[::]|使用时机|使用時機|触发条件|觸發條件|用[户戶](?:[说說](?![明的])|提到(?!的)|要求(?!的)|[请請]求(?!的|数据|數據|时间|時間|参数|參數|内容|內容|日志|日誌|头|頭|量))",
"zh": r"(?<!任)何时|(?<!任)何時|什么时候|什麼時候|的时候|的時候|(?<![相适適恰应應妥得充担擔])[当當](?![前然即作做成地天日晚年初今中]|事人)[^。;!?;!?]{1,80}?(?<![实實同隨随及定按即暂暫不准準])(?<!(?<![水])平)(?<!(?<![没沒])有)(?<!(?<![合来來])[适適临臨])(?<!(?<![较較很过過太更最])小)[时時](?![间間区區])|(?<![实實同隨随及定按即暂暫不准準耗超延])(?<!(?<![水])平)(?<!(?<![没沒])有)(?<!(?<![合来來])[适適临臨])(?<!(?<![较較很过過太更最])小)[时時][,,]?(?:请|請)?(?:使用|调用|調用|触发|觸發|启用|啟用|激活|运行|運行|执行|執行)|"

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

bug · medium
Must-fix — retain timeout triggers separated from the use instruction. Trigger: 请求超时,请使用本技能重试。 (use this skill to retry when the request times out). Evidence: adding 超 to the negative lookbehind rejects 时,请使用; no other alternative matches, so when_clause_found changes from True to False and the validator emits description-no-when. Impact: valid timeout-based usage instructions receive a false warning. Distinguish the comma-separated conditional from capability wording such as 超时调用, and add a positive regression case. Workaround: write 当请求超时,请使用本技能重试。. Severity: medium because this is a concrete, bounded validation-verdict regression, not a routing or execution failure.

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fixed in 927c4ec: 超时, 延时 or 耗时 followed by a comma or 请 and a use verb counts again; 超时调用 stays no cue. tests/test_description_when_cues.py::test_a_native_when_cue_is_accepted[zh-chaoshi-comma-use] pins it and fails without the change.

Comment thread dokimi/dokimi/external/when_cues.py Outdated
r"适用于[^。]{0,20}?(?:场景|场合|情况|(?<![实實同隨随及定按即暂暫不准準耗超延])(?<!(?<![水])平)(?<!(?<![没沒])有)(?<!(?<![合来來])[适適临臨])(?<!(?<![较較很过過太更最])小)时(?![间間段区區序代]))|適用於[^。]{0,20}?(?:場景|場合|情況|(?<![實実同隨随及定按即暫暂不準准耗超延])(?<!(?<![水])平)(?<!(?<![没沒])有)(?<!(?<![合来來])[适適临臨])(?<!(?<![较較很过過太更最])小)時(?![間间段區区序代]))|(?:适用|適用|使用)(?:场景|場景)[::]|使用时机|使用時機|触发条件|觸發條件|用[户戶](?:[说說](?![明的])|提到(?!的)|要求(?!的)|[请請]求(?!的|数据|數據|时间|時間|参数|參數|内容|內容|日志|日誌|头|頭|量))",
"ja": r"(?<!ひ)(?:とき(?!どき|おり|たま|には)|(?<=[^\s、。,,.!?はもがでにをとやか])ときには)|(?<![同当一平瞬即随適臨戦日])(?<!(?<![一-鿿])定)(?<!(?<![一-鿿])常)(?:時に(?!は)|(?<!ある)時は|(?<=[^\s、。,,.!?はもがでにをとやか])時には)|(?<!ほとんどの)(?<!多くの)(?<!大半の)場合(?!によ|分け|の数)|(?:場面|シーン)で(?:使う|起動|発動)|使(?:う|用)タイミング|トリガー[::]|(?:など|」|』|))で(?:起動|発動)|"
r"(?<![実国交])際(?:に|は)|いつ(?!まで|も|でも|か|の間)",
r"(?<![実国交])際(?:に|は)|いつ(?=[使呼起発実用利])",

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

bug · medium
Must-fix — allow an object between いつ and the use verb. Trigger: いつこのスキルを使うか:リリース前。 (when to use this skill: before a release). Evidence: the new immediate-character lookahead rejects いつこ, and none of the other cues match; the previous pattern accepted it. Impact: inserting the natural object このスキルを into the newly tested いつ使うか wording incorrectly changes the result to False and produces description-no-when. Use a bounded phrase-aware match that permits the object while continuing to reject 思いつく, 追いつく, 食いつく and こいつ, and test this wording. Workaround: omit the object and write いつ使うか:リリース前。. Severity: medium because an explicit usage-time description gets an incorrect validation verdict; execution remains unaffected.

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fixed in 927c4ec: いつ counts when a use verb follows within 20 characters in the sentence; 思いつく, 追いつく, 食いつく, こいつ stay no cue. tests/test_description_when_cues.py::test_a_native_when_cue_is_accepted[ja-itsu-object-tsukau] pins it and fails without the change.

@codecov-commenter

codecov-commenter commented Nov 14, 2026 •

Copy link
Copy Markdown

⚠️ Please install the 'codecov app svg image' to ensure uploads and comments are reliably processed by Codecov.

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 95.72%. Comparing base (37ba217) to head (927c4ec).
⚠️ Report is 26 commits behind head on main.
❗ Your organization needs to install the Codecov GitHub app to enable full functionality.

Additional details and impacted files
@@            Coverage Diff             @@
##             main     #743      +/-   ##
==========================================
- Coverage   95.73%   95.72%   -0.01%     
==========================================
  Files         403      415      +12     
  Lines       54061    55366    +1305     
  Branches     9308     9516     +208     
==========================================
+ Hits        51755    53001    +1246     
- Misses       1421     1463      +42     
- Partials      885      902      +17     
Files with missing lines Coverage Δ
dokimi/dokimi/external/when_cues.py 100.00% <100.00%> (ø)

... and 60 files with indirect coverage changes

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

@antonwolfy antonwolfy left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Review at 96705f7. The three capability fixes work: Hilft beim …, 耗时调用, 超时调用, 延时执行, 当事人, 思いつく, 追いつく, 食いつく and こいつ no longer count as cues. The new cue shapes are too narrow, though. Four real when-clauses that the base accepted now get description-no-when, and the new tests do not catch it. I would fix the four inline comments and the test gap before merge.

What I ran, on a Linux review host at 96705f7:

  • make ci-local: 32 passed, 0 failed, 14 skipped; 19,768 tests; changed-line coverage 100%
  • reproductions of each inline case with when_clause_found on the PR head and on the base: all four flip from True at the base to False at the head
  • mutation runs on the new cue lists against tests/test_description_when_cues.py (212 tests, all pass at the head)

Test gap. The new tests pin one character or one verb per cue list, so most edits to the lists pass the full file. Mutants that survive, all 212 tests green:

  • de, when_cues.py:22-23: dropping any one of nutze, benutze, verwenden, nutzen, benutzen, einsetzen, aufrufen from the verb-first branch, or any of nutzen, benutzen, einsetzen, aufrufen from the beim-first branch. Widening [^.;!?] to ., which drops the sentence boundary, also survives: Nutze das Tool. Hilft beim Schreiben. is False at the head and would turn True.
  • zh, when_cues.py:28: dropping 耗, 超 or 延 from the 适用于 lookbehind or from the 適用於 lookbehind. Only the 时+verb lookbehind is pinned. 适用于监控超时的工具。 and 適用於延時的工具。 are False at the head and would become cues.
  • ja, when_cues.py:30: dropping any of 呼, 起, 発, 実, 用, 利 from the いつ lookahead. Only 使 is pinned, yet いつ起動するか不明 is True because of 起.
    These lines decide the description-no-when verdict. Add one HAS_WHEN case per verb or character in the lists, one traditional-script NO_WHEN case, and the sentence-boundary case above, plus the regression cases from the inline comments.

Checked, no issues

  • Commits: all four non-merge commits carry Signed-off-by.
  • Changelog fragment: present. It lists only verwenden, nutzen, benutzen, einsetzen, aufrufen, while the verb-first branch also takes verwende, nutze, benutze.
  • Docs: the external-content.md sentence matches the fixed words.
  • Capability phrases: all the PR's NO_WHEN sentences are False at the head, and 当请求超时,自动重试并记录日志。 and 当事件到达时,发送通知。 stay True.

Comment thread dokimi/dokimi/external/when_cues.py Outdated
"es": r"(?<!vez en )\bcuando\b|\bcuándo\b|\bsiempre que\b|\bcada vez que\b|\ben caso de\b|\bal (?:usar|utilizar|invocar|ejecutar)\b",
"fr": r"\bquand\b(?!\s+même)|\blorsqu\w*|\bdès que\b|\bchaque fois que\b|\ben cas de\b|\blors (?:de|d')",
"de": r"\bwenn\b|\bbeim\b|\bsobald\b|\bimmer dann\b",
"de": r"\bwenn\b|\b(?:verwende|nutze|benutze|verwenden|nutzen|benutzen|einsetzen|aufrufen)\b[^.;!?]{0,60}?\bbeim\b|"

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I agree with exai-bot here; the 60-character cap and the verb list turn German use instructions into a missing-when warning. Reproduced on the head and the base with when_clause_found:

  • Verwenden Sie diesen Skill zur Prüfung und Aktualisierung der Dokumentation beim Erstellen eines Release.: base True, head False (gap before beim is over 60 characters).
  • Rufe diesen Skill beim Deployen auf. and Setze diesen Skill beim Deployen ein.: base True, head False. The separable forms of aufrufen and einsetzen never match.
  • Verwende diesen Skill z. B. beim Erstellen eines Release.: base True, head False. The z. ends the match at the period.

Suggestion: drop the {0,60} cap in favour of [^.;!?]*? and accept the conjugated and separable forms, for example ruf\w*\b[^.;!?]*\bauf\b and setz\w*\b[^.;!?]*\bein\b. Add these inputs to HAS_WHEN.

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fixed in 927c4ec: beim now takes any distance to a use verb in the same sentence, with the conjugated and separable forms. tests/test_description_when_cues.py::test_a_native_when_cue_is_accepted[de-beim-long-gap] pins it and fails without the change.

Comment thread dokimi/dokimi/external/when_cues.py Outdated
"fr": r"\bquand\b(?!\s+même)|\blorsqu\w*|\bdès que\b|\bchaque fois que\b|\ben cas de\b|\blors (?:de|d')",
"de": r"\bwenn\b|\bbeim\b|\bsobald\b|\bimmer dann\b",
"de": r"\bwenn\b|\b(?:verwende|nutze|benutze|verwenden|nutzen|benutzen|einsetzen|aufrufen)\b[^.;!?]{0,60}?\bbeim\b|"
r"\bbeim\b[^.;!?]{0,60}?\b(?:verwenden|nutzen|benutzen|einsetzen|aufrufen)\b|\bsobald\b|\bimmer dann\b",

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I agree with exai-bot here; the beim-first branch lacks the imperatives, so the word order decides the verdict. Beim Erstellen eines Release verwende diesen Skill. is False at the head and Verwende diesen Skill beim Erstellen eines Release. is True. Beim Erstellen eines Release: nutze diesen Skill. is False, and so is the benutze variant. All of them are True at the base.

Suggestion: add verwende|nutze|benutze to this branch, and add the reversed-order cases to HAS_WHEN.

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fixed in 927c4ec: The beim-first branch takes verwende, nutze, benutze and the separable forms. tests/test_description_when_cues.py::test_a_native_when_cue_is_accepted[de-beim-first-verwende] pins it and fails without the change.

Comment thread dokimi/dokimi/external/when_cues.py Outdated
r"\bпри (?:работе|использовании|необходимости|запросе|наличии|вызове|запуске|получении)\b",
"zh": r"(?<!任)何时|(?<!任)何時|什么时候|什麼時候|的时候|的時候|(?<![相适適恰应應妥得充担擔])[当當](?![前然即作做成地天日晚年初今中])[^。;!?;!?]{1,80}?(?<![实實同隨随及定按即暂暫不准準])(?<!(?<![水])平)(?<!(?<![没沒])有)(?<!(?<![合来來])[适適临臨])(?<!(?<![较較很过過太更最])小)[时時](?![间間区區])|(?<![实實同隨随及定按即暂暫不准準])(?<!(?<![水])平)(?<!(?<![没沒])有)(?<!(?<![合来來])[适適临臨])(?<!(?<![较較很过過太更最])小)[时時][,,]?(?:请|請)?(?:使用|调用|調用|触发|觸發|启用|啟用|激活|运行|運行|执行|執行)|"
r"适用于[^。]{0,20}?(?:场景|场合|情况|(?<![实實同隨随及定按即暂暫不准準])(?<!(?<![水])平)(?<!(?<![没沒])有)(?<!(?<![合来來])[适適临臨])(?<!(?<![较較很过過太更最])小)时(?![间間段区區序代]))|適用於[^。]{0,20}?(?:場景|場合|情況|(?<![實実同隨随及定按即暫暂不準准])(?<!(?<![水])平)(?<!(?<![没沒])有)(?<!(?<![合来來])[适適临臨])(?<!(?<![较較很过過太更最])小)時(?![間间段區区序代]))|(?:适用|適用|使用)(?:场景|場景)[::]|使用时机|使用時機|触发条件|觸發條件|用[户戶](?:[说說](?![明的])|提到(?!的)|要求(?!的)|[请請]求(?!的|数据|數據|时间|時間|参数|參數|内容|內容|日志|日誌|头|頭|量))",
"zh": r"(?<!任)何时|(?<!任)何時|什么时候|什麼時候|的时候|的時候|(?<![相适適恰应應妥得充担擔])[当當](?![前然即作做成地天日晚年初今中]|事人)[^。;!?;!?]{1,80}?(?<![实實同隨随及定按即暂暫不准準])(?<!(?<![水])平)(?<!(?<![没沒])有)(?<!(?<![合来來])[适適临臨])(?<!(?<![较較很过過太更最])小)[时時](?![间間区區])|(?<![实實同隨随及定按即暂暫不准準耗超延])(?<!(?<![水])平)(?<!(?<![没沒])有)(?<!(?<![合来來])[适適临臨])(?<!(?<![较較很过過太更最])小)[时時][,,]?(?:请|請)?(?:使用|调用|調用|触发|觸發|启用|啟用|激活|运行|運行|执行|執行)|"

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I agree with exai-bot here; adding 超 to the lookbehind also kills a timeout followed by a use instruction. All of these are True at the base and False at the head:

  • 请求超时,请使用本技能重试。
  • 超时,请使用本技能重试。
  • 延时,请使用此技能。
  • 遇到超时请使用本技能。

The same lookbehind is in the 适用于 and 適用於 branches (lines 28-29 of this diff), which are not pinned by any test. Suggestion: keep the 耗/超/延 exclusion only for a 时 that is followed directly by the verb (超时调用), and let a 时 with ,, , or 请 before the verb count again, e.g. a second alternative (?<=[耗超延])[时時][,,请請]+(?:请|請)?(?:使用|调用|調用|触发|觸發|启用|啟用|激活|运行|運行|执行|執行). Add 请求超时,请使用本技能重试。 to HAS_WHEN and check that 监控超时调用并发出告警。 stays False.

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fixed in 927c4ec: 超时, 延时 or 耗时 followed by a comma or 请 and a use verb counts again; 超时调用 stays no cue. tests/test_description_when_cues.py::test_a_native_when_cue_is_accepted[zh-chaoshi-comma-use] pins it and fails without the change.

Comment thread dokimi/dokimi/external/when_cues.py Outdated
r"适用于[^。]{0,20}?(?:场景|场合|情况|(?<![实實同隨随及定按即暂暫不准準耗超延])(?<!(?<![水])平)(?<!(?<![没沒])有)(?<!(?<![合来來])[适適临臨])(?<!(?<![较較很过過太更最])小)时(?![间間段区區序代]))|適用於[^。]{0,20}?(?:場景|場合|情況|(?<![實実同隨随及定按即暫暂不準准耗超延])(?<!(?<![水])平)(?<!(?<![没沒])有)(?<!(?<![合来來])[适適临臨])(?<!(?<![较較很过過太更最])小)時(?![間间段區区序代]))|(?:适用|適用|使用)(?:场景|場景)[::]|使用时机|使用時機|触发条件|觸發條件|用[户戶](?:[说說](?![明的])|提到(?!的)|要求(?!的)|[请請]求(?!的|数据|數據|时间|時間|参数|參數|内容|內容|日志|日誌|头|頭|量))",
"ja": r"(?<!ひ)(?:とき(?!どき|おり|たま|には)|(?<=[^\s、。,,.!?はもがでにをとやか])ときには)|(?<![同当一平瞬即随適臨戦日])(?<!(?<![一-鿿])定)(?<!(?<![一-鿿])常)(?:時に(?!は)|(?<!ある)時は|(?<=[^\s、。,,.!?はもがでにをとやか])時には)|(?<!ほとんどの)(?<!多くの)(?<!大半の)場合(?!によ|分け|の数)|(?:場面|シーン)で(?:使う|起動|発動)|使(?:う|用)タイミング|トリガー[::]|(?:など|」|』|))で(?:起動|発動)|"
r"(?<![実国交])際(?:に|は)|いつ(?!まで|も|でも|か|の間)",
r"(?<![実国交])際(?:に|は)|いつ(?=[使呼起発実用利])",

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I agree with exai-bot here; いつ(?=[使呼起発実用利]) only accepts a verb that starts right after いつ. True at the base and False at the head:

  • いつこのスキルを使うか:リリース前。
  • いつ、このスキルを使うか。
  • いつこれを使う?
  • いつこのスキルを起動する?
  • いつ参照すべきか, いつ適用するか

Suggestion: allow a short phrase before the verb and exclude the four capability words with a lookbehind, for example (?<![こ思追食])いつ(?=[^。!?!?\n]{0,20}?(?:使|呼び出|起動|発動|実行|利用)). Check it against the four NO_WHEN cases. Add いつこのスキルを使うか to HAS_WHEN.

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fixed in 927c4ec: いつ counts when a use verb follows within 20 characters in the sentence; 思いつく, 追いつく, 食いつく, こいつ stay no cue. tests/test_description_when_cues.py::test_a_native_when_cue_is_accepted[ja-itsu-object-tsukau] pins it and fails without the change.

Fixes the narrowed cues that turned real when-clauses into
description-no-when:
- de: no 60-character cap, imperative and separable use verbs
  (ruf auf, setz ein), either order, z. B. does not end the sentence
- zh: 超时/延时/耗时 followed by a comma or 请 and a use verb counts
  again; 当事人签字时 counts
- ja: いつ with an object before the use verb counts
- de: a full stop before an acronym ends the sentence
- zh: 当事人 with an unrelated 用时 is no cue
- ja: そいつ, あいつ and どいつ are no いつ cue
- tests: one case per verb or character in the cue lists

Signed-off-by: vchamarthi <123653452+vchamarthi@users.noreply.github.com>
@vchamarthi

Copy link
Copy Markdown
Contributor

Pushed 1 commit to fix/validate-when-cue-followups (96705f7..927c4ec).

  • 当事人签字时,生成归档报告。 counts as a when cue again.
  • A full stop before an acronym ends the German sentence.
  • 当事人 with an unrelated 用时 is no cue.
  • そいつ, あいつ and どいつ are no いつ cue.

Threads: 8 fixed, 0 answered as not a defect.
Run: the full suite with main merged in, 20510 passed.

From the comments:

  • test gap: surviving mutants on de verbs, zh 适用于 lookbehind, ja lookahead characters: fixed, one HAS_WHEN case per verb and lookahead character, the sentence-boundary case, and NO_WHEN cases for 耗/超/延 after 适用于 and 適用於 in tests/test_description_when_cues.py.
  • changelog fragment omits verwende, nutze, benutze: fixed, fragment lists every accepted form.


# German `beim` is a cue only in a sentence with a use verb, in either order; a full stop that is
# followed by a capitalised word or an acronym ends the sentence, so `z. B.` does not.
_DE_S = r"(?:[^.;!?]|\.(?!\s*(?-i:[A-ZÄÖÜ][A-Za-zäöüß])))*?"

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

bug · medium
Must-fix — German sentence-boundary detection produces incorrect validation verdicts. Trigger: Beim Deployen mit z. B. GitHub Actions verwende diesen Skill. is rejected, while Nutze das Tool. „Hilft beim Schreiben“ beschreibt seine Funktion. is accepted. Evidence: the negative lookahead blocks the period after B because GitHub starts with an uppercase letter followed by another letter, but permits the actual sentence-ending period before „Hilft because the quote bypasses that check. Neither example has another matching cue. Impact: catalog check emits description-no-when for the valid instruction and suppresses it for the capability-only description. Handle abbreviations and quoted sentence starts explicitly, and add both regression cases (including warning assertions). Workaround: use an explicit wenn clause for instructions. Severity: medium, because this is a concrete but bounded validation-verdict failure.

r"\bпри (?:работе|использовании|необходимости|запросе|наличии|вызове|запуске|получении)\b",
"zh": r"(?<!任)何时|(?<!任)何時|什么时候|什麼時候|的时候|的時候|(?<![相适適恰应應妥得充担擔])[当當](?![前然即作做成地天日晚年初今中])[^。;!?;!?]{1,80}?(?<![实實同隨随及定按即暂暫不准準])(?<!(?<![水])平)(?<!(?<![没沒])有)(?<!(?<![合来來])[适適临臨])(?<!(?<![较較很过過太更最])小)[时時](?![间間区區])|(?<![实實同隨随及定按即暂暫不准準])(?<!(?<![水])平)(?<!(?<![没沒])有)(?<!(?<![合来來])[适適临臨])(?<!(?<![较較很过過太更最])小)[时時][,,]?(?:请|請)?(?:使用|调用|調用|触发|觸發|启用|啟用|激活|运行|運行|执行|執行)|"
r"适用于[^。]{0,20}?(?:场景|场合|情况|(?<![实實同隨随及定按即暂暫不准準])(?<!(?<![水])平)(?<!(?<![没沒])有)(?<!(?<![合来來])[适適临臨])(?<!(?<![较較很过過太更最])小)时(?![间間段区區序代]))|適用於[^。]{0,20}?(?:場景|場合|情況|(?<![實実同隨随及定按即暫暂不準准])(?<!(?<![水])平)(?<!(?<![没沒])有)(?<!(?<![合来來])[适適临臨])(?<!(?<![较較很过過太更最])小)時(?![間间段區区序代]))|(?:适用|適用|使用)(?:场景|場景)[::]|使用时机|使用時機|触发条件|觸發條件|用[户戶](?:[说說](?![明的])|提到(?!的)|要求(?!的)|[请請]求(?!的|数据|數據|时间|時間|参数|參數|内容|內容|日志|日誌|头|頭|量))",
"zh": r"(?<!任)何时|(?<!任)何時|什么时候|什麼時候|的时候|的時候|(?<![相适適恰应應妥得充担擔])[当當](?![前然即作做成地天日晚年初今中])(?!事人[^。;!?;!?时時]*[耗超延用][时時][^。;!?;!?时時]*(?:[。;!?;!?]|$))[^。;!?;!?]{1,80}?(?<![实實同隨随及定按即暂暫不准準])(?<!(?<![水])平)(?<!(?<![没沒])有)(?<!(?<![合来來])[适適临臨])(?<!(?<![较較很过過太更最])小)[时時](?![间間区區])|(?<![实實同隨随及定按即暂暫不准準耗超延])(?<!(?<![水])平)(?<!(?<![没沒])有)(?<!(?<![合来來])[适適临臨])(?<!(?<![较較很过過太更最])小)[时時][,,]?(?:请|請)?(?:使用|调用|調用|触发|觸發|启用|啟用|激活|运行|運行|执行|執行)|[耗超延][时時](?:[,,]\s*(?:请|請)?|[请請])(?:使用|调用|調用|触发|觸發|启用|啟用|激活|运行|運行|执行|執行)|"

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

bug · medium
Must-fix — the new 当事人 exclusion stops working when a sentence contains two duration words. Trigger: 整理当事人信息,统计查询耗时和超时次数。 has no when clause. Evidence: both character classes in the negative lookahead exclude 时, so it cannot reach the sentence end past the second 时; the subsequent 当…时 alternative consequently accepts 当事人信息,统计查询耗时. Impact: when_clause_found returns True and _description_findings suppresses description-no-when, retaining the false pass this change promises to remove. Exclude capability-only closing times without requiring exactly one 时 in the sentence, and add this case to NO_WHEN while preserving 当事人签字时. Workaround: omit 当事人 or split the capability list into separate sentences. Medium severity: a bounded but incorrect catalog validation verdict.

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

5 participants