fix(helpers): prevent citation metadata bleeding across neighboring citations

[heronfonsaca]

Description

Fixes both repros in #321: FullCaseCitation.metadata picking up the previous citation's plaintiff/defendant, and separately its year, instead of its own.

Fix 1 — missing stop word

When scanning backward from a citation to find its case name, eyecite stops the scan as soon as it hits a known "stop word" (STOP_WORD_REGEX / STOP_WORDS in eyecite/regexes.py) such as see, accord, cf, etc. If
the text immediately before a citation isn't a recognized stop word, the backward scan keeps walking past the sentence boundary and can pick up the previous citation's plaintiff/defendant as this citation's case name.

compare (as in Compare X v. Y ... with A v. B ... signal phrases) was missing from STOP_WORDS, so a citation introduced with Compare had nothing to stop the backward scan, and it inherited the prior citation's defendant instead of its own.

from eyecite import get_citations
from eyecite.tokenizers import AhocorasickTokenizer

text = (
    'The court in Smith v. Jones, 347 U.S. 999 (1954), held that '
    '"schools must serve pizza." Smith v. Jones, 347 U.S. 999 (1954). '
    'Compare Brown v. Board of Education, 347 U.S. 483 (1954).'
)
for c in get_citations(text, tokenizer=AhocorasickTokenizer()):
    print(c.corrected_citation(), repr(c.metadata.plaintiff), repr(c.metadata.defendant))

Before:

347 U.S. 999 'Smith' 'Jones'
347 U.S. 999 'Smith' 'Jones'
347 U.S. 483 ''      'Jones'      # <- Brown's cite, defendant leaked from the prior sentence

After:

347 U.S. 999 'Smith' 'Jones'
347 U.S. 999 'Smith' 'Jones'
347 U.S. 483 'Brown' 'Board of Education'

Fix: add compare to STOP_WORDS in eyecite/regexes.py so the backward case-name scan stops there instead of crossing into the previous citation's text, the same mechanism see/accord/cf already rely on.

Fix 2 — year leaking from the previous citation's parenthetical

The same backward scan in _scan_for_case_boundaries (eyecite/helpers.py) also picks up a (YYYY) year token as it walks past the citation it's building a case name for. It didn't distinguish between a year parenthetical that belongs to this citation (appearing before the v. token) and one that belongs to an earlier citation in the text
(appearing after the v. token has already been seen, i.e. the year is on the far side of a case name the scan has already fully consumed). As a result, a citation could silently inherit the previous citation's year instead of using its own trailing (YYYY).

text = "Miranda v. Arizona, 347 U.S. 483 (1990). NLRB v. IBEW, 346 U.S. 464 (1953)."
for c in get_citations(text, tokenizer=AhocorasickTokenizer()):
    print(c.corrected_citation(), c.metadata.year)

Before:

347 U.S. 483 1990
346 U.S. 464 1990        # <- text says (1953) immediately after this cite

After:

347 U.S. 483 1990
346 U.S. 464 1953

Fix: in _scan_for_case_boundaries, once a v_token has been recorded for the current citation, hitting another (YYYY) token means the scan has crossed into an earlier citation's parenthetical — so the loop now breaks
instead of overwriting pre_cite_year with that unrelated year.

Both fixes are covered by new cases added to tests/test_FindTest.py.

AI Disclosure

• No AI tools were used to create the content of this PR.
• Parts of this PR were created with the help of an AI tool, and I have carefully reviewed all of its content and take full responsibility for it.

[claude]

Claude Code Review

This pull request is from a fork — automated review is disabled. A repository maintainer can comment @claude review to run a one-time review.

[CLAassistant]

All committers have signed the CLA.

[flooie]

@claude review

[claude]

Nothing blocking. The comments below are optional suggestions. There is no need to push a fix for them before merging.

Beyond the inline finding, I checked the new early break in _scan_for_case_boundaries (helpers.py:234) added in the latest commit: although it exits without recomputing start_index/candidate_case_name, those are already populated at the point v_token is first recorded, so no case-name text is lost by breaking early. I also confirmed the guard's v_token is not None scope is appropriate, since the year-bleed bug it targets only arises for citations whose case name contains "v."

Extended reasoning...

This run's inline comment reconfirms a pre-existing, unaddressed issue (regexes.py:93: adding "compare" to the shared STOP_WORDS also widens strip_stop_words's unanchored, case-insensitive substitution, so any real party name containing "compare" as a standalone word would be corrupted) — that concern predates the latest commit and is carried by the inline comment, not restated here. What is new since the last review is the second commit (fc9b4bc), which adds the v_token-aware break in _scan_for_case_boundaries for the year-boundary fix; I traced that logic against two candidate concerns (loss of already-extracted case-name state on early break, and the narrow v_token-only scope of the guard) and ruled both out based on the surrounding code's control flow, matching this run's ruled-out list.

[claude]

🟡 (optional) Adding "compare" to STOP_WORDS also widens strip_stop_words (helpers.py), which reuses the same STOP_WORD_REGEX to clean already-extracted plaintiff/defendant text via a global, case-insensitive, non-anchored substitution. Any real party name containing "compare" as a standalone word (e.g. "Compare Networks, Inc.", "Compare.com") now gets that word silently stripped from metadata.plaintiff/defendant, which did not happen on base. Fix: scope the new stop-word entry to the tokenizer's boundary-detection use only (or anchor strip_stop_words's substitution to leading/signal-phrase position) so cleanup of already-parsed case names doesn't remove ordinary words that happen to match a stop word.

Extended reasoning...

STOP_WORD_REGEX (regexes.py:95-97) is built from STOP_WORDS and used both by AhocorasickTokenizer to detect signal words during backward scanning AND by strip_stop_words (helpers.py:805-831), which is called on already-extracted plaintiff/defendant strings at helpers.py:373,379,386,624,700,703,742,745,801. strip_stop_words's second re.sub (line 823-828, flags=re.IGNORECASE) matches the whole-word pattern space_boundaries_re anywhere in the string, not just at the start. Given defendant text "Compare Networks, Inc." extracted from HTML tags (helpers.py:734-745), the word "Compare" is stripped, producing metadata.defendant = "Networks, Inc." instead of the correct name -- a data-corruption regression not present before this diff.

Verification: nit. The mechanism is real and introduced by this diff. Adding "compare" to STOP_WORDS (regexes.py:93) feeds the shared STOP_WORD_REGEX (regexes.py:95-97 -> (?:^|\s)([^\sa-zA-Z0-9]*(?P<stop_word>...|compare)[^\sa-zA-Z0-9]*)(?:\s|$)), which strip_stop_words reuses on already-extracted party text. The second substitution (helpers.py:822-828) runs with flags=re.IGNORECASE and is not…

[flooie]

So this is good, but can I ask you to look at a few more examples I came up with.

Try these.

Smith v. Jones, 347 U.S. 999. The court disagreed. Brown v. Board, 347 U.S. 483 (1954).
Smith v. Jones, 347 U.S. 999. Brown v. Board, 900 F.3d 1 (5th Cir. 2019).
Smith v. Jones, 347 U.S. 999 (1952). Citizens United v. FEC, 558 U.S. 310 (2010).
Smith v. Jones, 347 U.S. 999 (1952). Trustees of Dartmouth College v. Woodward, 17 U.S. 518 (1819).

Right now it gives College. There's more than one thing going on. Add tests for whatever you find.

@heronfonsaca

[heronfonsaca]

I can see that the two last examples are specific scenarios that I did not foresee before, but I have already addressed them. Let me break down each one:

Multi-word Plaintiff Name

citation_text = "Smith v. Jones, 347 U.S. 999 (1952). Citizens United v. FEC, 558 U.S. 310 (2010)."
for c in get_citations(citation_text):
   print(c.corrected_citation(), c.metadata.year, repr(c.metadata.plaintiff), repr(c.metadata.defendant))

347 U.S. 999 1952 'Smith' 'Jones'
558 U.S. 310 2010 'United' 'FEC'  <-- missing 'Citizens'

Not updating state["start_index"] and state["candidate_case_name"] when hitting parenthetical year boundary in eyecite.helpers._scan_for_case_boundaries would lead to only the latest plaintiff word to be extracted. This has been solved by 16bc042.

Plaintiff name including exceptional lowercase-only words ('of', 'and')

citation_text = "Smith v. Jones, 347 U.S. 999 (1952). Trustees of Dartmouth College v. Woodward, 17 U.S. 518 (1819)."
for c in get_citations(citation_text):
   print(c.corrected_citation(), c.metadata.year, repr(c.metadata.plaintiff), repr(c.metadata.defendant))

347 U.S. 999 1952 'Smith' 'Jones'
17 U.S. 518 1819 'Trustees  Dartmouth College' 'Woodward'  <-- missing 'of'

Lowercase-only words are removed from eyecite.helpers._process_case_name plaintiff name processing. I have fixed this by updating the regex substitution to not account for the lowercase-only word of and and (de4965d). Let me know if these candidates are adequate, @flooie .

I am still working on the first two examples provided before:

Smith v. Jones, 347 U.S. 999. The court disagreed. Brown v. Board, 347 U.S. 483 (1954).
Smith v. Jones, 347 U.S. 999. Brown v. Board, 900 F.3d 1 (5th Cir. 2019).

It looks like those are not the same kind of issue as the previous ones, and both bleed the last citation year to the first one.

[flooie]

They are not, but I am curious how you would or if you could tackle them. Plaintiff names have been particularly difficult but in parallel like this I would suspect we could manage.

[heronfonsaca]

@flooie I can see that in the following text:

Smith v. Jones, 347 U.S. 999. The court disagreed. Brown v. Board, 347 U.S. 483 (1954).

The subtext:

The court disagreed. Brown v. Board, 347 U.S. 483 (1954).

is being handled as post-citation metadata, and having its string (1954) parsed as year for the first citation.

Do you think that setting "." as a boundary to handling post-citation text is a good intervention to prevent this issue? Are there any cases that I might be missing where this would not work?

[flooie]

I'm not sure. I would think it could be, but it could cause down stream issues. Maybe a test example exists?

[heronfonsaca]

@flooie Great point. I have just tested a parallel citation example composed with a unit test case and it cause conflicts:

citation_text = 'bob Lissner v. Test 1 U.S. 12, 347-348. See also Baz v. Qux, 2 U.S. 2 (2013).'
for c in get_citations(citation_text):
   print(c.corrected_citation(), c.metadata.year, repr(c.metadata.plaintiff), repr(c.metadata.defendant))

1 U.S. 12 2013 'Lissner' 'Test'  <-- 2013 incorrectly assigned
2 U.S. 2 2013 'Baz' 'Qux'

I believe that this matter would be better handled in a new Github issue of its own.

Let me know if I can update the top description of this PR and we can move forward with it.