Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -30,7 +30,7 @@ adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0.html).
- Control characters and bidi/format characters are stripped during normalization, so they can no longer survive into name fields and downstream exports.
- Nickname delimiters containing a comma, NUL, whitespace, or control characters are ignored instead of silently corrupting the structural-comma split.
- The token budget counts the same separator class the parser splits on, and the uniform-uppercase scan is capped, so hostile separator-heavy rows stay within the documented 65,536-token cost.
- The structural-comma shield degrades to a bounded byte-level mask past 4 KB instead of switching off, keeping surname/given assignment stable on long rows.
- Single-byte nickname delimiter configurations use a bounded byte-level structural-comma mask without the 4,096-byte cutoff. Configurations containing any valid multi-character or multibyte delimiter still stop shielding nickname commas above 4,096 bytes; see the README parsing limits.
- `Smith, Jr` reads like `Smith, Junior` (Lastname plus given name) instead of a bare suffix with no given name.
- `Parser` internals are split into `StructuralCommaSplitter`, `CommaCredentialTail`, and `SegmentParserFactory` with `TokenCredentialClass` replacing the credential classifier ints; `Name` dispatches on `::class` constants. No behavior change.

Expand Down
9 changes: 9 additions & 0 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -330,6 +330,15 @@ instead of tokenizing the source again. Explicitly supplied token arrays are
also limited to 65,536 entries and 1,048,576 aggregate token bytes; supplying
tokens does not bypass validation of the original string.

Custom nickname delimiters have a separate comma-splitting limit. If any valid
configured opener or closer is longer than one byte (including ASCII `<<` / `>>`
and Unicode `«` / `»`), nickname commas are not shielded when the normalized
input exceeds 4,096 bytes. This applies even when that longer pair is unused in
the input. A comma inside a nickname can then become a surname/given-name
separator and change the parsed fields. For long rows with nickname commas,
use only single-byte delimiter pairs, such as the defaults; their byte-level
scan has no 4,096-byte cutoff. The overall input limits still apply.

### Ambiguous inputs

Some inputs have no structural signal. A comma followed only by credentials can
Expand Down
3 changes: 2 additions & 1 deletion src/StructuralCommaSplitter.php
Original file line number Diff line number Diff line change
Expand Up @@ -96,7 +96,8 @@ public static function mask(string $name, array $nicknameDelimiters): string
return self::maskAscii($name, $pairs, $symmetric);
}

// Bound character-array allocation for multibyte delimiters; longer input is unshielded.
// Any multi-character or multibyte endpoint selects this bounded character scan.
// Above 4096 bytes, even commas inside single-byte pairs in a mixed config are unshielded.
if (strlen($name) > 4096) {
return $name;
}
Expand Down
2 changes: 1 addition & 1 deletion tests/CommaSegmentTest.php
Original file line number Diff line number Diff line change
Expand Up @@ -353,7 +353,7 @@ public function testSurnameSegmentBindsPastMidSegmentNickname(): void

/**
* Use multibyte delimiters to exercise the 4096-byte shielding limit;
* the ASCII scan has no length cap.
* the scan for exclusively single-byte delimiter pairs has no such cutoff.
*
* @return array<string, array{int, string, string}>
*/
Expand Down
Loading