Skip to content

README.md

Latest commit

 

History

History
56 lines (45 loc) · 3.5 KB

README.md

File metadata and controls

56 lines (45 loc) · 3.5 KB
Raw

Lokad.Utf8Regex.PythonRe

Lokad.Utf8Regex.PythonRe is a net10.0 side extension for Lokad.Utf8Regex that exposes a UTF-8-first Python re flavored API through Utf8PythonRegex.

It is intended for workloads where the input already exists as UTF-8 bytes and you need Python-style regular expression semantics on top of the base Lokad.Utf8Regex library.

Support Scope

  • Semantic target: Utf8PythonRegex models the standard re surface, while remaining a managed .NET library.
  • Implementation model: parser-first translation. Python-style patterns are parsed into a dedicated AST before any backend is selected.
  • Translation-first execution: when a Python-style pattern can be translated without semantic drift, execution is delegated to Utf8Regex.
  • Fallback execution: if UTF-8 native execution cannot be created, the package falls back to managed .NET Regex on the translated pattern.
  • Dependency model: the only non-BCL implementation dependency is Lokad.Utf8Regex.
  • Primary I/O model: the main API surface operates on UTF-8 ReadOnlySpan<byte>.
  • Capture-aware APIs use PythonRe-specific result/context types, independent from the PCRE2 sidecar.
  • Culture model: the profile is culture-invariant by construction; Python re.LOCALE is intentionally unsupported.
  • Named Unicode escapes: \N{...} is intentionally unsupported to avoid embedding a large Unicode-name table in the package.
  • FindAll(...) is a structural full-match collector.
  • FindAllToStrings(...) provides CPython-shaped re.findall(...) string result shaping.
  • FindAllToUtf8(...) provides the same host-facing shape with UTF-8 byte outputs.
  • SearchToString(...), MatchToString(...), and FullMatchToString(...) provide host-friendly scalar helpers.
  • SearchDetailedData(...), MatchDetailedData(...), FullMatchDetailedData(...), and FindIterDetailed(...) provide non-ref struct match snapshots for runtime hosts.
  • ReplaceToString<TState>(...) / SubnToString<TState>(...) support evaluator-driven replacement over match snapshots.
  • Subn(...) / Subn<TState>(...) provide UTF-8 replacement outputs with replacement counts for bytes-first hosts.
  • Replace<TState>(...) / Subn<TState>(...) also expose byte-oriented evaluator overloads for hosts that want to stay UTF-8-native end to end.
  • SplitDetailed(...) exposes split output with segment/capture metadata.

Public API

using Lokad.Utf8Regex.PythonRe;

var regex = new Utf8PythonRegex(
    pattern: @"(?P<word>foo)-(?P=word)",
    options: PythonReCompileOptions.None);

bool isMatch = regex.IsMatch("foo-foo"u8);
var first = regex.Search("xx foo-foo yy"u8);
int count = regex.Count("foo-foo x foo-foo"u8);
byte[] replaced = regex.Replace("foo x foo"u8, "<\\g<word>>");
Utf8PythonMatchData[] matches = regex.FindAll("foo-foo xx foo-foo"u8);
var iter = regex.FindIterDetailed("foo-foo xx foo-foo"u8);
string?[] parts = regex.SplitToStrings("foo-foo xx foo-foo"u8);

String-returning helpers such as SearchToString(...), ReplaceToString(...), and FindAllToStrings(...) are also available as host-friendly conveniences when a runtime needs scalar string results.

Development Notes