Skip to content

Unicode output: MathS.Settings.UnicodeOutput, and the parser reads the glyphs it prints - #2469

Merged
Rafael-SOWNet merged 1 commit into
masterfrom
feature/unicode-stringize
Nov 16, 2026
Merged

Rafael-SOWNet merged 1 commit into
masterfrom
feature/unicode-stringize

Conversation

@Rafael-SOWNet

Copy link
Copy Markdown
Member

MathS.Settings.UnicodeOutput, a scoped Setting<bool>, off by default: with it on, Stringize and ToString print π ℯ, +∞ −∞, ≤ ≥, ≠ (for not a = b), ∈ ⊆, ≡ (a congruence), ∀ ∃ ∃!, ∪ ∩, ¬ ∧ ∨, − (U+2212) for minus, and superscript digits for a plain integer exponent. x ^ 2 - 1 <= pi prints as x² − 1 ≤ π. The parser reads exactly those glyphs back. This is the plan posted on the issue on 2026-11-15.

  • A superscript keeps its ^ where it would mislead the eye: before another ^ (x ^ 2 ^ 3 is x ^ 2³), and before !, !! or [. Only integer literal exponents are superscripted.
  • The printer builds the ASCII form, lexes it, and swaps tokens for glyphs, so names, numbers and calls are never touched (notable, 1E-20, sin().
  • Nothing the library computes depends on the flag: every internal path that reads printed text (term ordering, monomial keys, the integrator's radicand spelling, e-graph leaf keys, the JSON converter, SymPy code, LaTeX's set letter) now uses an internal StringizeAscii(). LaTeX output is unchanged. JSON stays ASCII whatever the flag, so a serialised form does not depend on a display setting.
  • A bare ∞ stays a parse error, as bare oo is a variable (+oo and -oo are members of RR and CC #2220); the printer always writes +∞ or −∞. √, ·, × stay errors too.
glyph master now
≤ ≥ ≠ ∈ ∪ ∩ ¬ ∧ ∨ token recognition error <= >= <> in \/ /\ not and or
− (U+2212) error -, including a − b, −x, 2 − −3
+∞, −∞, -∞ error +oo, -oo
superscripts error x² is x^2, 2x² is 2*x^2, x²³ is x^23, −x² is -(x^2), x⁻¹ is x^(-1)
π ℯ ⊆ ⊇ ≡ ∀ ∃ ∃! already read unchanged, now also printed

Every new glyph was an error on master, so no input that parses changes meaning, and there is no BREAKING-CHANGES.md entry.

The grammar changes at the lexer only: UNICODE_SYNONYM, lexed as the ASCII token it replaces, and SUPERSCRIPT, expanded to ^ n in Parser.cs before the implicit-operator pass (also under ExplicitParsingOnly). No parser rule changes; the regenerated parser differs by token numbers. Master's grammar was regenerated unchanged first and matched its generated files byte for byte.

StringizeRoundTripTest runs every case again with the flag on (UnicodeStringizeRoundTripTest, 313 cases), plus 59 Unicode-sensitive shapes in both modes and checks that ordering, LaTeX and SymPy do not change. The workspace round-trip harness with the flag on: 5,000 expressions, 0 that fail to parse back, 0 that read back with another value, and 0 where the Unicode form reads differently from the ASCII one, again at 20,000 expressions of depth 4. Syntax.md documents the setting and the glyphs.

Full check on this commit: 20,486 unit tests, 0 failed; MCP 444, Terminal 69, F# 134, Interactive 19; AOT OK; BoundCheck 0 disagreements, RootCheck 596/596, CasBench 120/121 with 0 wrong, PropCheck 1355/0, SimpSweep 10463/10463, RuleCheck clean, CanonCheck and Confluence at their baselines.

Closes #1242.

🤖 Generated with Claude Code

A setting, off by default, under which Stringize prints π ℯ +∞ −∞ ≤ ≥ ≠ ∈ ⊆ ≡ ∀ ∃ ∃! ∪ ∩ ¬ ∧ ∨,
the minus sign U+2212, and superscript digits for an integer exponent (x², x⁻¹). The parser
reads each of them back.

The parser side is lexer-only: each new glyph is lexed as the ASCII token it stands for
(UNICODE_SYNONYM), and a superscript is a SUPERSCRIPT token that Parser.cs expands to ^ n
or ^ - n before the omitted tokens are inserted. No parser rule changed. Every glyph and
superscript was a token recognition error on master, so no existing input reads differently.

The printer respells the ASCII form token by token through the lexer, so names, numbers
and function calls are never touched and the parser sees the token stream the ASCII form
gives. A superscript is not used where it would misplace the exponent to the eye (before
a further ^, a factorial or an entry). Notf over an equality and Congruentf print <> and ≡
themselves, since their extent is not visible to a token pass.

What the library reads for itself -- the ordering of leaves by printed form, the monomial
keys in evaluation, the radicand spelling in integration, the e-graph's leaf keys, the
SymPy code and the JSON form -- uses StringizeAscii, so no answer depends on the setting.

StringizeRoundTripTest runs again with the setting on (UnicodeStringizeRoundTripTest),
with a theory of the respelled shapes added to both.

Part of #1242

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@codecov

codecov Bot commented Nov 16, 2026 •

Copy link
Copy Markdown

Codecov Report

❌ Patch coverage is 93.84615% with 8 lines in your changes missing coverage. Please review.
✅ Project coverage is 86.96%. Comparing base (3abd4d9) to head (f6676e3).
⚠️ Report is 1 commits behind head on master.

Files with missing lines Patch % Lines
...iMath/Functions/Output/ToString/UnicodeSpelling.cs 93.15% 3 Missing and 2 partials ⚠️
...ources/AngouriMath/Core/Antlr/AngouriMathParser.cs 83.33% 0 Missing and 1 partial ⚠️
...ions/Output/ToSympy/ToSympy.Arithmetics.Classes.cs 0.00% 1 Missing ⚠️
...Functions/Output/ToSympy/ToSympy.Number.Classes.cs 50.00% 1 Missing ⚠️
Additional details and impacted files

Impacted file tree graph

@@            Coverage Diff             @@
##           master    #2469      +/-   ##
==========================================
+ Coverage   86.95%   86.96%   +0.01%     
==========================================
  Files         320      321       +1     
  Lines       71457    71600     +143     
  Branches    20864    20894      +30     
==========================================
+ Hits        62132    62267     +135     
- Misses       5005     5009       +4     
- Partials     4320     4324       +4     
Flag Coverage Δ
cpp 37.77% <ø> (ø)
csharp 87.49% <93.84%> (+0.01%) ⬆️
fsharp 32.97% <ø> (ø)
interactive 39.81% <ø> (ø)

Flags with carried forward coverage won't be shown. Click here to find out more.

Files with missing lines Coverage Δ
Sources/AngouriMath/Convenience/MathS.cs 90.77% <100.00%> (+0.02%) ⬆️
Sources/AngouriMath/Core/Antlr/AngouriMathLexer.cs 99.26% <ø> (+0.02%) ⬆️
Sources/AngouriMath/Core/Parser.cs 80.50% <100.00%> (+4.21%) ⬆️
...ouriMath/Core/Serialization/EntityJsonConverter.cs 100.00% <100.00%> (ø)
Sources/AngouriMath/Core/Transformations/EGraph.cs 91.89% <100.00%> (ø)
...es/AngouriMath/Core/Transformations/EntityOrder.cs 83.33% <100.00%> (ø)
...Continuous/Integration/IndefiniteIntegralSolver.cs 87.00% <100.00%> (-0.01%) ⬇️
...Math/Functions/Evaluation/Evaluation.Definition.cs 91.63% <100.00%> (ø)
...iMath/Functions/Output/Latex/Latex.Omni.Classes.cs 98.30% <100.00%> (ø)
...ngouriMath/Functions/Output/ToString.Definition.cs 100.00% <100.00%> (ø)
... and 6 more

... and 3 files with indirect coverage changes

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

@Rafael-SOWNet
Rafael-SOWNet merged commit c530e40 into master Nov 16, 2026
36 checks passed
@Rafael-SOWNet
Rafael-SOWNet deleted the feature/unicode-stringize branch November 16, 2026 00:52
@github-project-automation github-project-automation Bot moved this from Todo to Done in AngouriMath Nov 16, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

Status: Done

Development

Successfully merging this pull request may close these issues.

Unicode Stringize and parsing

1 participant