Repository navigation
Unicode output: MathS.Settings.UnicodeOutput, and the parser reads the glyphs it prints - #2469
Merged
Merged
Conversation
A setting, off by default, under which Stringize prints π ℯ +∞ −∞ ≤ ≥ ≠ ∈ ⊆ ≡ ∀ ∃ ∃! ∪ ∩ ¬ ∧ ∨, the minus sign U+2212, and superscript digits for an integer exponent (x², x⁻¹). The parser reads each of them back. The parser side is lexer-only: each new glyph is lexed as the ASCII token it stands for (UNICODE_SYNONYM), and a superscript is a SUPERSCRIPT token that Parser.cs expands to ^ n or ^ - n before the omitted tokens are inserted. No parser rule changed. Every glyph and superscript was a token recognition error on master, so no existing input reads differently. The printer respells the ASCII form token by token through the lexer, so names, numbers and function calls are never touched and the parser sees the token stream the ASCII form gives. A superscript is not used where it would misplace the exponent to the eye (before a further ^, a factorial or an entry). Notf over an equality and Congruentf print <> and ≡ themselves, since their extent is not visible to a token pass. What the library reads for itself -- the ordering of leaves by printed form, the monomial keys in evaluation, the radicand spelling in integration, the e-graph's leaf keys, the SymPy code and the JSON form -- uses StringizeAscii, so no answer depends on the setting. StringizeRoundTripTest runs again with the setting on (UnicodeStringizeRoundTripTest), with a theory of the respelled shapes added to both. Part of #1242 Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Codecov Report❌ Patch coverage is Additional details and impacted files@@ Coverage Diff @@
## master #2469 +/- ##
==========================================
+ Coverage 86.95% 86.96% +0.01%
==========================================
Files 320 321 +1
Lines 71457 71600 +143
Branches 20864 20894 +30
==========================================
+ Hits 62132 62267 +135
- Misses 5005 5009 +4
- Partials 4320 4324 +4
Flags with carried forward coverage won't be shown. Click here to find out more.
🚀 New features to boost your workflow:
|
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
MathS.Settings.UnicodeOutput, a scopedSetting<bool>, off by default: with it on,StringizeandToStringprintπ ℯ,+∞ −∞,≤ ≥,≠(fornot a = b),∈ ⊆,≡(a congruence),∀ ∃ ∃!,∪ ∩,¬ ∧ ∨,−(U+2212) for minus, and superscript digits for a plain integer exponent.x ^ 2 - 1 <= piprints asx² − 1 ≤ π. The parser reads exactly those glyphs back. This is the plan posted on the issue on 2026-11-15.^where it would mislead the eye: before another^(x ^ 2 ^ 3isx ^ 2³), and before!,!!or[. Only integer literal exponents are superscripted.notable,1E-20,sin().StringizeAscii(). LaTeX output is unchanged. JSON stays ASCII whatever the flag, so a serialised form does not depend on a display setting.∞stays a parse error, as bareoois a variable (+oo and -oo are members of RR and CC #2220); the printer always writes+∞or−∞.√,·,×stay errors too.≤ ≥ ≠ ∈ ∪ ∩ ¬ ∧ ∨<= >= <> in \/ /\ not and or−(U+2212)-, includinga − b,−x,2 − −3+∞,−∞,-∞+oo,-oox²isx^2,2x²is2*x^2,x²³isx^23,−x²is-(x^2),x⁻¹isx^(-1)π ℯ ⊆ ⊇ ≡ ∀ ∃ ∃!Every new glyph was an error on master, so no input that parses changes meaning, and there is no
BREAKING-CHANGES.mdentry.The grammar changes at the lexer only:
UNICODE_SYNONYM, lexed as the ASCII token it replaces, andSUPERSCRIPT, expanded to^ ninParser.csbefore the implicit-operator pass (also underExplicitParsingOnly). No parser rule changes; the regenerated parser differs by token numbers. Master's grammar was regenerated unchanged first and matched its generated files byte for byte.StringizeRoundTripTestruns every case again with the flag on (UnicodeStringizeRoundTripTest, 313 cases), plus 59 Unicode-sensitive shapes in both modes and checks that ordering, LaTeX and SymPy do not change. The workspace round-trip harness with the flag on: 5,000 expressions, 0 that fail to parse back, 0 that read back with another value, and 0 where the Unicode form reads differently from the ASCII one, again at 20,000 expressions of depth 4.Syntax.mddocuments the setting and the glyphs.Full check on this commit: 20,486 unit tests, 0 failed; MCP 444, Terminal 69, F# 134, Interactive 19; AOT OK; BoundCheck 0 disagreements, RootCheck 596/596, CasBench 120/121 with 0 wrong, PropCheck 1355/0, SimpSweep 10463/10463, RuleCheck clean, CanonCheck and Confluence at their baselines.
Closes #1242.
🤖 Generated with Claude Code