mhchem arrows produce PUA characters in AssistiveMML, causing broken speech text (MathJax 4+)

[brichwin]

Issue Summary

When using the mhchem extension to render chemical equations with arrows (e.g., \ce{SO4^2- + Ba^2+ -> BaSO4 v} ), MathJax 4.x produces Private Use Area (PUA) characters in the AssistiveMML output. These PUA characters are passed to the Speech Rule Engine (SRE), resulting in tofu characters (□) in the generated speech text and braille.

This issue exists both in MathJax 4.0 and in MathJax 4.1. MathJax 3.2.2 uses standard Unicode characters and does not have the issue. The issue exists with multiple types of mhchem reaction arrow tested, not just -> used in the examples here.

Steps to Reproduce:

LaTeX input:

\ce{SO4^2- + Ba^2+ -> BaSO4 v}

Any other information you want to share that is relevant to the issue
being reported. Especially, why do you consider this to be a bug? What
do you expect to happen instead?

Expected Behavior (as seen in MathJax 3.2.2)

The <mo> element for the reaction arrow uses the standard Unicode character U+27F6 (LONG RIGHTWARDS ARROW):

<mo stretchy="false" data-semantic-type="relation" data-semantic-role="arrow" 
    data-semantic-id="16" data-semantic-parent="31" 
    data-semantic-operator="multirel">&#x27F6;</mo>

Speech text output:

upper S upper O Subscript 4 Baseline Superscript 2 minus Baseline plus 
upper B a Superscript 2 plus Baseline long right arrow 
upper B a upper S upper O Subscript 4 Baseline down arrow

The arrow is spoken meaningfully as "long right arrow".

Actual Behavior (MathJax 4.0 and 4.1)

The <mo> element for the reaction arrow contains a PUA character (the element contains U+E429):

MathJax 4.1:

<mo data-mjx-variant="-mhchem" data-mjx-texclass="REL" stretchy="true" 
    data-latex="\mhchemlongrightarrow" data-semantic-type="operator" 
    data-semantic-role="unknown" data-semantic-annotation="depth:3" 
    data-semantic-="" data-semantic-parent="30" 
    data-semantic-attributes="latex:\mathrel{\mhchemlongrightarrow};texclass:REL" 
    data-semantic-operator="infixop," data-semantic-level-number="2" 
    data-speech-node="true"></mo>

MathJax 4.0:

<mo data-mjx-variant="-mhchem" data-mjx-texclass="REL" stretchy="true" 
    data-latex="\mhchemlongrightarrow" data-semantic-type="operator" 
    data-semantic-role="unknown" data-semantic-annotation="depth:3" 
    data-semantic-="" data-semantic-parent="30" 
    data-semantic-attributes="latex:\mathrel{\mhchemlongrightarrow};texclass:REL" 
    data-semantic-operator="infixop," aria-level="2" 
    data-speech-node="true"></mo>

Speech text output (both 4.0 and 4.1):

SO sub 4 raised to the 2 minus power plus 
Ba raised to the 2 plus power  BaSO sub 4 down arrow, math

The arrow is rendered as a tofu character (□) in the speech output because SRE does not map the PUA character to spoken text.

Technical details:

Affected Versions

• MathJax 4.0.0 - Bug present
• MathJax 4.1.0 - Bug present
• MathJax 3.2.2 - Works as expected

I am using the following MathJax configuration (MathJax 4.1):

    window.MathJax = {
      loader: { load: ['[tex]/mhchem'] },
      tex: { packages: { '[+]': ['mhchem'] } },
      svg: { fontCache: 'none' },
      options: {
        enableMenu: true,
        menuOptions: {
          settings: {
            enrich: true,
            speech: true,
            braille: false,
            collapsible: false,
            assistiveMml: true
          }
        }
      },
      startup: {
        pageReady: () => {
          return MathJax.startup.defaultPageReady().then(() => {
            setTimeout(extractAssistiveMML, 100);
          });
        }
      }
    };

and loading MathJax via

  <script id="MathJax-script" async src="https://cdn.jsdelivr.net/npm/mathjax@4.1/tex-svg.js"></script>

Supporting information:

• link to a (live) minimal example page
• A screen snapshot that illustrates the problem:

[dpvc]

Thanks for the report. You are correct that this interferes with the speech generation. The reason for the use of PUA characters is twofold: first, there are several characters used in mhchem that don't have Unicode positions (like the bonds produced by \ce{A\bond{~--}B\bond{~=}C\bond{-~-}D}, and the arrows in \ce{A <=>> B} and \ce{A <<=> B}); and second, because version 4.0 uses a font extension to load these characters and the pieces needed to create stretchy arrows. In v4, the mhchem arrows always use versions based on the mathjax-newcm arrows so that all the arrows in an expression will be consistent. The problem is that MathJax's handling of stretchy characters (like the arrows used in mhchem) doesn't provide for having separate versions of the same arrow from different fonts. Therefore, the mhchem font uses arrows in the PUA in order not to override the ones in the main font being used (even if that is the mathjax-newcm font).

In order to overcome this, MathJax's handling of delimiters needs to be extended to allow different stretchy characters with the same Unicode position from different fonts. That is not a minor change, so I'm not sure when it will happen.

It might be possible in the meantime to improve the results from the speech-rule-engine for these PUA characters.

[NSoiffer]

FYI: Unicode 17 adds support for the arrows used for \ce{A <=>> B} (U+1F8D2) and \ce{A <<=> B} (U+1F8D3). There are several other additions for chemistry including the 'Plimsoll mark" used for standard state (U+1CEF0).

The bonds were not added. I, along with several other people, submitted the proposal for the 10 new symbols that Unicode added. I'm not a chemist and no one else mentioned the bond symbols, but if we can show some books that use those symbols, it is very likely a proposal to add them would be accepted.

Needless to say, using private use areas is very problematic. Earlier versions of MathType make extensive use of them. Thankfully, it seems that MathType started at 0xe900 and goes up to 0xf55f, so there is no conflict with MathJax's usage.

The fewer private use characters that are used, the less likely there is collision and the more likely there is compatibility with other apps. This is particularly important when exporting to a standard like MathML or LaTeX. @dpvc explained why internally they are needed at the moment (to distinguish rendering), but that doesn't explain why those characters that have Unicode code points can't be mapped to those code points when exporting to MathML or LaTeX. @dpvc: is that an option?