# compilation problem: “Extended mathchar used as mathchar” with mathastext

When compiling a file using XeLaTex in TeXShop on Mac OS X, the console persistently hangs after the detection of the XeTeX driver:

*geometry* driver: auto-detecting
*geometry* detected driver: xetex
./paper.tex:53: Extended mathchar used as mathchar (71311379).
\relax
l.53 \begin{document}


It's possible to continue the compilation, but I'd like to understand what is going on. The problem seems to be related to the mathastext package, which I use to allow math and text formatting both to use the TeX Gyre Termes font; commenting out the \usepackage line for mathastext relieves the problem.

I've seen discussions from 2006 (long predating the existence of mathastext) saying that there is a bug in the XeTeX distro (v. 0.995 rev. D) and the only solution is to compile from source. (Google search results here.) The version of XeTeX currently installed on Mac OS 10.6.7 by TeX Live is 0.9997.4, but the problem persists, and I wonder if there is any newer intelligence on how to repair it or what causes it. I am having trouble building XeTeX manually.

Edit, 20110505

Because the discussion below became quite involved, I am summarizing the two main conclusions:

• There turns to be a bug in amsmath that requires the option noendash to be passed to mathastext. An alternate fix is offered by Philipp.
• Discussion of how to get math formatting to apply to non-math text, without looking like math mode, was transfered to a different posting: applying math formatting to a non-math font.

Edit: minimal example, as requested by Philipp

%!TEX TS-program = xelatex
%!TEX encoding = UTF-8 Unicode

\documentclass[12pt]{article}
\usepackage{geometry}
\usepackage{amsmath}
\usepackage{mathspec}            % replaces amssymb; do not use unicode-math
\usepackage[italic]{mathastext}  % propagates document text font to math mode
\usepackage{xeCJK}
\usepackage{fontspec}

\defaultfontfeatures{Mapping=tex-text}
\setromanfont{TeX Gyre Termes}

\begin{document}

The expected number of collisions in a given unit is $E(collisions)=k\times\left(1-\left(1-\left(1/n\right)\right)^{{\left(k-1\right)}}\right)$

\end{document}


Another edit, 2011.05.04, replying to JFBU's comment about [no-math].

With no-math being passed to fontspec:

\usepackage[no-math]{fontspec}
...
\setmainfont[Scale=1,Mapping=tex-text]{TeX Gyre Termes}
...
\usepackage[italic,noendash]{mathastext} % set *after* setmainfont
...
\begin{document}
au$$_{\rm3yā}$$
\end{document}


I get . But without no-math being passed to fontspec:

\usepackage{fontspec}
...
\setmainfont[Scale=1,Mapping=tex-text]{TeX Gyre Termes}
...
\usepackage[italic,noendash]{mathastext} % set *after* setmainfont
...
\begin{document}
au$$_{\rm3yā}$$
\end{document}


I get . I believe these are the same; I chose ā because the height of the macron is very distinctive in Termes.

Edit, 2011.05.04b: In response to another comment of @JFBU, appearance of \centerline{0123456789}\centerline{$0123456789$} with and without no-math passed to fontspec.

With no-math:

W/o no-math:

I see no obvious difference.

-
Could you please post a minimal example? –  Philipp May 3 '11 at 13:29
hello, I was idling on this site when I saw your message pop up. mathastext has hardly been tested with unicode engines (I know from first-hand source that the author does not even have a working xe(la)tex on his system... and only did a bit of testing with LuaLaTeX). The source of your problem may indeed quite probably be with mathastext. Try loading it with the basic option to see if it helps. –  jfbu May 3 '11 at 13:33
@brannerchinese: (breaking news) the author of mathastext has informed me that at his office, they have TL2010 since a few days, and that for the first time he could try out XeLaTeX. He says that a minimal document works fine:  \documentclass{article} \usepackage[no-math]{fontspec} \setmainfont{Andale Mono} \usepackage{mathastext} \setmainfont{Arial Black} \Mathastext[Arial] \begin{document} \MTversion{normal} Hello $world$ \MTversion{Arial} Hi $again$ \end{document} so please provide a minimal example. –  jfbu May 3 '11 at 13:45
@Philipp: Minimal example included in edit. –  brannerchinese May 3 '11 at 14:22
@brannerchinese: as a side remark, when using mathastext, one should load it after the setmainfont command. To return to your problem, it goes away when amsmath is not loaded. This should help locate the source of the error in mathastext. Will transmit to its author. –  jfbu May 3 '11 at 14:41

There is indeed a problem due it seems to an interaction between amsmath and mathastext. As a temporary fix, load mathastext with option noendash.

EDIT [2012/10/18]. Versions 1.15x of mathastext uploaded to CTAN late September have incorporated a work-around to this amsmath+Unicode fonts problem. So it is not necessary to use the noendash option anymore. END OF EDIT.

The following compiles on my system:

\documentclass[12pt]{article}
\usepackage{geometry}
\usepackage{amsmath}
\usepackage{mathspec}
\usepackage{xeCJK}
\usepackage{fontspec}

\defaultfontfeatures{Mapping=tex-text}
\setromanfont{Times New Roman}

\usepackage[italic,noendash]{mathastext}

\begin{document}

The expected number of collisions in a given unit is $E(collisions)=k\times\left(1-\left(1-\left(1/n\right)\right)^{{\left(k-1\right)}}\right)$

\end{document}


I switched to Times New Roman, as the TeX Gyre fonts are not available on the system I am currently on. I will edit my answer once I understand better what's going on.

Ok, it seems that the problem is related to the presence of the following two lines in amsmath.sty (lines 745-747)

\AtBeginDocument{%
\mathchardef\std@minus\mathcode\-\relax
\mathchardef\std@equal\mathcode\=\relax
}


and to the fact that mathastext redefines the minus sign in math mode with the following instruction:

\XeTeXmathcode\- ="2 \symmtoperatorfont "2013 \relax


Furthermore the file amsopn.sty (loaded by amsmath) makes a definition of DeclareMathOperator which seems to involve also a certain macro which will contain \mathcode'\- (read the ' as a backtick). It seems that once mathastext has done its job with the line above to use the en-dash as minus sign, any call to \mathcode'\- throws an error, as one sees by compiling just the following two lines

\XeTeXmathcode\- ="2 "0 "2013 \relax
\mathchardef\coucou\mathcode\- \relax


in a TeX source with no package at all. So this situation creates a general incompatibility between mathastext and amsmath (same for LuaTeX as engine !!), for which the only (due to the AtBeginDocument above) current work-around I see is the following:

1. load mathastext with option noendash
2. immediately after \begin{document} put the line written below
3. if you use LuaLaTeX rather than XeLaTeX put the second line.

Line for XeTeX:

\XeTeXmathcode\- ="2 \symmtoperatorfont "2013 \relax  % xetex


Line for LuaLaTeX:

\luatexUmathcode\-="2 \symmtoperatorfont "2013 \relax % lualatex


Hope it helps. The author of mathastext should put this stuff in an AtBeginDocument instruction, but only if it is guaranteed this will be processed by TeX after the amsmath stuff. The following works on my system:

\documentclass[12pt]{article}
\usepackage{geometry}
\usepackage{amsmath}
\usepackage{mathspec}
\usepackage{xeCJK}
\usepackage{fontspec}

\defaultfontfeatures{Mapping=tex-text}
\setromanfont{Times New Roman}

\usepackage[noendash]{mathastext}

\begin{document}
\XeTeXmathcode\- ="2 \symmtoperatorfont "2013 \relax

The expected number of collisions in a given unit is $E(collisions)=k\times\left(1-\left(1-\left(1/n\right)\right)^{{\left(k-1\right)}}\right)$

a\textendash b

$a-b$

\end{document}

-
Compiles here with TeX Gyre Termes. –  brannerchinese May 3 '11 at 15:11
@brannerchinese: ok thanks for the info. I have now provided a more complete answer, hopefully it will work on your system. –  jfbu May 3 '11 at 16:07
as an aside, the doc of mathastext says to load fontspec with option no-math. In the small example above this does not seem to matter, but I don't know in general. Anyhow this is completely unrelated to the question addressed here. –  jfbu May 3 '11 at 16:11
in fact the reason for loading fontspec with option no-math when using mathastext is the following: without it the 0,1,2,3,4,5,6,7,8,9,:,;,=,+,-,(,),[,],/,! and ? characters mysteriously escape the mathastextization. An alternative is to immediately follow \usepackage{fontspec} with: \ExplSyntaxOn \bool_set_false:N \g_fontspec_math_bool \ExplSyntaxOff –  jfbu May 3 '11 at 18:08
@jfbu: I see no difference with and without option no-math in my own code, but I'll leave it in anyway. –  brannerchinese May 3 '11 at 18:12

There is a glitch in amsmath that prevents it from working whenever a Unicode math font is used. Here is a fix for XeTeX:

\documentclass{article}
\usepackage[no-math]{fontspec}

\ExplSyntaxOn
\int_new:N \l_mathcode_minus_int
\int_new:N \l_mathcode_equal_int
\exp_args:Nx \AtBeginDocument {
\exp_not:n {
\int_set:Nn \l_mathcode_minus_int { \XeTeXmathcodenum \- }
\int_set:Nn \l_mathcode_equal_int { \XeTeXmathcodenum \= }
}
\mathcode \int_eval:n { \- } = \number \mathcode \- \scan_stop:
\mathcode \int_eval:n { \= } = \number \mathcode \= \scan_stop:
}
\usepackage{amsmath}
\AtBeginDocument {
\XeTeXmathcodenum \- = \l_mathcode_minus_int
\XeTeXmathcodenum \= = \l_mathcode_equal_int
}
\ExplSyntaxOff

\usepackage[italic]{mathastext}
\setromanfont{TeX Gyre Termes}

\begin{document}

The expected number of collisions in a given unit is $E(collisions)=k\times\left(1-\left(1-\left(1/n\right)\right)^{{\left(k-1\right)}}\right)$

\end{document}

-
@Philipp: great! the author of mathastext will be relieved to see he is not 100% the culprit ;-) by the way in your code the \usepackage[italic]{mathastext} should be after \setromanfont (check for the fonts included in the pdf and you will see that if you do not permute you get LatinModern in math mode rather than TeX Gyre Termes). –  jfbu May 3 '11 at 16:45
@Philipp: will this fix to amsmath be somehow incorporated transparently one day in the Unicode engines? are there other problems related with amsmath in this regard? –  jfbu May 3 '11 at 18:14
I don't know what the policy regarding such engine-specific fixes is, but I'll ask on the LaTeX mailing list. Yes, there are numerous problems caused by the switch to Unicode fonts, see for example the fixes incorporated in the unicode-math and the lualatex-math packages. –  Philipp May 3 '11 at 19:18
@Philipp: thanks for the info. –  jfbu May 3 '11 at 22:51
Answer from Will: “Speaking for myself, I guess it largely depends. We've spoken here before about adding a switch to deactivate inputenc in XeLaTeX/LuaLaTeX but no proposals ever got off the ground. As for amsmath, that's still maintained by the AMS, and I believe they're currently working on an update to that at the moment -- it would be best to contact them directly. (I'm not sure who the best contact there would be.)” –  Philipp May 4 '11 at 12:22