# Incompatibility with mathtools and unicode-math in xelatex?

I find unexpected behaviour in using mathtools in conjunction with unicode-math, which shows up if the 'range' command is used. Some incompatibilities have been noted before, so perhaps this is related. However, it only showed up for me in Texlive2016; previous versions worked as expected. In the following example I wish to draw calligraphic fonts from a different set than the main maths fonts, and some calligraphic symbols, and a regular italic bold face math, are missing.

\documentclass{article}
\usepackage{mathtools}
\usepackage{fontspec}
\usepackage{unicode-math}
\unimathsetup{math-style=ISO,bold-style=ISO,sans-style=italic}
\setmainfont{xits-regular.otf}
\setmathfont{xits-math.otf}
\setmathfont[range={\mathcal,\mathbfcal}]{latinmodern-math.otf}
%\setmathfont[range={}]{xits-math.otf} % Try with and without this line
%
\begin{document}
Test
$a+b = \mathcal{C H G}C \mathscr{D}$
$$-\frac{a}{b} \mathcal{O} .$$
$$\mathrm{d} = \mathcal G(\varphi,x,y,z,t) ,$$
The symbol $\mathcal{G}$ is missing.

The following equation is wrong if I include the line  setmathfont[range=\{\}] \{xits-math.otf\}''
$$\mathup{d} = \symbf{u}(\varphi,x,y,z,t) ,$$
The glyph $\symbf{u}$ is missing.
\end{document}


If mmathtools is called after unicode-math the problem does not arise, but this ordering is not recommend by the unicode-math developers.

My specific question is, is this a bug or is it something I am doing wrong?

• The glyph should be there if you use \symbf instead of \mathbf. The real problem when you add the line with the empty range is that the previous range declaration is lost. – egreg Oct 21 '16 at 13:39
• @egreg GeoffV does not seem to be using \mathbf, which \mathbf are you referring to ? – user4686 Oct 21 '16 at 20:29
• @jfbu Apparently they have been changed. I copied the original version. – egreg Oct 21 '16 at 20:31
• @GeoffV updated answer to fix the unicode-math issue with \newfam. Will make a PR on the developer's site. – user4686 Oct 23 '16 at 15:23

### update:

The github issue related to this and opened by @UlrikeFischer was closed some time ago already at unicode-math v0.8e, and currently unicode-math is at v0.8m. So don't use the hack below.

Update. Here is an attempt at addressing the \newfam issue. We temporarily make innocent \newfam during unicode-math loading (I checked it was used only once; as it is used during patching of mathtools, mathtools must be loaded before unicode-math). Then we patch ourselves the mathtools macro. For this, I use the math family number 255 rather than creating a new one. It is not very probable the document uses 256 math families. One could make this more intrinsic by using \e@mathgroup@top - 1 but I don't know how to subtract 1 in expl3 lingua. Caveat: barely tested.

This addresses only one the two issues. The one of effect of empty range is not handled here.

\documentclass{article}
\usepackage{fontspec}

% *must* be before unicode-math
\usepackage{mathtools}

\makeatletter
\let\ORInewfam\newfam
\def\newfam #1{\typeout{message from \string\newfam: killing myself}}
\usepackage{unicode-math}
\let\newfam\ORInewfam
\makeatother

\unimathsetup{math-style=ISO,bold-style=ISO,sans-style=italic}
\setmainfont{xits-regular.otf}
\setmathfont{xits-math.otf}
\setmathfont[range={\mathcal,\mathbfcal}]{latinmodern-math.otf}
%\setmathfont[range={}]{xits-math.otf} % Try with and without this line

\ExplSyntaxOn
\makeatletter
\def\MT_cramped_internal:Nn #1#2
{
\hbox_set:Nn \l_tmpa_box
{
\color@setgroup
\c_math_toggle_token
\m@th
#1
\dim_zero:N \nulldelimiterspace
\XeTeXradical \c_two_hundred_fifty_five \c_zero { #2 }
\c_math_toggle_token
\color@endgroup
}
\box_set_ht:Nn \l_tmpa_box
{
\box_ht:N \l_tmpa_box
}
\box_use_drop:N \l_tmpa_box
}
\makeatother
\ExplSyntaxOff

\begin{document}

Test
$a+b = \mathcal{C H G}C \mathscr{D}$
$$-\frac{a}{b} \mathcal{O} .$$
$$\mathrm{d} = \mathcal{G}(\varphi,x,y,z,t) ,$$
% The symbol $\mathcal{G}$ is NOT missing.

% The following equation is wrong if I include the line  setmathfont[range=\{\}] \{xits-math.otf\}''
$$\mathup{d} = \symbf{u}(\varphi,x,y,z,t) ,$$
% The glyph $\symbf{u}$ is NOT missing.

Example from mathtools manual

$\cramped{x^2} \leftrightarrow x^2 \quad \cramped[\scriptstyle]{x^2} \leftrightarrow {\scriptstyle x^2}$

\end{document}


code updated 2018/12/28 as per @egreg comment "\box_use_clear:N is deprecated as of 2018-12-31; the right function to use is \box_use_drop:N"

please don't upvote this until it has received confirmation by competent people. It is true that removing some code from unicode-math-xetex.sty makes the thing work, but it may well be that OP's code is not correct use of unicode-math: as I am not familiar with it I can't judge.

please start upvoting this answer massively as I have grown competent enough to be certain of the origin of the problem (see bottom)

ok, you may stop upvoting and start transferring your votes to Ulrike's answer as she had already opened quite some time ago the ticket https://github.com/wspr/unicode-math/issues/345 mentioning the pertubation caused by naked use of \newfam on math alphabets and thus finding in anticipation which code line in unicode-math ultimately resulted in the disappearing glyphs as here in the OP's question.

## Part I

I have identified the source of the problem in file unicode-math-xetex.sty.

Steps to reproduce:

kpsewhich unicode-math-xetex.sty
copy the file to your working repertory
comment out lines #L2040 to #L2091 inclusive


Then your document compiles fine with the expected glyphs with both mathtools and your  \setmathfont[range={}]{xits-math.otf}

Note: the code of unicode-math looks strange with the \AtEndOfPackageFile * { mathtools } nested into another one... but I checked balancing of braces. Thus, lines 2038 and 2039 must remain non-commented.

It could be that the important thing in the commented out code is that it does \newfam \g__um_empty_fam creating a new \mathgroup. This may cause a problem of numbering ?

Not being familiar with unicode-math I can't comment more. But it does look more like a problem in unicode-math than one in mathtools (as far as I can tell.)

extra info: I have checked that \mv@normal has exact same meaning in the two situations, confirming that there may be an offset in numbering of math families on the unicode-math side.

image showing it does work: (the math calligraphic are from XITS not Latin Modern, due to effect of range={})

## Part II

Here is a minimal example to demonstrate the bug lies in unicode-math (see the various comments below this answer for more context).

\documentclass{article}

\usepackage{unicode-math}

\newfam\MyFam  % comment this to get correct glyph rather than invisible one

\setmainfont{lmroman10-regular.otf}
\setmathfont[range={cal}]{latinmodern-math.otf}

\begin{document}

$\mathrm{d} = \mathcal{G}$

\showoutput
\end{document}

%%% Local Variables:
%%% TeX-engine: xetex
%%% End:


The use of \newfam before \setmainfont/\setmathfont (the latter using the cal not the deprecated range={\mathcal}) is enough to create the problem: . Without the \newfam we get .

In case some doubt arises because Latin Modern is default use

\setmathfont[range={cal}]{texgyretermes-math.otf}


I have opened a ticket at https://github.com/wspr/unicode-math/issues/368.

And the point is that the single (direct) use of \newfam in unicode-math code is in the line

    \newfam \g__um_empty_fam


which is encountered at the start of its patch of mathtools which in Part I I advised to comment out. If this patch is executed too soon in the preamble a problem arises in using \math.. alphabet macros: the wrong font is used. This may have to do with unicode-math way of patching \use@mathgroup but I have not pursued further because the LaTeX code for math fonts is very complex and whenever I stay a few months without looking at it I forget it entirely and irrevocably, and it would take me quite some effort to re-immerse me into it, not to mention that I would need to read unicode-math code as well.

• kudos on the warning. i'm not upvoting, as warned, since i'm not sufficiently competent, but it does sound like you've uncovered some very useful and relevant information. – barbara beeton Oct 21 '16 at 18:27
• I confirm what I wrote in my comment to the question. There is no missing symbol, but the calligraphic letters revert to the XITS ones. I get exactly the same output with or without the lines you tell to remove. – egreg Oct 21 '16 at 20:39
• @egreg Yes, the calligraphic letters do revert to the XITS ones due to effect as you said of \setmathfont[range={}]{xits-math.otf}, and the image above shows them. I don't understand why you get the same output. If I don't remove the lines I get different output where both the calligraphic G and the bold u are missing and replaced by open squares. – user4686 Oct 21 '16 at 21:03
• due to a moment of confusion I suppressed a comment here which was confirming my sayings, to be brief it is all as in my answer. Only if I patch unicode-math I get correct output with or without the range={} (in case it is used the calligraphic revert to XITS as in my image). Else: Missing character: There is no 𝒢 in font [xits-regular]/OT:script=latn;language=DFLT;! – user4686 Oct 21 '16 at 21:13
• I knew I had seen it already ... github.com/wspr/unicode-math/issues/345 – Ulrike Fischer Oct 22 '16 at 18:01

I don't quite understand why you are using an empty range, but imho your syntax is wrong. You are still using in many place \mathXX commands where \symXX should be used, and in the range command you should use the names for the blocks and not commands. This here works fine for me on texlive 2016:

\documentclass{article}
\usepackage{mathtools}
\usepackage{fontspec}
\usepackage{unicode-math}
\unimathsetup{math-style=ISO,bold-style=ISO,sans-style=italic}
\setmainfont{xits-regular.otf}
\setmathfont{xits-math.otf}
\setmathfont[range={cal,bfcal}]{latinmodern-math.otf}
\setmathfont[range={}]{xits-math.otf} % Try with and without this line
%
\begin{document}
Test
$a+b = \symcal{C H G}C \symscr{D}$
$$-\frac{a}{b} \symcal{O} .$$
$$\symrm{d} = \symcal G(\varphi,x,y,z,t) ,$$
The symbol $\symcal{G}$ is missing.

The following equation is wrong if I include the line  setmathfont[range=\{\}] \{xits-math.otf\}''
$$\symup{d} = \symbf{u}(\varphi,x,y,z,t) ,$$
The glyph $\symbf{u}$ is missing.
\end{document}


• True that it should be \symrm, but \mathrm is allowed anyway (for different purposes). Since using \mathrm causes the disparition of the symbols, it's clear that there is a problem. Second issue: a further range declaration cancels the previous one. – egreg Oct 21 '16 at 21:53
• @egreg: Somehow my comment from yesterday night got lost: You are naturally right, that \mathrm is allowed but imho unicode-math is complicated enough so one should at least start with correct documents when investigating ... (I will try to look into the range problem) – Ulrike Fischer Oct 22 '16 at 18:04
• @egreg: In part (regarding symbols) \newfam ("which one shouldn't use") is responsable for the range problem too, but the main problem is your old bug report:github.com/wspr/unicode-math/issues/331 – Ulrike Fischer Oct 22 '16 at 21:27
• With the latest unicode-math and mathtools packages I no longer see the errant behaviour. If indeed the the bug is fixed perhaps the question should be closed? – GeoffV Jan 15 '18 at 10:53
• What version are you using that avoids the errors? – Peter Gerdes Feb 14 '18 at 20:46