Comparing the path from \int or ∫ in the .tex file to ∫ in the .pdf file with type1 and OpenType math fonts in luatex

I'm trying to understand better how math fonts are handled and, in particular, how an OpenType math font can do the same job as a type1 font with far less files. My reference engine is luatex (version beta-0.80.0 (TeX Live 2015) (rev 5238)).

So consider the following files and their outputs:

\documentclass[varwidth]{standalone}
\usepackage{newunicodechar}
\newunicodechar{∫}{\int}
\usepackage{lmodern}
\begin{document}
$\int ∫$
$$\int ∫$$
\end{document}


and

\documentclass[varwidth]{standalone}
\usepackage{unicode-math}
\setmathfont{Latin Modern Math}
\begin{document}
$\int ∫$
$$\int ∫$$
\end{document}


I'd like to understand how the \int or the unicode input ∫ is finally rendered to ∫ in the .pdf output file in the eight cases above (type1 and OpenType, \int and ∫ inputs, inline and displaystyle math). Would anyone be kind enough to give me a detailed explanation?

Side note: notice the difference in spacing. Why?

• The OTF file includes the information that is divided into TFM and PFB in the traditional setup; the information usually in the .fd file is built on the fly by fontspec. – egreg May 12 '16 at 13:00
• @egreg Ok so my question is maybe about the internals of fontspec and about the way LaTeX treats operators like \int. – cjorssen May 12 '16 at 13:52

In the first case (an 8-bit engine), the set up

\newunicodechar{∫}{\int}


uses active characters to make the multi-byte sequence ∫ expand to the control sequence \int. It is that which is therefore important. A quick \show reveals that \int expands to \intop\nolimits and that \intop is defined (\mathchardef) as \mathchar"1352. That's then the 'raw' instruction that TeX takes forward outside of the macro layer. As you can read in detail in The TeXbook, the \mathchar here is defined as a 'large operator' (class 1), with the glyph taken from family 3 and in position "52 in that font. The various font dimensions needed to deal with the size of the glyph, spacing, etc., are read by TeX when it loads the font metric from the .tfm file. (TeX itself doesn't actually use the glyph, which is inserted into the output by the driver.)

In the Unicode case (with unicode-math), \int is \let to ∫, which itself is math active and expands to \intop\nolimits (once again). Now \intop is defined as a Unicode math char (\Umathchar"1"0"222B). Once again this is a large operator, here taken from font 0, with glyph position "222B. Unlike the classical situation, Unicode math fonts are 'self contained' (the fontdimens needed are part of the .otf file itself) and there is no special requirement to use a particular font number: this is why we can use font 0 or any other for the glyph here. Once gain, at the point we get to a \Umathchar the involvement of the macro level ends and the engine deals with sizing and positioning the glyph based on the data in the .otf file.

The size of the 'large operator' characters used is determined by the current style: \displaystyle, \textstyle, etc. In the classical situation, the glyph set up using \mathchar is the 'right' size for use in \textstyle (an inline formula). The .tfm file will then specify a 'successor' glyph to be used in \displaystyle, and this will be used in display equations. There are no 'small' glyphs to be selected automatically, so subscript \int will come out 'too big'. In the Unicode case, the same idea of styles applies but now we have only one glyph slot. This will be scaled using information in the font (.otf) by the font shaper (classical TeX does not use a font shaper at all). As this can scale both ways, a 'smaller' \int will appear if used in a subscript.

In terms of the meaning of font slots, the classical TeX fonts are what they are: there is no requirement that any particular slot has any particular meaning. Indeed, as each math font contains only 128 slots, you need to know the semantics of the font file to establish the meaning/appearance of the glyphs. In contrast, Unicode fonts have a clearly defined set of meanings for each position. Thus any well-formed Unicode math font will have the integral sign at position "222B. Indeed, one can check UnicodeData.txt to find out the logical meaning of all currently-assigned slots in Unicode.

(Spacing differences may be down to different font dimensions or may be due to issues in the engines: the Unicode math area is still very much in the development stage.)

• Thanks, that's the kind of answer I was looking for (+1). Two questions more. (a) How does the engine deal with inline math and display math: are the glyphs used the same? (b) Is glyph position "222B a standard or does it depend on the font itself? – cjorssen May 12 '16 at 15:05
• Thanks again! One more (hopefully last) thing... "This will be scaled using information in the font (.otf) by the font shaper": if I understand correctly, it is fontspec that is at work here and, in the end, the engine (luatex in my case). How is this information read in the .otf? – cjorssen May 12 '16 at 16:11
• BTW, UnicodeData.txt is part of my TeXlive distribution: /usr/local/texlive/2015/texmf-dist/tex/generic/unicode-data/UnicodeData.txt. – cjorssen May 12 '16 at 16:14
• @cjorssen Yes, I know: I decided it was a good idea to have it available from TeX rather than looking up every time on the Unicode Consortium website. :-) (The data is there primarily as it is now used at format-construction time by XeTeX and LuaTeX for setting up the various extended TeX data structures that need input from Unicode.) – Joseph Wright May 12 '16 at 16:17
• @cjorssen fontspec doesn't do any shaping: in the end if is a clever interface for the \font primitive. In XeTeX, \font is extended 'out of the box' to read .otf files, which uses the HarfBuzz font shaper code, built-in to that engine. In LuaTeX you need to load some Lua code to do the same: 'out of the box' LuaTeX does not extend \font. The Lua font loader is big (10k lines plus): this is not at all trivial! – Joseph Wright May 12 '16 at 16:19