8

Dvi files reference glyphs by their unicode number. However, a document often contain some unencoded glyphs, such as large math symbols. By inspecting dvi files containing such glyphs, I conclude that these glyphs are referenced by some non-unicode (Private Use Area) character codes.

How can I know which character code LuaTeX will choose for a certain glyph, say, the display version of the integral sign? The same question goes for XeTeX, in case they are different.

Example:

\documentclass{standalone}
\usepackage{fontspec}
\usepackage{unicode-math}
\setmainfont{XITS}
\setmathfont{XITS Math}
\begin{document}
$\int\displaystyle\int$
\end{document}

When compiled with LuaLaTeX, using the --output-format=dvi option, the small integral is encoded as 0x222B (as expected) and the big integral is encoded as 0x0F001C (somewhat mystifying).

Update:

Some experimentation confirms that the encoding of the integral sign differs between fonts. The display version of the integral is 0x0F05C2 in Cambria Math, 0x10FF99 in Asana Math and 0x0F0314 in STIX Math, for instance.

  • There is a mapping between glyphs and character codes in OpenType fonts. When a ligature is created for example f_i, the map is looked for this glyph and returned PUA is inserted. – michal.h21 Dec 13 '15 at 19:53
  • @michal.h21, It is my understanding that OpenType fonts contain glyphs that are not mapped in the CMAP table. My question is how these glyphs are coded in the dvi file. – akvilas Dec 13 '15 at 20:30
  • 1
    I'm not sure one can use OpenType fonts when the output is DVI. Can you make an example? – egreg Dec 13 '15 at 20:40
  • 1
    @egreg luatex hapilly produces DVI file with OpenType fonts. The question is whether there is any tool which can work with them (tex4ht, dvipng, dviasm and dvitype cannot) – michal.h21 Dec 13 '15 at 20:52
  • @egreg: In 2011 I once wrote "The dvi's generated by luatex (if I inspect them with dv2dt) look actually quite similar to dvi's generated by latex. E.g. they contain - if I use an unicode font and input an € - the entry "s1 8364" (8364 = hex 20AC)." But I don't know if this is still valid. – Ulrike Fischer Dec 13 '15 at 20:55
9

LuaTeX uses a pretty simple algorithm to choose the DVI character codes: While reading the OTF file, it checks whether a character has a Unicode point assigned or not. If there's an entry in the Unicode mapping table, LuaTeX uses the Unicode point. Otherwise, it populates the Unicode range starting at 0xF0000. The first unmapped character is assigned to 0xF0000, the second one to 0xF0001 etc. Thus, you can only predict the DVI character codes if you evaluate the Unicode table of the OTF fonts yourself. Especially, there's no relation between the code and the semantics of an unmapped character, i.e. the big integral sign can get different DVI codes depending on the mapping tables and the characters present in the various font files.

XeTeX, on the other hand, simply uses the glyph indexes of the characters in an OTF file if the characters are set with XDV command XGlyphArray or XGlyphString. Each glyph in a font has a unique internal value with no relation to any encoding. The encoding itself is realized by the already mentioned encoding tables that, for example, map from Unicode or ISO-8859-1 to glyph indexes. So, XeTeX ignores the encodings present in the font file and uses the glyph indexes directly -- more precisely, those provided by the FreeType library (which may rearrange the values).

  • Great answer! Is this behavior documented, or is it buried deep within the XeTeX/LuaTeX source code? – akvilas Dec 14 '15 at 18:35
  • I don't think these details are documented in some kind of manual -- at least I'm not aware of any. I got the information from the source codes and some LuaTeX details from earlier correspondence with Khaled Hosny. – Martin Dec 14 '15 at 19:37

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.