8

Dvi files reference glyphs by their unicode number. However, a document often contain some unencoded glyphs, such as large math symbols. By inspecting dvi files containing such glyphs, I conclude that these glyphs are referenced by some non-unicode (Private Use Area) character codes.

How can I know which character code LuaTeX will choose for a certain glyph, say, the display version of the integral sign? The same question goes for XeTeX, in case they are different.

Example:

\documentclass{standalone}
\usepackage{fontspec}
\usepackage{unicode-math}
\setmainfont{XITS}
\setmathfont{XITS Math}
\begin{document}
$\int\displaystyle\int$
\end{document}

When compiled with LuaLaTeX, using the --output-format=dvi option, the small integral is encoded as 0x222B (as expected) and the big integral is encoded as 0x0F001C (somewhat mystifying).

Update:

Some experimentation confirms that the encoding of the integral sign differs between fonts. The display version of the integral is 0x0F05C2 in Cambria Math, 0x10FF99 in Asana Math and 0x0F0314 in STIX Math, for instance.

9
  • There is a mapping between glyphs and character codes in OpenType fonts. When a ligature is created for example f_i, the map is looked for this glyph and returned PUA is inserted.
    – michal.h21
    Dec 13, 2015 at 19:53
  • @michal.h21, It is my understanding that OpenType fonts contain glyphs that are not mapped in the CMAP table. My question is how these glyphs are coded in the dvi file.
    – akvilas
    Dec 13, 2015 at 20:30
  • 1
    I'm not sure one can use OpenType fonts when the output is DVI. Can you make an example?
    – egreg
    Dec 13, 2015 at 20:40
  • 1
    @egreg luatex hapilly produces DVI file with OpenType fonts. The question is whether there is any tool which can work with them (tex4ht, dvipng, dviasm and dvitype cannot)
    – michal.h21
    Dec 13, 2015 at 20:52
  • @egreg: In 2011 I once wrote "The dvi's generated by luatex (if I inspect them with dv2dt) look actually quite similar to dvi's generated by latex. E.g. they contain - if I use an unicode font and input an € - the entry "s1 8364" (8364 = hex 20AC)." But I don't know if this is still valid. Dec 13, 2015 at 20:55

1 Answer 1

9

LuaTeX uses a pretty simple algorithm to choose the DVI character codes: While reading the OTF file, it checks whether a character has a Unicode point assigned or not. If there's an entry in the Unicode mapping table, LuaTeX uses the Unicode point. Otherwise, it populates the Unicode range starting at 0xF0000. The first unmapped character is assigned to 0xF0000, the second one to 0xF0001 etc. Thus, you can only predict the DVI character codes if you evaluate the Unicode table of the OTF fonts yourself. Especially, there's no relation between the code and the semantics of an unmapped character, i.e. the big integral sign can get different DVI codes depending on the mapping tables and the characters present in the various font files.

XeTeX, on the other hand, simply uses the glyph indexes of the characters in an OTF file if the characters are set with XDV command XGlyphArray or XGlyphString. Each glyph in a font has a unique internal value with no relation to any encoding. The encoding itself is realized by the already mentioned encoding tables that, for example, map from Unicode or ISO-8859-1 to glyph indexes. So, XeTeX ignores the encodings present in the font file and uses the glyph indexes directly -- more precisely, those provided by the FreeType library (which may rearrange the values).

2
  • Great answer! Is this behavior documented, or is it buried deep within the XeTeX/LuaTeX source code?
    – akvilas
    Dec 14, 2015 at 18:35
  • I don't think these details are documented in some kind of manual -- at least I'm not aware of any. I got the information from the source codes and some LuaTeX details from earlier correspondence with Khaled Hosny.
    – Martin
    Dec 14, 2015 at 19:37

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .