Let luatex output a list of glyphs and their positions

Question

In the Python plotting library Matplotlib (https://matplotlib.org), text can be rendered using either Matplotlib's own text layout algorithm (which includes a parser for a subset of TeX-like mathmode syntax), or using latex: the text is wrapped in a TeX source file, which is processed by (la)tex,

for raster backends, on the one side we parse the dvi file (using a custom-written parser, matplotlib.dviread) to extract the baseline of the glyphs (so that multiple text objects can be aligned relative to their baselines), and on the other we convert the dvi to png with dvipng, then load the png image and render it to Matplotlib's canvas;
for vector backends (pdf/ps/svg), we parse the dvi file to extract the baseline of the glyphs and the list of glyphs and boxes (fraction bars, for example, are represented as rectangular boxes in the dvi file) themselves, then render the glyphs and boxes to the vector file, again taking the baseline into account.

I would like to add support for using luatex instead of tex; mostly for improved font support. Unfortunately, as noted elsewhere[1], luatex's dvi output appears to not be parseable in the same way as tex's (due to different references to fonts) and also confuses dvipng. Instead of trying to fix the parser, I was wondering whether it would be possible to use luatex's callbacks to directly output a list of glyphs, i.e., a list of {path to font file + face index + glyph index + glyph coordinates}, and a list of boxes?

(The face index is the index of the face in the font file when a file (e.g. in .ttc format) contains multiple faces, i.e. the third argument to https://www.freetype.org/freetype2/docs/reference/ft2-base_interface.html#FT_Open_Face.)

[1] How are non-unicode glyphs referenced in LuaTeX .dvi files?; LuaLaTeX/dvipng: conversion fails due to missing font

Thanks in advance.

What is a “face index” and what do you mean by “output a list of boxes”? — Henri Menke, Dec 14 '18 at 21:27
I clarified the definition of face index and box above. (The face index specifies the face for font files that embed multiple faces, and boxes are used e.g. to draw fraction bars.) — Antony Lee, Dec 15 '18 at 15:34
Is this different from the currently listed support of LuaLaTeX in the pgf backend? — aoi, Dec 15 '18 at 16:56
Yes. The pgf backend generates some pgf code for the entire plot and compiles the whole thing with a tex engine, whereas here I only want to exploit tex's ability to position glyphs. (The pgf backend does have the ability to figure out the baseline of a block of text by saving it with \sbox0 and then displaying \wd0/\ht0/\dp0, but that doesn't tell me where each individual glyph is.) — Antony Lee, Dec 15 '18 at 21:40

Henri Menke · Accepted Answer · 2018-12-16 20:47:07Z

It is pretty easy to get the font filename and the glyph index. I don't know what you mean by “face index”. Getting the position on the page is much more tricky and is not contained in the example below, because at the point where the mlist_to_hlist callback is triggered it is not yet know where on the page the glyphs will end up.

\documentclass{article}
\usepackage{amsmath}
\usepackage{unicode-math}
\directlua{

local glyph_id = node.id("glyph")

recurse_sublists = function(head)
    for n in node.traverse(head) do
        if n.id == glyph_id then
            print("Glyph")
            print("  Filename: " .. font.fonts[n.font].filename)
            print("  Glyph index: " .. n.char)
        end
        if n.list then
            recurse_sublists(n.list)
        end
    end
end

local print_slots = function(head, display_type, penalties)
    local newhead = node.mlist_to_hlist(head, display_type, penalties)
    recurse_sublists(newhead)
    return newhead
end

luatexbase.add_to_callback("mlist_to_hlist", print_slots, "print_slots")

}
\begin{document}

\[
  \frac{1}{2\pi i} \int\limits_\gamma f\Bigl(x^{\mathbf{N}\in\mathbb{C}^{N\times 10}}\Bigr)
  = \sum_{k=1}^m n(\gamma;a_k)\operatorname{Res}(f;a_k)\,.
\]

\end{document}

In the log I get

Glyph
 Filename: /usr/local/texlive/2018/texmf-dist/fonts/opentype/public/lm-math/latinmodern-math.otf
 Glyph index: 984713
Glyph
 Filename: /usr/local/texlive/2018/texmf-dist/fonts/opentype/public/lm-math/latinmodern-math.otf
 Glyph index: 984714
Glyph
 Filename: /usr/local/texlive/2018/texmf-dist/fonts/opentype/public/lm-math/latinmodern-math.otf
 Glyph index: 49
Glyph
 Filename: /usr/local/texlive/2018/texmf-dist/fonts/opentype/public/lm-math/latinmodern-math.otf
 Glyph index: 50
Glyph
 Filename: /usr/local/texlive/2018/texmf-dist/fonts/opentype/public/lm-math/latinmodern-math.otf
 Glyph index: 120587
Glyph
 Filename: /usr/local/texlive/2018/texmf-dist/fonts/opentype/public/lm-math/latinmodern-math.otf
 Glyph index: 119894
Glyph
 Filename: /usr/local/texlive/2018/texmf-dist/fonts/opentype/public/lm-math/latinmodern-math.otf
 Glyph index: 984962
Glyph
 Filename: /usr/local/texlive/2018/texmf-dist/fonts/opentype/public/lm-math/latinmodern-math.otf
 Glyph index: 120574
Glyph
 Filename: /usr/local/texlive/2018/texmf-dist/fonts/opentype/public/lm-math/latinmodern-math.otf
 Glyph index: 119891
Glyph
 Filename: /usr/local/texlive/2018/texmf-dist/fonts/opentype/public/lm-math/latinmodern-math.otf
 Glyph index: 984713
Glyph
 Filename: /usr/local/texlive/2018/texmf-dist/fonts/opentype/public/lm-math/latinmodern-math.otf
 Glyph index: 119909
Glyph
 Filename: /usr/local/texlive/2018/texmf-dist/fonts/opentype/public/lm/lmroman7-bold.otf
 Glyph index: 78
Glyph
 Filename: /usr/local/texlive/2018/texmf-dist/fonts/opentype/public/lm-math/latinmodern-math.otf
 Glyph index: 8712
Glyph
 Filename: /usr/local/texlive/2018/texmf-dist/fonts/opentype/public/lm-math/latinmodern-math.otf
 Glyph index: 8450
Glyph
 Filename: /usr/local/texlive/2018/texmf-dist/fonts/opentype/public/lm-math/latinmodern-math.otf
 Glyph index: 119873
Glyph
 Filename: /usr/local/texlive/2018/texmf-dist/fonts/opentype/public/lm-math/latinmodern-math.otf
 Glyph index: 215
Glyph
 Filename: /usr/local/texlive/2018/texmf-dist/fonts/opentype/public/lm-math/latinmodern-math.otf
 Glyph index: 49
Glyph
 Filename: /usr/local/texlive/2018/texmf-dist/fonts/opentype/public/lm-math/latinmodern-math.otf
 Glyph index: 48
Glyph
 Filename: /usr/local/texlive/2018/texmf-dist/fonts/opentype/public/lm-math/latinmodern-math.otf
 Glyph index: 984714
Glyph
 Filename: /usr/local/texlive/2018/texmf-dist/fonts/opentype/public/lm-math/latinmodern-math.otf
 Glyph index: 61
Glyph
 Filename: /usr/local/texlive/2018/texmf-dist/fonts/opentype/public/lm-math/latinmodern-math.otf
 Glyph index: 119898
Glyph
 Filename: /usr/local/texlive/2018/texmf-dist/fonts/opentype/public/lm-math/latinmodern-math.otf
 Glyph index: 984973
Glyph
 Filename: /usr/local/texlive/2018/texmf-dist/fonts/opentype/public/lm-math/latinmodern-math.otf
 Glyph index: 119896
Glyph
 Filename: /usr/local/texlive/2018/texmf-dist/fonts/opentype/public/lm-math/latinmodern-math.otf
 Glyph index: 61
Glyph
 Filename: /usr/local/texlive/2018/texmf-dist/fonts/opentype/public/lm-math/latinmodern-math.otf
 Glyph index: 49
Glyph
 Filename: /usr/local/texlive/2018/texmf-dist/fonts/opentype/public/lm-math/latinmodern-math.otf
 Glyph index: 119899
Glyph
 Filename: /usr/local/texlive/2018/texmf-dist/fonts/opentype/public/lm-math/latinmodern-math.otf
 Glyph index: 40
Glyph
 Filename: /usr/local/texlive/2018/texmf-dist/fonts/opentype/public/lm-math/latinmodern-math.otf
 Glyph index: 120574
Glyph
 Filename: /usr/local/texlive/2018/texmf-dist/fonts/opentype/public/lm-math/latinmodern-math.otf
 Glyph index: 59
Glyph
 Filename: /usr/local/texlive/2018/texmf-dist/fonts/opentype/public/lm-math/latinmodern-math.otf
 Glyph index: 119886
Glyph
 Filename: /usr/local/texlive/2018/texmf-dist/fonts/opentype/public/lm-math/latinmodern-math.otf
 Glyph index: 119896
Glyph
 Filename: /usr/local/texlive/2018/texmf-dist/fonts/opentype/public/lm-math/latinmodern-math.otf
 Glyph index: 41
Glyph
 Filename: /usr/local/texlive/2018/texmf-dist/fonts/opentype/public/lm/lmroman10-regular.otf
 Glyph index: 82
Glyph
 Filename: /usr/local/texlive/2018/texmf-dist/fonts/opentype/public/lm/lmroman10-regular.otf
 Glyph index: 101
Glyph
 Filename: /usr/local/texlive/2018/texmf-dist/fonts/opentype/public/lm/lmroman10-regular.otf
 Glyph index: 115
Glyph
 Filename: /usr/local/texlive/2018/texmf-dist/fonts/opentype/public/lm-math/latinmodern-math.otf
 Glyph index: 40
Glyph
 Filename: /usr/local/texlive/2018/texmf-dist/fonts/opentype/public/lm-math/latinmodern-math.otf
 Glyph index: 119891
Glyph
 Filename: /usr/local/texlive/2018/texmf-dist/fonts/opentype/public/lm-math/latinmodern-math.otf
 Glyph index: 59
Glyph
 Filename: /usr/local/texlive/2018/texmf-dist/fonts/opentype/public/lm-math/latinmodern-math.otf
 Glyph index: 119886
Glyph
 Filename: /usr/local/texlive/2018/texmf-dist/fonts/opentype/public/lm-math/latinmodern-math.otf
 Glyph index: 119896
Glyph
 Filename: /usr/local/texlive/2018/texmf-dist/fonts/opentype/public/lm-math/latinmodern-math.otf
 Glyph index: 41
Glyph
 Filename: /usr/local/texlive/2018/texmf-dist/fonts/opentype/public/lm-math/latinmodern-math.otf
 Glyph index: 46

Even after your comments I am not really sure, which information you actually need. That is why I have implemented some sort of JSON serialization, which simply writes the complete mlist to a file.

\documentclass{article}
\usepackage{amsmath}
\usepackage{unicode-math}
\directlua{

local glyph_id = node.id("glyph")

f = io.open("\jobname.mlist", "w")

recurse_sublists = function(head, indent)
    indent = indent or 0
    local s = string.rep(" ", indent)
    f:write("{\string\n")
    for n in node.traverse(head) do
        for _,v in ipairs(node.fields(n.id)) do
        f:write(s .. '"' .. v .. '": ')
        if type(n[v]) == "number" then
            f:write(n[v])
        elseif type(n[v]) == "nil" then
            f:write('null')
        else
            f:write('"' .. tostring(n[v]) .. '"')
        end
        f:write(',\string\n')
        end
        if n.list then
            f:write(s .. '"list": ')
            recurse_sublists(n.list, indent + 2)
        end
    end
    f:write(string.rep(" ", indent - 2) .. "},\string\n")
end

local print_slots = function(head, display_type, penalties)
    local newhead = node.mlist_to_hlist(head, display_type, penalties)
    recurse_sublists(newhead, 2)
    return newhead
end

luatexbase.add_to_callback("mlist_to_hlist", print_slots, "print_slots")

}

\AtEndDocument{\directlua{f:close()}}
\begin{document}

\[
  \frac{1}{2\pi i} \int\limits_\gamma f\Bigl(x^{\mathbf{N}\in\mathbb{C}^{N\times 10}}\Bigr)
  = \sum_{k=1}^m n(\gamma;a_k)\operatorname{Res}(f;a_k)\,.
\]

\end{document}

Thanks. Note that per the use case above, what we care about is not really the absolute position of the glyph in the page, but its position relative to the baseline and the beginning of the line; not sure if this makes things simpler. — Antony Lee, Dec 15 '18 at 15:35
Sorry, I completely missed the fact that you edited your previous response; thanks. Actually, the list of boxes and face index is kind of a distraction; by far the most important would be to get the glyph positions (whether absolute on the page, or relative to some arbitrary (known)origin), which, according to your comment above, is unfortunately not available yet when mlist_to_hlist is called :(. — Antony Lee, Dec 29 '18 at 22:13

asked	6 months ago
viewed	112 times
active	6 months ago

Stack Exchange Network

current community

your communities

more stack exchange communities

Let luatex output a list of glyphs and their positions

1 Answer 1

Your Answer

Not the answer you're looking for? Browse other questions tagged luatex or ask your own question.

Linked

Hot Network Questions

Let luatex output a list of glyphs and their positions

1 Answer 1

Your Answer

Sign up or log in

Post as a guest

Not the answer you're looking for? Browse other questions tagged luatex or ask your own question.

Linked

Related

Hot Network Questions