# Why user-defined whatsit kicks out kerning?

I ran in to trouble marking math content (in this example I'll be talking about simple superscript) with user-defined whatsit. Font kerning does not appear if whatsit is in between characters where normally kern would be placed.

Marking superscript is done in mlist_to_hlist callback and later (in pre_output_filter callback) I'm processing node list and I need to know where superscript begins/ends to do something with it.

luatex documentation mentions not much about user-defined whatsis but it seems they should be virtual and should not influence the output. I tried to fix kerning by creating it with node.kerning() and adding to the right place but then I need some how to recalculate all the outer boxes (hlist's, vlist's). Thought using kerning callback I could do something but it seems it has now access to display math converted nodes. I see only hlist node but it has no head.

Question: is there a safe way to add user-defined whatsit in math without influencing the output. Or is there another way to mark some positions in math nodes and get them in pre_output_filter.

test.tex

\documentclass{article}
\usepackage[T1]{fontenc}
\usepackage{luatexbase}
\directlua{dofile("mathsup.lua")}

\begin{document}
\loggingall
$$xdf$$

$$xdf^2$$
\end{document}


mathsup.lua

local USER_MARK   = node.new("whatsit","user_defined")
USER_MARK.type    = 115
USER_MARK.value   = "0"

local function tag_superscript (h, d, p)
for n in node.traverse(h) do
if n.sup then
-- tagging superscript Begin and End
local A = node.copy(USER_MARK)
A.value = "pre_superscript"
h = node.insert_before(h, n, A)

local B = node.copy(USER_MARK)
B.value = "post_superscript"
node.insert_after(h, n, B)
end
end
return node.mlist_to_hlist(h, d, p)
end


In output of the first display there is \kern-1.66667:

...\hbox(6.94444+1.94444)x15.22572, shifted 164.88715, direction TLT
....\OML/cmm/m/it/10 x
....\OML/cmm/m/it/10 d
....\kern-1.66667 (font)
....\OML/cmm/m/it/10 f
....\kern1.0764 (italic)


In output of the second display kern is missing:

...\hbox(8.64003+1.94444)x21.37852, shifted 161.81075, direction TLT
....\OML/cmm/m/it/10 x
....\OML/cmm/m/it/10 d
....\whatsit0="pre_superscript"
....\OML/cmm/m/it/10 f
....\kern1.0764 (italic)
....\hbox(4.51111+0.0)x4.48613, shifted -4.12892, direction TLT
.....\OT1/cmr/m/n/7 2
....\whatsit0="post_superscript"

• unrelated but you shouldn't normally do \usepackage[T1]{fontenc} with luatex and only need \usepackage{luatexbase} if you need compatibility with early lualatex – David Carlisle Jul 18 '18 at 15:39
• What are you trying to do in the end? (I just looked at this callback and so I'm curious). – Ulrike Fischer Jul 18 '18 at 17:02
• Also, you shouldn't use  for maths in LaTeX (deprecated since sometime in the 1980s, I think). – cfr Jul 19 '18 at 3:38
• That is just an example. I'm trying to tag a mathml. \usepackage[T1]{fontenc} I use to suppress OTF fonts (because I use DVI viewer). – Linuxss Jul 19 '18 at 5:38

You can use node attributes to mark some nodes:

local sup_attr = luatexbase.new_attribute("sup_attr")

local glyph_id = node.id "glyph"
local hlist_id = node.id "hlist"
local vlist_id = node.id "vlist"

local function tag_superscript (h, d, p)
for n in node.traverse(h) do
if n.sup then
node.set_attribute(n.sup, sup_attr, 1)
end
end
return node.mlist_to_hlist(h, d, p)
end

local uchar = unicode.utf8.char
local id = n.id
if id == glyph_id  and node.has_attribute(n, sup_attr) then
print("here is a subscript", uchar(n.char), node.get_attribute(n, sup_attr))
node.unset_attribute(n, sup_attr)
elseif id == hlist_id or id == vlist_id then
end
end
end


Custom attribute is defined using luatexbase.new_attribute("sup_attr"), it is set in math nodes which has the sup field. The attribute is set directly to the node in sup field.
Note that your particular example is processed by the vpack_filter, because pre_linebreak_filter doesn't process display math.