Why user-defined whatsit kicks out kerning?

Question

I ran in to trouble marking math content (in this example I'll be talking about simple superscript) with user-defined whatsit. Font kerning does not appear if whatsit is in between characters where normally kern would be placed.

Marking superscript is done in mlist_to_hlist callback and later (in pre_output_filter callback) I'm processing node list and I need to know where superscript begins/ends to do something with it.

luatex documentation mentions not much about user-defined whatsis but it seems they should be virtual and should not influence the output. I tried to fix kerning by creating it with node.kerning() and adding to the right place but then I need some how to recalculate all the outer boxes (hlist's, vlist's). Thought using kerning callback I could do something but it seems it has now access to display math converted nodes. I see only hlist node but it has no head.

Question: is there a safe way to add user-defined whatsit in math without influencing the output. Or is there another way to mark some positions in math nodes and get them in pre_output_filter.

test.tex

\documentclass{article}
\usepackage[T1]{fontenc}
\usepackage{luatexbase}
\directlua{dofile("mathsup.lua")}

\begin{document}
\loggingall
$$ xdf$$

$$ xdf^2 $$
\end{document}

mathsup.lua

local USER_MARK   = node.new("whatsit","user_defined")
USER_MARK.type    = 115   
USER_MARK.value   = "0"   

local function tag_superscript (h, d, p)
    for n in node.traverse(h) do
        if n.sup then
            -- tagging superscript Begin and End
            local A = node.copy(USER_MARK)
            A.value = "pre_superscript"
            h = node.insert_before(h, n, A)

            local B = node.copy(USER_MARK)
            B.value = "post_superscript"
            node.insert_after(h, n, B)    
        end
    end
    return node.mlist_to_hlist(h, d, p)
end
luatexbase.add_to_callback('mlist_to_hlist', tag_superscript, 'Tag superscript')

In output of the first display there is \kern-1.66667:

...\hbox(6.94444+1.94444)x15.22572, shifted 164.88715, direction TLT
....\OML/cmm/m/it/10 x
....\OML/cmm/m/it/10 d
....\kern-1.66667 (font)
....\OML/cmm/m/it/10 f
....\kern1.0764 (italic)

In output of the second display kern is missing:

...\hbox(8.64003+1.94444)x21.37852, shifted 161.81075, direction TLT
....\OML/cmm/m/it/10 x
....\OML/cmm/m/it/10 d
....\whatsit0="pre_superscript"
....\OML/cmm/m/it/10 f
....\kern1.0764 (italic)
....\hbox(4.51111+0.0)x4.48613, shifted -4.12892, direction TLT
.....\OT1/cmr/m/n/7 2
....\whatsit0="post_superscript"

Answer 1

You can use node attributes to mark some nodes:

local sup_attr = luatexbase.new_attribute("sup_attr")

local glyph_id = node.id "glyph"
local hlist_id = node.id "hlist"
local vlist_id = node.id "vlist"

local function tag_superscript (h, d, p)
    for n in node.traverse(h) do
        if n.sup then
          node.set_attribute(n.sup, sup_attr, 1)
        end
    end
    return node.mlist_to_hlist(h, d, p)
end

local uchar = unicode.utf8.char
local function traverse_hlist(head, groupcode)
  for n in node.traverse(head) do
    local id = n.id
    if id == glyph_id  and node.has_attribute(n, sup_attr) then
      print("here is a subscript", uchar(n.char), node.get_attribute(n, sup_attr))
      -- do your stuff here
      node.unset_attribute(n, sup_attr)
    elseif id == hlist_id or id == vlist_id then
      traverse_hlist(n.head)
    end
  end
  return head
end


luatexbase.add_to_callback('mlist_to_hlist', tag_superscript, 'Tag superscript')
luatexbase.add_to_callback("pre_linebreak_filter", traverse_hlist, "Find superscript")
luatexbase.add_to_callback("hpack_filter", traverse_hlist, "Find superscript")
luatexbase.add_to_callback("vpack_filter", traverse_hlist, "Find superscript")

Custom attribute is defined using luatexbase.new_attribute("sup_attr"), it is set in math nodes which has the sup field. The attribute is set directly to the node in sup field.

In the node traversing callback, we are looking for glyphs which have set our attribute. You can do your stuff at this place. The attribute is then removed, in order to prevent multiple executions of the node processing.

Note that your particular example is processed by the vpack_filter, because pre_linebreak_filter doesn't process display math.