# luatex hlist node shift direction seems incorrect, what is the cleanest way to achieve pdf object translations (displacement)?

From the description of hlist nodes in luatex manual, its field 'shift' seems to be a vertical shift. But when I change it using post_linebreak_filter, it adds horizontal shift to hlist. This overleaf post too describes shift as a vertical shift for hlist: "Another parameter listed in the “metadata” is shift: this is the value of box displacement resulting from applying TeX commands: \raise, \lower (applied to an \hbox);" Have I encountered a bug in luatex, or is my usage pattern incorrect? Either way, I would like select a hlist node, and shift it horizontally and vertically on the page while not affecting other pdf contents on the page (in effect apply pdf translation matrix to the hlist). Unfortunately, pdf_setmatrix node cannot be used for vertical/horizontal displacement (called translation in PDF 1.7 manual) as pdftex has disabled it (read the definition of \pdfsetmatrix in pdftex manual), and it warns against using \pdfliteral (pdftex equivalent of luatex pdf_literal node). Though am not sure whether the warning is true for luatex during post linebreak phase? If it is, then what's the safest way to achieve horizontal and vertical translation? (My guess for horizontal translation is to add a kern node at the beginning of hlist, is that the cleanest? And am not sure about the cleanest way to do vertical translation...)

Here's a screenshot of output I get, followed by complete code to add vertical and horizontal translations to hlist of line number 3, and below the code are some useful screenshot from documents mentioned in the description above:

code: >>lualatex test.tex

% test.tex
\documentclass[notitlepage,letterpaper]{article}
\usepackage[letterpaper,left=2in,right=2in,top=1in,bottom=1in]{geometry}
\usepackage{blindtext}

\directlua{
local linenumber=1;
local HLIST = node.id("hlist")
local WHATSIT = node.id("whatsit")
local KERN = node.id("kern")
for n in node.traverse(head) do
if n.id==HLIST then
if linenumber==3 then
% Add vertical translation
n.shift=20*65536; % This should shift linenumber 3 vertically by 20, in reality it shifts horizontally
% Add horizontal translation
local hkern = node.new(KERN)
hkern.subtype=1;
hkern.attr=n.attr;
hkern.kern = 80*65535;
end
linenumber=linenumber+1;
end
end
end
luatexbase.add_to_callback('post_linebreak_filter', my_post_lb_filter, 'Play with luatex node library')
}

\begin{document}
\blindtext[1]
\end{document}



hlist shift description from luatex manual:

pdf object translation:

pdfliteral vs pdfsetmatrix from pdftex manual:

Regarding shift: The documentation is misleading, but the box is shifted vertically if it is contained in a horizontal list (\lower or \raise in horizontal mode), it is shifted horizontally if it is contained in a vertical list (\moveleft and \moveright in vertical mode) and it is not shifted at all if it is not in any outer list.

So one "fix" would be to wrap it in an outer hbox:

% test.tex
\documentclass[notitlepage,letterpaper]{article}
\usepackage[letterpaper,left=2in,right=2in,top=1in,bottom=1in]{geometry}
\usepackage{blindtext}

\directlua{
local linenumber=1;
local HLIST = node.id("hlist")
local WHATSIT = node.id("whatsit")
local KERN = node.id("kern")
for n in node.traverse(head) do
if n.id==HLIST then
if linenumber==3 then
% Add vertical translation
n.shift=20*65536; % This should shift linenumber 3 vertically by 20, in reality it shifts horizontally
% Add horizontal translation
local hkern = node.new(KERN)
hkern.subtype=1;
hkern.attr=n.attr;
hkern.kern = 80*65535;
% We want to wrap n in a hlist, but we have to keep
% node.traverse happy so we can't easily change the
% current node. Instead, create a new hlist inside of the
% current list, inheriting all attributes and the content
% of n. Then this list becomes the new head of n.
%
% First a little optimization: We want to create a copy of n with
% local nn = node.copy(n)
% but doing so now would lead to n.head being deep-copied
% into nn.head. Given that we want to overwrite nn.head
% anyway, this would waste memory and time, especially if
% the line is very complicated. So save the line
% make it empty to hide it and avoid copying something
% and then make the copy. Now nothing in n..head is copied
% (because nothing is there)
local nn = node.copy(n)
% Finally we again want nn to  contain the original content contained by n, so assign the saved content.
% So much for the inner hlist nn. But what should be the new
% content of the outer hlist n? it should contain only the
% inner hlist nn. Given that nn was just created, it doesn't
% has a next node anyway, so we can just assign it as new head.
n.shift = 0
end
linenumber=linenumber+1;
end
end
end
luatexbase.add_to_callback('post_linebreak_filter', my_post_lb_filter, 'Play with luatex node library')
}

\begin{document}
\blindtext[1]
\end{document}


(This version keeps the dimensions to simulate the PDF operator variant, leading to overlap)

Of course, a IMO nicer alternative is to just use normal kern nodes as you did for horizontal positioning:

\documentclass[notitlepage,letterpaper]{article}
\usepackage[letterpaper,left=2in,right=2in,top=1in,bottom=1in]{geometry}
\usepackage{blindtext}

\directlua{
local linenumber=1;
local HLIST = node.id("hlist")
local WHATSIT = node.id("whatsit")
local KERN = node.id("kern")
for n in node.traverse(head) do
if n.id==HLIST then
if linenumber==3 then
% Add vertical translation
local vkern = node.new(KERN)
vkern.kern = 20*65536
if 'you want the text to overlap' then
vkern = node.copy(vkern)
vkern.kern = -vkern.kern
end
% Add horizontal translation
local hkern = node.new(KERN)
hkern.subtype=1;
hkern.attr=n.attr;
hkern.kern = 80*65535;
end
linenumber=linenumber+1;
end
end
end
luatexbase.add_to_callback('post_linebreak_filter', my_post_lb_filter, 'Play with luatex node library')
}

\begin{document}
\blindtext[1]
\end{document}


(If you do not want overlapping text, change 'you want the text to overlap' to false or remove the block.

• Thanks, I confirm it works. It would be great if you could add some comments explaining each line of following code: local saved_head = n.head; n.head = nil; local nn = node.copy(n); nn.head = saved_head; n.head = nn;. Am new to both lua, and linked lists so this piece of code is little hard to visualize. Some questions that come to mind are: why did you have to save the head of linked list, and the assign nil to it before doing a deep copy, and then assign saved head to nn.head? Code seems to work even if I do not assign nil to n.head before deep copy. What does n.head = nn do? – reportaman May 14 '20 at 3:59
• And to add to that, I reduced those lines of code (highlighted in the comment above) to just these two lines: local nn = node.copy(n); n.head = nn;, and the output is still the same. So it will be even more helpful to compare/contrast both. Like when would one need to do that n.head = nil; before deep copy, and assign nn.head to previously saved head local saved_head = n.head; ..... nn.head = saved_head;. Does local nn = node.copy(n) imply nn.head = n.head when it gets executed? – reportaman May 14 '20 at 4:17
• @reportaman I added comments inline, but basically: Not doing this would (as you said) do a deep-copy. This technically works, but in the process it will create lots of unused nodes, wasting memory and time. These extra nodes will serve no further purpose and will be either freed directly or even leak and waste memory for the entire remaining run, so it is better to just avoid creating them in the first place. (If you would not do this and just execute local nn = node.copy(n); n.head = nn;, you should at least add node.flush_list(n.head) before n.head = nn to free them and avoid leaks) – Marcel Krüger May 14 '20 at 9:03