luatex hlist node shift direction seems incorrect, what is the cleanest way to achieve pdf object translations (displacement)?

Question

From the description of hlist nodes in luatex manual, its field 'shift' seems to be a vertical shift. But when I change it using post_linebreak_filter, it adds horizontal shift to hlist. This overleaf post too describes shift as a vertical shift for hlist: "Another parameter listed in the “metadata” is shift: this is the value of box displacement resulting from applying TeX commands: \raise, \lower (applied to an \hbox);" Have I encountered a bug in luatex, or is my usage pattern incorrect? Either way, I would like select a hlist node, and shift it horizontally and vertically on the page while not affecting other pdf contents on the page (in effect apply pdf translation matrix to the hlist). Unfortunately, pdf_setmatrix node cannot be used for vertical/horizontal displacement (called translation in PDF 1.7 manual) as pdftex has disabled it (read the definition of \pdfsetmatrix in pdftex manual), and it warns against using \pdfliteral (pdftex equivalent of luatex pdf_literal node). Though am not sure whether the warning is true for luatex during post linebreak phase? If it is, then what's the safest way to achieve horizontal and vertical translation? (My guess for horizontal translation is to add a kern node at the beginning of hlist, is that the cleanest? And am not sure about the cleanest way to do vertical translation...)

Here's a screenshot of output I get, followed by complete code to add vertical and horizontal translations to hlist of line number 3, and below the code are some useful screenshot from documents mentioned in the description above:

code: >>lualatex test.tex

% test.tex
\documentclass[notitlepage,letterpaper]{article}
\usepackage[letterpaper,left=2in,right=2in,top=1in,bottom=1in]{geometry}
\usepackage{blindtext}

\directlua{
    function my_post_lb_filter(head,groupcode)
      local linenumber=1;
      local HLIST = node.id("hlist")
      local WHATSIT = node.id("whatsit")
      local KERN = node.id("kern")
      for n in node.traverse(head) do
        if n.id==HLIST then
          if linenumber==3 then
            % Add vertical translation
            n.shift=20*65536; % This should shift linenumber 3 vertically by 20, in reality it shifts horizontally
            % Add horizontal translation
            local hkern = node.new(KERN)
            hkern.subtype=1;
            hkern.attr=n.attr;
            hkern.kern = 80*65535;
            n.head = node.insert_before(n.list,n.head,hkern)
          end
        linenumber=linenumber+1;
        end
      end
      return head
    end
  luatexbase.add_to_callback('post_linebreak_filter', my_post_lb_filter, 'Play with luatex node library')
}

\begin{document}
  \blindtext[1]
\end{document}

hlist shift description from luatex manual:

pdf object translation:

pdfliteral vs pdfsetmatrix from pdftex manual:

Answer 1

Regarding shift: The documentation is misleading, but the box is shifted vertically if it is contained in a horizontal list (\lower or \raise in horizontal mode), it is shifted horizontally if it is contained in a vertical list (\moveleft and \moveright in vertical mode) and it is not shifted at all if it is not in any outer list.

So one "fix" would be to wrap it in an outer hbox:

% test.tex
\documentclass[notitlepage,letterpaper]{article}
\usepackage[letterpaper,left=2in,right=2in,top=1in,bottom=1in]{geometry}
\usepackage{blindtext}

\directlua{
    function my_post_lb_filter(head,groupcode)
      local linenumber=1;
      local HLIST = node.id("hlist")
      local WHATSIT = node.id("whatsit")
      local KERN = node.id("kern")
      for n in node.traverse(head) do
        if n.id==HLIST then
          if linenumber==3 then
            % Add vertical translation
            n.shift=20*65536; % This should shift linenumber 3 vertically by 20, in reality it shifts horizontally
            % Add horizontal translation
            local hkern = node.new(KERN)
            hkern.subtype=1;
            hkern.attr=n.attr;
            hkern.kern = 80*65535;
            n.head = node.insert_before(n.list,n.head,hkern)
            % We want to wrap n in a hlist, but we have to keep
            % node.traverse happy so we can't easily change the
            % current node. Instead, create a new hlist inside of the
            % current list, inheriting all attributes and the content
            % of n. Then this list becomes the new head of n.
            %
            % First a little optimization: We want to create a copy of n with
            % local nn = node.copy(n)
            % but doing so now would lead to n.head being deep-copied
            % into nn.head. Given that we want to overwrite nn.head
            % anyway, this would waste memory and time, especially if
            % the line is very complicated. So save the line
            local saved_head = n.head
            % make it empty to hide it and avoid copying something
            n.head = nil
            % and then make the copy. Now nothing in n..head is copied
            % (because nothing is there)
            local nn = node.copy(n)
            % Finally we again want nn to  contain the original content contained by n, so assign the saved content. 
            nn.head = saved_head
            % So much for the inner hlist nn. But what should be the new
            % content of the outer hlist n? it should contain only the
            % inner hlist nn. Given that nn was just created, it doesn't
            % has a next node anyway, so we can just assign it as new head.
            n.head = nn
            n.shift = 0
          end
        linenumber=linenumber+1;
        end
      end
      return head
    end
  luatexbase.add_to_callback('post_linebreak_filter', my_post_lb_filter, 'Play with luatex node library')
}

\begin{document}
  \blindtext[1]
\end{document}

(This version keeps the dimensions to simulate the PDF operator variant, leading to overlap)

Of course, a IMO nicer alternative is to just use normal kern nodes as you did for horizontal positioning:

\documentclass[notitlepage,letterpaper]{article}
\usepackage[letterpaper,left=2in,right=2in,top=1in,bottom=1in]{geometry}
\usepackage{blindtext}

\directlua{
    function my_post_lb_filter(head,groupcode)
      local linenumber=1;
      local HLIST = node.id("hlist")
      local WHATSIT = node.id("whatsit")
      local KERN = node.id("kern")
      for n in node.traverse(head) do
        if n.id==HLIST then
          if linenumber==3 then
            % Add vertical translation
            local vkern = node.new(KERN)
            vkern.kern = 20*65536
            head = node.insert_before(head, n, vkern)
            if 'you want the text to overlap' then
              vkern = node.copy(vkern)
              vkern.kern = -vkern.kern
              node.insert_after(head, n, vkern)
            end
            % Add horizontal translation
            local hkern = node.new(KERN)
            hkern.subtype=1;
            hkern.attr=n.attr;
            hkern.kern = 80*65535;
            n.head = node.insert_before(n.list,n.head,hkern)
          end
          linenumber=linenumber+1;
        end
      end
      return head
    end
  luatexbase.add_to_callback('post_linebreak_filter', my_post_lb_filter, 'Play with luatex node library')
}

\begin{document}
  \blindtext[1]
\end{document}

(If you do not want overlapping text, change 'you want the text to overlap' to false or remove the block.