Discovering which (La)Tex element is shown by lua-visual-debug (print to stdout)?

Question

I am trying to troubleshoot what happens in the code in Wrong placement (and/or size) of picture / plot with groupplot of table (... and discontinuity)?. So I added \usepackage{lua-visual-debug} to that example, and built it with lualatex test.tex; and this is what comes out:

As far as I can tell (by consulting lvdebug-doc.pdf documentation of lua-visual-debug), those gray horizontal lines "pushing away" the plot from the left are glues. The problem is - I have no idea how to tell which glues those are; even worse, at end of run, lualatex notes:

...
9093 words of node memory still in use:
123 hlist, 1 vlist, 1 rule, 32 math, 638 glue, 33 glyph, 69 glue_spec, 1 wri
te, 1561 pdf_literal, 18 pdf_colorstack nodes
avail lists: 1:1,2:44,3:2341,4:710,6:41,7:1,9:130
...

... so even if I had a list of glue names of sorts, it would be difficult to trawl manually through 638 glues.

Ideally, what I'd want is to typeset the name of the LaTex glue, right by its rendered lua-visual-debug line; e.g. I can see that the code responsible for it in lua-visual-debug.lua is:

  elseif head.id == 10 then -- glue
      print "Got glue" -- added for debug
      local wd = head.spec.width
      local color = "0.5 G"
      if parent.glue_sign == 1 and parent.glue_order == head.spec.stretch_order then
        wd = wd + parent.glue_set * head.spec.stretch
        color = "0 0 1 RG"
      elseif parent.glue_sign == 2 and parent.glue_order == head.spec.shrink_order then
        wd = wd - parent.glue_set * head.spec.shrink
        color = "1 0 1 RG"
      end
      local pdfstring = node.new("whatsit","pdf_literal")
      local wd_bp = math.round(wd / number_sp_in_a_pdf_point,2)
      if parent.id == 0 then --hlist
        pdfstring.data = string.format("q %s [0.2] 0 d  0.5 w 0 0  m %g 0 l s Q",color,wd_bp)
      else -- vlist
        pdfstring.data = string.format("q 0.1 G 0.1 w -0.5 0 m 0.5 0 l -0.5 %g m 0.5 %g l s [0.2] 0 d  0.5 w 0.25 0  m 0.25 %g l s Q",-wd_bp,-wd_bp,-wd_bp)
      end
      parent.list = node.insert_before(parent.list,head,pdfstring)
      -- print("Got glue; wd " .. tostring(wd))

So, basically, if I'd want to typeset those names, I'd have to insert PDF primitives; I tried adding (Hello world) Tj to the pdfstring.data, but nothing comes out.

Considering typesetting the 638 names will be a bit crowded, I'd also settle for typesetting a numeric index in the PDF, and then printing to stdout a list with the index, the LaTex name of command/length/glue, and possibly its width/length value; that should allow somewhat easier inspection.

What I really cannot tell is this - so these are my questions:

• At this point in the lua-visual-debug code, are originating Tex command names even available? (I can see LuaTex reference has token.command_name, but at this point I have "nodes" [what nodes?], not "tokens")
• If I enable the last print there, I can print the widths; it turns out, most of those are 0, however, some are like "Got glue; wd 27571199" -- is this 27571199 in bp units? How could I print it out, say, in centimeters or in inches?
• Is it even possible to insert text PDF primitives at this point in the lua code?

Answer 1

Ok, I think I got somewhere - below the glues are labeled with "GL" + a number (which I don't know what it means, but is shown in Lua for any Lua node via tostring(node) - also the glue one - and seems to be unique), and it looks like this:

The problem is then relating this number with actual elements in the Latex source; the luatexref-t.pdf documentation doesn't mention what it might mean, but it mentions some properties of (glue?) nodes, which can be used to possibly get better info.

Very quickly, I'm short on time for writing properly - first, refs:

• How to generate plain-text source-code PDF examples that work in a document viewer? - Stack Overflow
• Imagemagick: generate raw image data for PDF flate embedding? - Stack Overflow
• Lua beginner: Table dump to console - Stack Overflow
• How to inspect userdata in lua - Stack Overflow
• How can I visualize boxes? - TeX - LaTeX Stack Exchange

Here an example working PDF stream:

/F1 24 Tf            % font select; must have! /F1 must be correct ref.; size (24) can be arbitrary 
0 1 0 rg 100 100 Td  % position text - text reacts on fill (rg) only!
( Hello World ) Tj   % output text
%q  1.5 w  0 0 m  30 30 l  S  Q % example line q/Q push/restore graph. stack, width, moveto, lineto
0 1 0 RG  1.5 w  0 0 m  30 30 l  S % example colored line - reacts on stroke (RG) only!

First, place inspect.lua (https://github.com/kikito/inspect.lua) in /path/to/texlive/2011/texmf-dist/tex/luatex/lua-visual-debug/ directory; then hack lua-visual-debug.lua like so:

...
module(...,package.seeall)

-- https://stackoverflow.com/questions/9168058/lua-table-dump-to-console
-- https://github.com/kikito/inspect.lua/blob/master/inspect.lua
-- this is enough
local inspectfile = assert(loadfile("/path/to/texlive/2011/texmf-dist/tex/luatex/lua-visual-debug/inspect.lua"))
local inspect = inspectfile()
print("inspect is" .. inspect(inspect))
...
function show_page_elements(parent)
...
  elseif head.id == 10 then -- glue
      local wd = head.spec.width
      ...
      local pdfstring = node.new("whatsit","pdf_literal")
      local wd_bp = math.round(wd / number_sp_in_a_pdf_point,2)
      if parent.id == 0 then --hlist
        pdfstring.data = string.format("q %s [0.2] 0 d  0.5 w 0 0  m %g 0 l s Q",color,wd_bp)
      else -- vlist
        pdfstring.data = string.format("q 0.1 G 0.1 w -0.5 0 m 0.5 0 l -0.5 %g m 0.5 %g l s [0.2] 0 d  0.5 w 0.25 0  m 0.25 %g l s Q",-wd_bp,-wd_bp,-wd_bp)
      end
      -- q: gsave; Q grestore; Tf selectfont; Td Move text position; Tj show text
--~       pdfstring.data = pdfstring.data .. "  q  0 1 0 RG  0 0 m 40 40 l  Q" -- this shows, but not color; '0 0 m' is relative!
      -- from a latex: /F8 15.0 Tf 0 0 Td (abcd)Tj should works; wrong size is rendered, wrong /F8 not - and even with all correct, if a character in string is not in the exported font, it will NOT work!
      -- so, in latex source file, make sure the used letters are output once! e.g.
      -- \node[text=white,anchor=west] at (0,0) {glueGLUE0123456789.};
      headstr = tostring(head)
      headmidnum = string.match(headstr, '.*<%s*(%d+)%s*>')
      pdfstring.data = pdfstring.data .. string.format("  q  /F1 24 Tf  0 0 Td  (GL%s) Tj  Q", tostring(headmidnum))
      parent.list = node.insert_before(parent.list,head,pdfstring)
      -- tostring(head.length):: head.length: Invalid field id length for node type glue (0)
      print("Got glue; wd " .. tostring(wd) .. "; " ..  "; " .. tostring(head))
      print("hd: " .. head.id .. ", " .. head.subtype .. ", " .. tostring(head.attr) .. ", ''" .. tostring(head.spec) .. "'', " .. tostring(head.leader) )
      -- no head.name: invalid field id name for node type glue (0)
      if (head.spec ~= nil) then
        print("hd spec: " .. head.spec.width .. ", " .. head.spec.stretch .. ", " .. head.spec.stretch_order .. ", " .. head.spec.shrink .. ", " .. head.spec.shrink_order .. ", " .. tostring(head.spec.writable))
      end
      if (head.attr ~= nil) then -- always nil
        print("attr: " .. node.fields(head.attr.id, head.attr.subtype ))
      end
      -- extract (apparently) "sequence index": the middle number in headstr='<node 126920 < 107034 > 107021'
      if (headmidnum ~= nil) then
        print("mid: " .. headmidnum)
      else print ("NO headmidnum!") end
      --print(inspect(head.tostring)) -- no __tostring() -- both invalid; no .number or .next.number
      -- print(inspect(head) .. " -- " .. inspect(getmetatable(head)) .. "--" .. inspect(node.fields(head.id, head.subtype))) -- <userdata 1>
      --~ Got glue; wd 0; ; <node 126920 < 107034 > 107021 : glue 2>
      --~ <userdata 1> -- {
      --~   __eq = <function 1>,
      --~   __index = <function 2>,
      --~   __newindex = <function 3>,
      --~   __tostring = <function 4>
      --~ }--{ "id", "subtype", "attr", "spec", "leader",
      --~   [-1] = "prev",
      --~   [0] = "next"

-- PDF font declarations - these seem to change always :(
--~ /Font
--~ <<
--~ /F47 5 0 R
--~ /F20 6 0 R
--~ /F46 7 0 R
--~ /F16 8 0 R
--~ /F45 9 0 R
--~ /F44 10 0 R
--~ /F39 11 0 R
--~ /F38 12 0 R
--~ /F35 13 0 R
--~ /F34 14 0 R
--~ /F33 15 0 R
--~ >>
...

With this done, in the terminal/console (here used bash), do:

# note that print() from lua goes to stdout - but not to Latex log!
# so capture entire stdout to another log with redirection: 
lualatex test.tex 2>&1 | tee test-lua.log

# must unpack the produced test.pdf file:
pdftk test.pdf output test-unc.pdf uncompress


# if want to make sure our literal is inserted:
# must use less -L to show entire contents:
less -L test-unc.pdf # or
grep '/F1 ' test-unc.pdf | head -2

# find one actual font reference - grep for 'GLUEgl'
# (note that regardless of \mbox, the text GLUEglue... may 
#  end up split in the .pdf!

grep 'glueGL' test-unc.pdf
# example output:
# /F15 9.96264 Tf 1 0 0 1 274.444 12.896 Tm [(glueGLU)]TJ

# now replace the `/F1 24` we had previously with a 
# proper reference (/F15) and smaller font size (5):
sed -i 's_/F1 24_/F15 5_g' test-unc.pdf

The test-unc.pdf can now be opened (say in evince), and should show labeled glues as in the image above. Note that, even if the PDF viewer might not complain with warning messages, the PDF would still be corrupt because after the changes, the PDF xref trailer cannot be correct. You might want to rerun the file through pdftk again to try correct that - but even with that, I couldn't use ImageMagick convert to render the test-unc.pdf (I could render it in GIMP though, which is how the image above was obtained).