LuaTeX callback for single line captions

Question

I am using LuaTeX and its associated callbacks (pre_linebreak_filter, post_linebreak_filter and mlist_to_hlist) to analyze lines and paragraphs during typesetting of a pdf document. This works fine but I encountered a problem with figure captions.

For a longer caption that is broken into multiple lines, everything works as expected. pre_linebreak_filter reports the full text, while post_linebreak_filter reports a vlist with the individual lines.

If the caption is short enough to fit into a single line (i.e. no line breaking is needed) the processing of the caption does not generate any callbacks. This leads to my questions:

1. Is this behavior intended, which would be in contrast to other single-line content, such as title or author lines, which DO generate these callbacks?
2. How am I able to get to know that a single-line caption is processed?

PS: Note that the caption content shows up in the output assembly callback of pre_output_filter. However, paragraph affiliation information is already lost at this stage. Thus, this does not help me.

PPS: An example would be just to add

\begin{figure}
  \centering
  \includegraphics{ FILENAME }
  \caption{ TEXT }
\end{figure}

in an ordinary article with TEXT fitting either in a single line or not. The callbacks are linked via the following code in the TeX source

\directlua{ luatexbase.add_to_callback("pre_linebreak_filter", pre_callback, "pre_callback") }
\directlua{ luatexbase.add_to_callback("post_linebreak_filter", post_callback, "post_callback") }

In a separate Lua file, the following functions process the callbacks

pre_callback = function (head)
  texio.write_nl("PRE CALLED")
  -- Recursive inspection of head
end

post_callback = function (head)
  texio.write_nl("POST CALLED")
  -- Recursive inspection of head
end

For both short and long caption texts, both pre_linebreak_filter and post_linebreak_filter report the image itself (in form of whatsit nodes with subtypes 41, 40, 42, 14) and output "CALLED" in the log file. But ONLY for the long caption, the caption text itself is reported in separate callbacks (thus "PRE CALLED" and "POST CALLED" appear twice in log file). For a short caption, no callback is raised for the caption text and "PRE CALLED", as well as "POST CALLED" appear only once.

PPPS: I am using LaTeX2e <2011/06/27> with LuaTeX 0.76.0.

Answer 1

If the caption is short enough to fit into a single line (i.e. no line breaking is needed) the processing of the caption does not generate any callbacks.

As explained by /u/egreg in a comment, the difference is that in some cases, no paragraph is created. For this reason the line break algorithm is not invoked on the horizontal list; consequently, pre_linebreak_filter callback is not triggered. You can observe this difference by putting text into a hbox or vbox:

\hbox {Horizontal list only, without line break.}         \par
\vbox {Vertical list, triggers the line break mechanism.} \par

Only the second line is a affected by line breaking.

Is this behavior intended, which would be in contrast to other single-line content, such as title or author lines, which DO generate these callbacks?

The behavior is indeed intended: LuaTeX offers another callback -- the hpack_filter -- that is invoked whenever TeX packages an hbox. You can use it the same way as the pre_linebreak_filter, although the callback receives further arguments on the Lua side that contain information about the state TeX is presently in.

---

To illustrate the problem I add some example code (tested with Luatex 0.78.2). First the Lua file; its file name should differ from the TeX code below only in the extension:

local traverse_nodes      = node.traverse
local iowrite             = io.write
local tableconcat         = table.concat
local utfchar             = unicode.utf8.char

local nodecodes           = table.mirrored (node.types())
local glyph_t             = nodecodes.glyph
local glue_t              = nodecodes.glue

local cbk = function (hd, group, size, pack)
  --[[--
    Demo function collecting text from glyph nodes in the list starting
    with ``hd``, printing them to stdout.
  --]]--
  local text = {
    "\n["
    .. (group == "" and "par" or group)
    --- the line below shows more info from the hpack_filter context
    --.. (size and (":" .. tostring (size) .. ":" .. pack) or "")
    .. "] "
  }
  for n in traverse_nodes (hd) do
    local ntype = n.id
    if ntype == glyph_t then
      text [#text + 1] = utfchar (n.char)
    elseif ntype == glue_t then
      text [#text + 1] = " "
    else -- print node type
      text [#text + 1] = "<" .. nodecodes [ntype] .. ":" .. tostring (n.subtype) .. ">"
    end
  end

  text [#text + 1] = "\n"
  iowrite (tableconcat (text))
  return true
end

luatexbase.add_to_callback ("pre_linebreak_filter", cbk, "document.nodeprinter")
luatexbase.add_to_callback ("hpack_filter",         cbk, "document.nodeprinter")

And some Latex code that shows different cases:

\documentclass    {scrartcl}
\usepackage       {luatexbase}
\RequireLuaModule {lualibs}
\directlua        {dofile "\jobname.lua"}

\begin {document}

  \hbox {Horizontal list only, without line break.}         \par
  \vbox {Vertical list, triggers the line break mechanism.} \par

  \begin {figure}
    \input ward
    \caption {Some brief caption.} %% same as with the hbox above
  \end {figure}

  This is some \hbox{ordinary} paragraph material.

\end {document}

You can also clone the code conveniently bundled as a gist.

---

Ordinarily, when processing text nodes you want to register your function in both callbacks. There are some caveats, though.

The main problem affects the order of execution: The hpack_filters are called in a separate step prior to handing over the paragraph to the line breaking mechanism. For instance, in the following snippet:

This is some \hbox{ordinary} paragraph material.

the hbox containing the text “ordinary” is processed in the hpack_filter before the rest of the text is passed to the pre_linebreak_filter. Depending on what you want to achieve, this can end up in a mess.

The solution to this problem is to not rely on the hpack_filter and traverse the node lists recursively, e. g.:

local traverse_nodes      = node.traverse
local iowrite             = io.write
local stringformat        = string.format
local stringrep           = string.rep
local nodelength          = node.length
local count_nodes         = node.count

local nodecodes           = table.mirrored (node.types())
local hlist_t             = nodecodes.hlist
local vlist_t             = nodecodes.vlist

local rec_cbk rec_cbk = function (hd, _, depth, where)
  --[[--
    Recursively walk a paragraph node list and print basic info.
  --]]--
  if not depth then
    iowrite "\n"
    depth = 1
    where = "par"
  end
  iowrite (stringrep (" ", depth) ..
           stringformat ("[%s:%d] %d nodes, %d glyphs\n",
                         where, depth,
                         nodelength (hd),
                         count_nodes (glyph_t, hd)))
  for n in traverse_nodes (hd) do
    local ntype = n.id
    if ntype == vlist_t then
      rec_cbk (n.head, nil, depth + 1, "vbox")
    elseif ntype == hlist_t then
      rec_cbk (n.head, nil, depth + 1, "hbox")
      --- else pass
    end
  end
  return true
end

luatexbase.add_to_callback ("pre_linebreak_filter", rec_cbk, "document.recursenodes")

Use this code as above and typeset something slightly more complex:

  This is some \vbox {\hbox {significantly}
                      \hbox {less}
                      \hbox {ordinary}} paragraph material.

The hbox’ed and vbox’ed material will now be encountered in its natural order.

Unfortunately this is not an option if you want to process the float captions. Also, the combined use of pre_linebreak_filter and hpack_filter usually results in better performance, though.