# Paragraph callback to help with widows/orphans hand tuning

It is often hard to tell when adding \looseness=-1 to a paragraph would actually succeed in producing one less row. How can I add a callback to the paragraph builder so that every paragraph is annotated to show where it would work?

I'm thinking something like this: for every paragraph, build it normally, and build it again with the decreased looseness to compare the row count. Then output the "normal" paragraph, but with the last row in red if the decreased-looseness paragraph would be shorter.

The reason I'm asking this: I'm hand-tuning a book to get rid of widows and orphans, and I spend a lot of time jumping around pages looking for places where a +1 or -1 looseness would work and thinking how several adjustments would work together. Currently I have to tentatively add \looseness=-1, recompile and check the result, so the proposed callback would be a huge help.

Edit: The code (update 2) now uses different colors to give more information:

• Red: paragraph can be shrunk
• Blue: paragraph can be expanded
• Magenta: paragraph can be either expanded or shrunk
• Dull cyan: paragraph can be expanded using the current emergency stretch value
• Dull pink: paragraph can be either expanded with emergency stretch or shrunk

Here's an example of the result:

Old: Here's a first attempt at a solution:

\documentclass[twocolumn]{article}
\usepackage[margin=2cm]{geometry}
\usepackage{luatexbase}
\usepackage{lipsum}

% This doesn't seem to work when using tex.linebreak
\setlength\parskip{0pt}

\begin{document}
\directlua{
% Use method aliases for better performance
local nodeid = node.id
local nodenew = node.new
local nodecopy = node.copy
local nodetail = node.tail
local nodeinsertbefore = node.insert_before
local nodeinsertafter = node.insert_after
local nodetraverseid = node.traverse_id

% Get node ids from their names
local HLIST = nodeid("hlist")
local WHAT = nodeid("whatsit")
local COL = node.subtype("pdf_colorstack")

% Make nodes for beginning and end of colored regions
local color_push = nodenew(WHAT,COL)
local color_pop = nodenew(WHAT,COL)
color_push.stack = 0
color_pop.stack = 0
color_push.data = "1 0 0 rg" % PDF code for RGB red
color_push.command = 1
color_pop.command = 2

% Function to color the last line in the given list
local color_last_line = function (n)
% Get the last hlist in the given list
local lastLine
for line in nodetraverseid(HLIST, n) do
lastLine = line
end

% Surround it with color start/stop
end

% Callback to color the last line wherever a decreased looseness would work
local linebreak_filter_test_looseness = function (head, is_display)
% Build a copy of the paragraph with decreased looseness
local nM1, iM1 = tex.linebreak(node.copy_list(head), {looseness=tex.looseness-1})

% Build the paragraph normally

% If decreasing the looseness does decrease the line count, color the last line
if iM1.prevgraf < i.prevgraf then
color_last_line(n)
end

return n
end

% Register callback
}

\lipsum[1]
\lipsum[6]
\lipsum[71]
\lipsum[3]
\lipsum[35]
\lipsum[76]
\lipsum[7-8]

\end{document}


(Thanks to this message and the chickenize documentation.)

You can see the result here.

The problem is that the parskip setting no longer works. For a comparison, see here for the output when the callback is not used: the lines in the left and right columns are perfectly aligned.

It seems to be a general problem when building paragraphs with a callback calling tex.linebreak... Just doing \directlua{ luatexbase.add_to_callback("linebreak_filter", tex.linebreak, "myfilter") } will reproduce the problem.

Update: Here is a workaround for the tex.linebreak issue: a pre-linebreak filter tests if \looseness=-1 would work, and sets a variable that is checked by the post-linebreak filter to do the coloring. It's a bit dirty to communicate the information through an external variable but this way I let LuaTeX build the paragraph internally, so tex.linebreak doesn't get a chance to mess with the baselineskip glues.

\documentclass[twocolumn]{article}
\usepackage[margin=2cm]{geometry}
\usepackage{luatexbase}
\usepackage{lipsum}

\usepackage[callback=]{nodetree}

% This doesn't seem to work when using tex.linebreak
\setlength\parskip{0pt}

\begin{document}
\directlua{
% Use method aliases for better performance
local nodeid = node.id
local nodenew = node.new
local nodecopy = node.copy
local nodetail = node.tail
local nodeinsertbefore = node.insert_before
local nodeinsertafter = node.insert_after
local nodetraverseid = node.traverse_id

% Get node ids from their names
local HLIST = nodeid("hlist")
local WHAT = nodeid("whatsit")
local COL = node.subtype("pdf_colorstack")

% Make nodes for beginning and end of colored regions
local color_push = nodenew(WHAT,COL)
local color_pop = nodenew(WHAT,COL)
color_push.stack = 0
color_pop.stack = 0
color_push.data = "1 0 0 rg" % PDF code for RGB red
color_push.command = 1
color_pop.command = 2

% Set to true when the next post-linebreak filter should color the last line
local ColorLastLine = false

% Function to color the last line in the given list
local color_last_line = function (n)
% Get the last hlist in the given list
local lastLine
for line in nodetraverseid(HLIST, n) do
lastLine = line
end

% Surround it with color start/stop
end

% Callback to check if decreasing the looseness would decrease the line count
local pre_linebreak_test_looseness = function (head, groupeCode)
% Build a copy of the paragraph with decreased looseness
local nM1, iM1 = tex.linebreak(node.copy_list(head), {looseness=tex.looseness-1})

% Build a copy of the paragraph normally

% Store whether decreasing the looseness does decrease the line count, for "post" callback
ColorLastLine = iM1.prevgraf < i.prevgraf

return true
end

% Callback to colorize the last line of the paragraph when ColorLastLine is true
local post_linebreak_color_last_line = function (head, groupcode)
if ColorLastLine then
end
return true
end

% Register callbacks
}

\lipsum[1]
\lipsum[6]
\lipsum[71]
\lipsum[3]
\lipsum[35]
\lipsum[76]
\lipsum[7-8]

\end{document}


Update 2: Here is a standalone Lua file that uses different colors for the last line of the paragraph, depending on whether it can be shrunk, expanded or both (as described at the top of this answer).

File widow-assist.lua:

-- Use method aliases for better performance
local nodecopy = node.copy
local nodecopylist = node.copy_list
local nodetail = node.tail
local nodeinsertbefore = node.insert_before
local nodeinsertafter = node.insert_after
local nodetraverseid = node.traverse_id

-- Get node ids from their names
local HLIST = node.id("hlist")
local WHAT = node.id("whatsit")
local COL = node.subtype("pdf_colorstack")

-- Make nodes for beginning and end of colored regions
local color_p1 = node.new(WHAT,COL)
local color_p1s = node.new(WHAT,COL)
local color_m1 = node.new(WHAT,COL)
local color_pm1 = node.new(WHAT,COL)
local color_pm1s = node.new(WHAT,COL)
local color_pop = node.new(WHAT,COL)
color_p1.stack = 0
color_p1.command = 1
color_p1.data = "0 0 1 rg" -- PDF code for RGB blue
color_p1s.stack = 0
color_p1s.command = 1
color_p1s.data = "0 0.7 0.7 rg" -- PDF code for RGB dark cyan
color_m1.stack = 0
color_m1.command = 1
color_m1.data = "1 0 0 rg" -- PDF code for RGB red
color_pm1.stack = 0
color_pm1.command = 1
color_pm1.data = "1 0 1 rg" -- PDF code for RGB magenta
color_pm1s.stack = 0
color_pm1s.command = 1
color_pm1s.data = "1 .5 .5 rg" -- PDF code for RGB pink
color_pop.stack = 0
color_pop.command = 2

-- Color to use for last line in the next post-linebreak filter call (nil = no color)
local LastLineColor = nil

-- Function to color the last line in the given list
local color_last_line = function (n)
-- Get the last hlist in the given list
local lastLine
for line in nodetraverseid(HLIST, n) do
lastLine = line
end

-- Surround it with color start/stop
end

-- Callback to check if changing the looseness by +-1 would affect the line count
local pre_linebreak_test_looseness = function (head, groupeCode)
-- Disable underfull and overfull boxes reporting

-- Build a copy of the paragraph normally

-- Build a copy of the paragraph with increased looseness and default emergency stretch
local nP1s, iP1s = tex.linebreak(nodecopylist(head), {looseness=tex.looseness+1})

local nP1, iP1
if iP1s.prevgraf > i.prevgraf then
-- It worked with the default emergency stretch, let's try without
nP1, iP1 = tex.linebreak(nodecopylist(head), {looseness=tex.looseness+1, emergencystretch=0})
else
-- Didn't work with emergency stretch, no point to try without
nP1, iP1 = n, i
end

-- Build a copy of the paragraph with decreased looseness
local nM1, iM1 = tex.linebreak(nodecopylist(head), {looseness=tex.looseness-1})

-- Reenable underfull and overfull boxes reporting
luatexbase.remove_from_callback("hpack_quality", "hpqfilter")
luatexbase.remove_from_callback("vpack_quality", "vpqfilter")

-- Set color to use in the post-linebreak callback
if iP1.prevgraf > i.prevgraf and iM1.prevgraf < i.prevgraf then
-- Both +1 and -1 looseness would work
LastLineColor = color_pm1
elseif iP1s.prevgraf > i.prevgraf and iM1.prevgraf < i.prevgraf then
-- Both +1 and -1 looseness would work, but +1 only with emergency stretch
LastLineColor = color_pm1s
elseif iP1.prevgraf > i.prevgraf then
-- Only +1 looseness would work
LastLineColor = color_p1
elseif iP1s.prevgraf > i.prevgraf then
-- Only +1 looseness would work and only thanks to the emergency stretch
LastLineColor = color_p1s
elseif iM1.prevgraf < i.prevgraf then
-- Only -1 looseness would work
LastLineColor = color_m1
else
LastLineColor = nil
end

return true
end

-- Callback to colorize the last line of the paragraph when ColorLastLine is true
local post_linebreak_color_last_line = function (head, groupcode)
if LastLineColor then
end
return true
end

-- Register callbacks


Usage is then as follows:

\documentclass{article}
\usepackage{lipsum}
\usepackage{luatexbase}
\directlua{dofile("widow-assist.lua")}
\begin{document}
\lipsum[1-5]
\end{document}

• This is great! It works like a charm and saves a lot of time fiddling around within multicols-env. Have you considered turning it into a package? Jan 13, 2018 at 19:22
• Glad that you found it useful! I'll think about making a small package out of it... Jan 15, 2018 at 9:27
• Wow, this is so incredibly useful! Packaging this (and I think the best would be to only activate it without the final document options) would be very nice. Thanks! Apr 20, 2018 at 9:19
• Just to get this clear: "Red" means \looseness=-1 will succeed, "Blue" means \looseness=1 will succeed? I guess, other conditions have to apply as well, e.g. the handling of widows. Jun 19, 2018 at 9:29
• @Martin that's right. I don't think there are other conditions... Please let me know if you find some! (I'll try to make a package during the summer, and this should be documented). Jun 19, 2018 at 12:01