# Using post_linebreak_filter in LuaTeX to set height and depth of boxes

After reading this answer Automatically prevent extra line spacing because of math by David Carlisle, I experimented with the code to get grid typesetting. My approach: The height of a line is always (n - 0.3)\baselineskip and the depth (n - 0.7)\baselineskip with n ∈ ℕ. Then I delete the glue. This works fine for big inline math or other big obejects in horizontal mode, but it does not work with display math as you can see in the MWE.

So I have some questions:

1. What ist wrong here?
2. How can I influence the height and depth of a display math line in post_linebreak_filter or somewhere else?
3. How can I get the spacing right in this case? That means that the baseline of the display math should be on the grid and the lines after the display math should also obey the grid.

MWE:

\documentclass[12pt]{article}

\usepackage{showframe,color,unicode-math}

% Check grid typesetting
\def\grid{%
\vtop to0pt{%
\hsize=0pt \noindent \color{red}%
X\\X\\X\\X\\X\\X\\X\\X\\X\\X\\X\\X\\X\\X\\X\\X\\X\\X\vss}}

\raggedbottom

\directlua{
function fixlines(h,c)
local B=tex.baselineskip.width
for n in node.traverse(h) do
%
% correct height and depth
if (n.id==0 or n.id==11) then
n.height = (math.ceil((n.height+.3*B)/B)-.3)*B
n.depth  = (math.ceil((n.depth+.7*B)/B)-.7)*B
end
%
% lineskip or baselineskip: kill glue
if n.id==12 and (n.subtype==1 or n.subtype==2) then
n.width=0 % kill glue
end
end
return h
end
}

% test big objects
\def\𝟙{\begin{pmatrix}1&0&0\\0&1&0\\0&0&1\end{pmatrix}}

\usepackage{lipsum}

\begin{document}

\parskip0pt
\lineskip0pt
\lineskiplimit0pt
% displaymath spacing
\abovedisplayskip\baselineskip
\abovedisplayshortskip\baselineskip
\belowdisplayskip\baselineskip
\belowdisplayshortskip\baselineskip

\noindent \grid I like grid typesetting. Let us test some inline math. Let us
test some very big inline math with : $𝟙₃ = \𝟙$. The next line obeys the grid,
very good! 2 baselines were omitted above, one below.

What about \detokenize{$$}-math? % Sorry for using$$, but I usually use Plain LuaTeX.
$$a+b+c+d~\hbox{Why don't I have the same baseline as the red X?}$$
The grid is not obeyed any more. What is wrong here? How can I influence the
height and depth of a display math line and its spacing in \detokenize{post_linebreak_filter}?

\end{document}


Edit: I have added the following code:

\directlua{
local B=tex.baselineskip.width
function killdisplayskips(h)
for n in node.traverse(h) do
if n.id==12 and (n.subtype==4 or n.subtype==5 or n.subtype==6 or n.subtype==7) then
n.width=0 % kill glue
end
end
return h
end
}


The spacing is still wrong (but it changed).

• display math doesn't line break so won't be triggered by the post linebreak callback append_to_vlist_filter perhaps that has a location argument that says when it is handling math Jan 31, 2021 at 14:34
• Is this somewhere documented? I tried to read the LuaTeX manual, but it ist too difficult for a beginner. Feb 1, 2021 at 16:03
• the manual gives a list of all the callbacks but finding usable documentation of how to use them involves searching the internet for examples Feb 1, 2021 at 16:19
• When I try the same with append_to_vlist_filter the spacing inside the matrix is also changed. But I am still not able to change the height and depth of a line that contains display math. Feb 1, 2021 at 21:06
• Slightly off-topic: ConTeXt Mark IV, also based on LuaTeX, does grid typesetting and is an alternative once you consider to give up on LaTeX...
– user226564
Feb 2, 2021 at 15:02

After a sleepless night I found a solution. In the append_to_vlist_filter you can manipulate normal lines (n.subtype==1), equations (n.subtype==6, that means lines of display math) und equationnumbers (n.subtype==7, contents after \eqno or \leqno). Now every display math line has a corrected height and depth, too. This is grid typesetting!!!! :)

Complete MWE:

\documentclass[12pt]{article}

\usepackage{showframe,color,unicode-math}
\usepackage{lua-visual-debug}

% Check grid typesetting
\def\grid{%
\vtop to0pt{%
\hsize=0pt \noindent \color{red}%
X\\X\\X\\X\\X\\X\\X\\X\\X\\X\\X\\X\\X\\X\\X\\X\\X\\X\vss}}

\raggedbottom

\directlua{
function fixlines(h,c)
local B=tex.baselineskip.width
for n in node.traverse(h) do
%
% correct height and depth
if n.id==0 and (n.subtype==1 or n.subtype==6 or n.subtype==7) then
n.height = (math.ceil((n.height+.29*B)/B)-.3)*B
n.depth  = (math.ceil((n.depth+.69*B)/B)-.7)*B
end
%
% lineskip or baselineskip: kill glue
if n.id==12 and (n.subtype==1 or n.subtype==2) then
n.width=0 % kill glue
end
end
return h
end
}

% test big objects
\def\𝟙{\begin{pmatrix}1&0&0\\0&1&0\\0&0&1\end{pmatrix}}

\begin{document}

\parskip0pt
\lineskip0pt
\lineskiplimit0pt
\normallineskiplimit0pt
\jot0pt
% displaymath spacing
\abovedisplayskip\baselineskip
\abovedisplayshortskip\baselineskip
\belowdisplayskip\baselineskip
\belowdisplayshortskip\baselineskip

\noindent \grid I like grid typesetting. Let us test some inline math. Let us
test some very big inline math with : $𝟙₃ = \𝟙\displaystyle∫$. The next line obeys the grid, very good!

What about \detokenize{$$}-math?%\vrule depth.3\baselineskip width2pt % Sorry for using$$, but I usually use Plain LuaTeX.
$$X+A+a+b+c+d∬~\hbox{Now I have the same baseline as the red X!}$$
The grid is obeyed after all diplay math lines. Their height and depth do not
matter any more!

\end{document}


Edit: The solution above suffers from gobbled baselineskip in some situations (see wipet's comment). I found out that you have to calculate the interline skips on your own when using append_to_vlist_filter, otherwise they are gobbled.

The calculation is taken from: Underline part of a word while preserving kerning

So replace the luacode in the MWE by:

\directlua{
function correctheight(h, loc)
local B=tex.baselineskip.width
for n in node.traverse(h) do
% correct height and depth of lines (post_linebreak) and display math incl.
% \(l)eqno
if loc=='post_linebreak' or loc=='equation' or loc=='equation_number' then
n.height = (math.ceil((n.height+.29*B)/B)-.3)*B
n.depth  = (math.ceil((n.depth+.69*B)/B)-.7)*B
end
end
return h
end
%
function makegrid(h, loc, prev, mirror)
correctheight(h, loc)
% If append_to_vlist_filter is used then you have to calculate interline skips
% on your own, otherwise they are gobbled!
local new_prev = mirror and h.height or h.depth
if prev > -65536000 then
local lineglue = tex.baselineskip.width - prev - (mirror and h.depth or h.height)
local skip
if lineglue < tex.lineskiplimit then
skip = node.new('glue', 1)
node.setglue(skip, node.getglue(tex.lineskip))
else
skip = node.new('glue', 2)
node.setglue(skip, node.getglue(tex.baselineskip))
skip.width = lineglue
end
skip.next = h
h = skip
end
return h, new_prev
end

• It is very interesting. But I don't understand why \vbox{\hsize=10pt \noindent A\\B\\C} works correctly but \vbox{\halign{#\cr A\cr B\cr C\cr}} breaks baselineskip between lines A, B, C. Feb 3, 2021 at 5:37
• @wipet: I have fixed this. append_to_vlist_filter gobbles the interline skips when not calculated new. Your example works now. See edit. Feb 3, 2021 at 14:05
• But there is still a problem: Alignment displays (like \displaylines) do not obey the grid. When I also set loc=='alignment' in the function correctheight then it works, but also the spacing inside a \matrix is changed to grid typesetting. This is a waste of space, of course. Feb 3, 2021 at 14:09
• So if someone knows how to treat alignments directlly inside display math mode and all the others differently, please give me some advice. Feb 3, 2021 at 14:43

This is more of a comment rather than an answer. FWIW, grid typesetting with math works out of the box in ConTeXt:

\setuplayout[grid=yes]
\showgrid
\definemathmatrix[pmatrix][left={\left(\,}, right={\,\right)},simplecommand=PMATRIX]
\def\𝟙{\PMATRIX{1,0,0; 0,1,0; 0,0,1}}
\starttext
I like grid typesetting. Let us test some inline math. Let us
test some very big inline math with : $𝟙₃ = \𝟙\displaystyle∫$. The next line obeys the grid, very good!

What about \detokenize{$$}-math?$$
X+A+a+b+c+d∬~\hbox{Now I have the same baseline as the red X!}

The grid is obeyed after all diplay math lines. Their height and depth do not
matter any more!

\stoptext


which gives

You can also add \showboxes in the preamble to visualize the boxes (but it is a bit crowded with the grid lines)