# Is there a LuaLaTeX solution to adjust superscript kerning on large delimiters?

Superscripts can sit very far from large parentheses, more for some fonts than others. I am seeking a Lua-based solution to adjust the kerning of math superscripts on delimiters.

Here is an example of this kind of wide placement of superscripts in Latin Modern. The effect is more noticable for larger parentheses.

In another thread, Mico gave a Lua solution for adjusting superscript and subscript placement was given, and that method can be used directly when the delimiter size is known (see below). However, a number of LaTeX environments (such as pmatrix) generate delimiters automatically, and so the delimiter size may not be known a priori. In those cases, a kern adjustment that works on a large size may fail on a small size. It seems that a better solution would ask "Which specific glyph is being superscripted?"

Here is an MWE with the results shown. For known delimiter size, good results are obtained, but for variable sizes (especially the last, nested example) things aren't so good!

% !TEX TS-program = lualatex
\documentclass[11pt]{article}
\usepackage{luacode, mathtools, unicode-math}

\usepackage{mleftright}

%% https://tex.stackexchange.com/questions/432200/ugly-horizontal-spacing-with-some-symbol-subscript-superscript-combinations
\begin{luacode}
function suppl_math_kerning ( s )

-- \Bigr)
s = s:gsub ("(\\Bigr%))%s-%^%s-(%b{})", "%1^{\\mkern-4mu  %2}" )
s = s:gsub ("(\\Bigr%))%s-%^%s-(%w)"  , "%1^{\\mkern-4mu  %2}" )
-- \biggr)
s = s:gsub ("(\\biggr%))%s-%^%s-(%b{})", "%1^{\\mkern-5mu  %2}" )
s = s:gsub ("(\\biggr%))%s-%^%s-(%w)"  , "%1^{\\mkern-5mu  %2}" )
-- \Biggr)
s = s:gsub ("(\\Biggr%))%s-%^%s-(%b{})", "%1^{\\mkern-6.5mu  %2}" )
s = s:gsub ("(\\Biggr%))%s-%^%s-(%w)"  , "%1^{\\mkern-6.5mu  %2}" )
-- \mright)
s = s:gsub ("(\\mright%))%s-%^%s-(%b{})", "%1^{\\mkern-6mu  %2}" )
s = s:gsub ("(\\mright%))%s-%^%s-(%w)"  , "%1^{\\mkern-6mu  %2}" )
-- pmatrix*
s = s:gsub ("(\\end{pmatrix%*})%s-%^%s-(%w)"  , "%1^{\\mkern-5mu %2 }" )
s = s:gsub ("(\\end{pmatrix%*})%s-%^%s-(%b{})", "%1^{\\mkern-5mu %2 }" )

return s
end
\end{luacode}
\newcommand\SupplKernOn{\directlua{luatexbase.add_to_callback ("process_input_buffer" , suppl_math_kerning , "suppl_math_kerning" ) }}
\newcommand\SupplKernOff{\directlua{luatexbase.remove_from_callback ("process_input_buffer" , "suppl_math_kerning" ) }}

\begin{document}
\SupplKernOn

$\verb|\Bigr):| \quad\cos\Bigl(\frac{a}{b}\Bigr)^2 \qquad \verb|\biggr):| \quad\cos\biggl(\frac{a^2}{b_1}\biggr)^2 \qquad \verb|\Biggr):| \quad\cos\Biggl(\frac{a^2}{b_1}\Biggr)^2$

$\verb|\end{pmatrix*}:| \quad \begin{pmatrix*}[r] 1 & 0 \\0 & 1 \\ -1 & 0 \end{pmatrix*}^T \begin{pmatrix*}[r] 1 & 1 & 0 \\- 1 & 1 & 1 \end{pmatrix*}^T$

$\verb|\mright):|\quad\cos\mleft(\frac{a_1}{\mleft(b_1+c_1\mright)^2}\mright)^{t_r}$

\end{document}


So, I have two questions:

1. Can the Lua approach in my MWE be adjusted to deal with the cases for which the delimiter size is unknown and variable; or
2. Can an appropriate kerning feature be created via fonts.handlers.otf.addfeature to adjust the kerning of superscripts on the basis of the specific delimiter glyph being called, e.g., along the lines given by Thérèse in this thread. I'm wondering whether a specific set of kerns might be defined for each of several specific parentheses glyphs, or whether the superscripting process itself might make that impossible.
• I'd use the mlist_to_hlist callback and check whether a close_noad is followed by a math_sup_list and insert kerning depending on the type of fence_noad. – Henri Menke Sep 12 at 22:50
• The correct way is imho to use a font with mathkern values (or set them). See section 6.14 Math kerning in super- and subscripts in the luatex documentation. I don't know which font really sets the values and also if the fontloader makes correct use of them and I don't have the time now to investigate but imho there was a discussion on the context list some time ago about it. – Ulrike Fischer Sep 13 at 7:56
• @UlrikeFischer - Thanks for this input. I agree that dealing with the mathkerns at the font level seems best. However, I lack the technical knowledge of Luatex syntax etc. to execute! If you know a link for the context list, I can explore. – John Sep 15 at 1:05
• @HenriMenke - Thanks for these suggestions on callbacks that can unpack the construction of the superscript. Can you point me toward an example? – John Sep 15 at 1:06
• You can find links to the mailing list in the contextgarden wiki.contextgarden.net/Main_Page – Ulrike Fischer Sep 15 at 7:17

Because you asked in the comments, I present you my blueprint for how to do this. This is far from complete and does only handle \left...\right.

Let me briefly explain, why it only handles \left...\right. The \left...\right pair is always a sub_mlist and the \right atom is always the last element in this sublist. So if the sublist has a superscript, I can simply scan whether there is a right fence (I don't even have to check whether it is last, because it is guaranteed to be last). I could not find where the fence scaling is stored, so currently all right fences will get the same amount of kerning, regardless of their size. Furthermore I haven't adapted kerning to the contained mathchar () vs. ]) but that's easy to do.

So good, so easy. It becomes exponentially more complicated with \big and friends and with \mleft...\mright. It is worst for \big because the implementation differs between plain LaTeX and AMSmath.

% plain LaTeX (actually inherited from Plain TeX)
\def\big#1{{\hbox{$\left#1\vbox to8.5\p@{}\right.\n@space$}}}
% amsmath
\renewcommand{\big}{\bBigg@\@ne}
\def\bBigg@#1#2{%
{\@mathmeasure\z@{\nulldelimiterspace\z@}%
{\left#2\vcenter to#1\big@size{}\right.}%
\box\z@}}
\def\@mathmeasure#1#2#3{\setbox#1\hbox{\frozen@everymath\@emptytoks
\m@th$#2#3$}}


Both pack the fence into a box but LaTeX uses \vbox whereas AMSmath uses \vcenter, so the resulting mlist will look different. The nested hlist just adds to the complication. When walking the list, you'd have to propagate the fact that there is a nested fence in the hlist to the surrounding mlist.

\documentclass[11pt]{article}
\usepackage{luacode, mathtools, unicode-math}

\begin{luacode*}

local noad_id = node.id("noad")
local fence_id = node.id("fence")
local subtypes = { fence = table.swapped(node.subtypes("fence")) }

local pt = 2^16
local sublist_has_sup = {}
if v.id == noad_id and v.nucleus and v.nucleus.list then
-- Descend into sublists
sublist_has_sup[level] = v.sup and true or false
kern_right_fence(v.nucleus.list, level+1)
sublist_has_sup[level] = false
elseif v.id == fence_id and v.subtype == subtypes.fence.right and sublist_has_sup[level-1] then
-- For a fence with a superscript insert kern
local kern = node.new("kern")
kern.kern = 10*pt
end
end
end

print("===mlist_to_hlist===")
end
,"kern right fence")

\end{luacode*}

\begin{document}

$\left(y\right)^2$

$\begin{pmatrix*}[r] 1 & 0 \\0 & 1 \\ -1 & 0 \end{pmatrix*}^T$

\end{document}


Here I inserted enourmous positive kerning for the purpose of illustration.

• The complexity of the problem and the needed solution gives plenty of food for thought. Thanks for the blueprint! – John Sep 22 at 14:52