Is there a LuaLaTeX solution to adjust superscript kerning on large delimiters?

Question

Superscripts can sit very far from large parentheses, more for some fonts than others. I am seeking a Lua-based solution to adjust the kerning of math superscripts on delimiters.

Here is an example of this kind of wide placement of superscripts in Latin Modern. The effect is more noticable for larger parentheses.

In another thread, Mico gave a Lua solution for adjusting superscript and subscript placement was given, and that method can be used directly when the delimiter size is known (see below). However, a number of LaTeX environments (such as pmatrix) generate delimiters automatically, and so the delimiter size may not be known a priori. In those cases, a kern adjustment that works on a large size may fail on a small size. It seems that a better solution would ask "Which specific glyph is being superscripted?"

Here is an MWE with the results shown. For known delimiter size, good results are obtained, but for variable sizes (especially the last, nested example) things aren't so good!

% !TEX TS-program = lualatex
\documentclass[11pt]{article}
\usepackage{luacode, mathtools, unicode-math}

\usepackage{mleftright}

%% Adapted from Mico's answer here: 
%% https://tex.stackexchange.com/questions/432200/ugly-horizontal-spacing-with-some-symbol-subscript-superscript-combinations
\begin{luacode}
function suppl_math_kerning ( s )

  -- \Bigr)
  s = s:gsub ("(\\Bigr%))%s-%^%s-(%b{})", "%1^{\\mkern-4mu  %2}" )
  s = s:gsub ("(\\Bigr%))%s-%^%s-(%w)"  , "%1^{\\mkern-4mu  %2}" )
  -- \biggr)
  s = s:gsub ("(\\biggr%))%s-%^%s-(%b{})", "%1^{\\mkern-5mu  %2}" )
  s = s:gsub ("(\\biggr%))%s-%^%s-(%w)"  , "%1^{\\mkern-5mu  %2}" )
  -- \Biggr)
  s = s:gsub ("(\\Biggr%))%s-%^%s-(%b{})", "%1^{\\mkern-6.5mu  %2}" )
  s = s:gsub ("(\\Biggr%))%s-%^%s-(%w)"  , "%1^{\\mkern-6.5mu  %2}" )
  -- \mright)
  s = s:gsub ("(\\mright%))%s-%^%s-(%b{})", "%1^{\\mkern-6mu  %2}" )
  s = s:gsub ("(\\mright%))%s-%^%s-(%w)"  , "%1^{\\mkern-6mu  %2}" )
  -- pmatrix*
  s = s:gsub ("(\\end{pmatrix%*})%s-%^%s-(%w)"  , "%1^{\\mkern-5mu %2 }" )
  s = s:gsub ("(\\end{pmatrix%*})%s-%^%s-(%b{})", "%1^{\\mkern-5mu %2 }" )

  return s
end
\end{luacode}
\newcommand\SupplKernOn{\directlua{luatexbase.add_to_callback ("process_input_buffer" , suppl_math_kerning , "suppl_math_kerning" ) }}
\newcommand\SupplKernOff{\directlua{luatexbase.remove_from_callback ("process_input_buffer" , "suppl_math_kerning" ) }}
%% End adapted code


\begin{document}
\SupplKernOn

\[ \verb|\Bigr):| \quad\cos\Bigl(\frac{a}{b}\Bigr)^2 \qquad
   \verb|\biggr):| \quad\cos\biggl(\frac{a^2}{b_1}\biggr)^2 \qquad
   \verb|\Biggr):| \quad\cos\Biggl(\frac{a^2}{b_1}\Biggr)^2\]

\[ \verb|\end{pmatrix*}:| \quad \begin{pmatrix*}[r] 1 & 0 \\0 & 1 \\ -1 & 0 \end{pmatrix*}^T 
                                \begin{pmatrix*}[r] 1 & 1 & 0  \\- 1 & 1 & 1 \end{pmatrix*}^T \]

\[ \verb|\mright):|\quad\cos\mleft(\frac{a_1}{\mleft(b_1+c_1\mright)^2}\mright)^{t_r} \]

\end{document}

So, I have two questions:

1. Can the Lua approach in my MWE be adjusted to deal with the cases for which the delimiter size is unknown and variable; or
2. Can an appropriate kerning feature be created via fonts.handlers.otf.addfeature to adjust the kerning of superscripts on the basis of the specific delimiter glyph being called, e.g., along the lines given by Thérèse in this thread. I'm wondering whether a specific set of kerns might be defined for each of several specific parentheses glyphs, or whether the superscripting process itself might make that impossible.

Answer 1

Because you asked in the comments, I present you my blueprint for how to do this. This is far from complete and does only handle \left...\right.

Let me briefly explain, why it only handles \left...\right. The \left...\right pair is always a sub_mlist and the \right atom is always the last element in this sublist. So if the sublist has a superscript, I can simply scan whether there is a right fence (I don't even have to check whether it is last, because it is guaranteed to be last). I could not find where the fence scaling is stored, so currently all right fences will get the same amount of kerning, regardless of their size. Furthermore I haven't adapted kerning to the contained mathchar () vs. ]) but that's easy to do.

So good, so easy. It becomes exponentially more complicated with \big and friends and with \mleft...\mright. It is worst for \big because the implementation differs between plain LaTeX and AMSmath.

% plain LaTeX (actually inherited from Plain TeX)
\def\big#1{{\hbox{$\left#1\vbox to8.5\p@{}\right.\n@space$}}}
% amsmath
\renewcommand{\big}{\bBigg@\@ne}
\def\bBigg@#1#2{%
  {\@mathmeasure\z@{\nulldelimiterspace\z@}%
     {\left#2\vcenter to#1\big@size{}\right.}%
   \box\z@}}
\def\@mathmeasure#1#2#3{\setbox#1\hbox{\frozen@everymath\@emptytoks
  \m@th$#2#3$}}

Both pack the fence into a box but LaTeX uses \vbox whereas AMSmath uses \vcenter, so the resulting mlist will look different. The nested hlist just adds to the complication. When walking the list, you'd have to propagate the fact that there is a nested fence in the hlist to the surrounding mlist.

\documentclass[11pt]{article}
\usepackage{luacode, mathtools, unicode-math}

\begin{luacode*}

local noad_id = node.id("noad")
local fence_id = node.id("fence")
local subtypes = { fence = table.swapped(node.subtypes("fence")) }

local pt = 2^16
local sublist_has_sup = {}
function kern_right_fence(head, level)
    for v in node.traverse(head) do
        if v.id == noad_id and v.nucleus and v.nucleus.list then
            -- Descend into sublists
            sublist_has_sup[level] = v.sup and true or false
            kern_right_fence(v.nucleus.list, level+1)
            sublist_has_sup[level] = false
        elseif v.id == fence_id and v.subtype == subtypes.fence.right and sublist_has_sup[level-1] then
            -- For a fence with a superscript insert kern
            local kern = node.new("kern")
            kern.kern = 10*pt
            node.insert_after(head, v, kern)
        end
    end
    return head
end

luatexbase.add_to_callback("mlist_to_hlist",
                           function(head, style, penalties)
                               print("===mlist_to_hlist===")
                               head = kern_right_fence(head, 0)
                               return node.mlist_to_hlist(head, style, penalties)
                           end
                           ,"kern right fence")

\end{luacode*}

\begin{document}

\[ \left(y\right)^2 \]

\[ \begin{pmatrix*}[r] 1 & 0 \\0 & 1 \\ -1 & 0 \end{pmatrix*}^T \]

\end{document}

Here I inserted enourmous positive kerning for the purpose of illustration.