# How do I define a new TeX register with arguments in LuaTeX?

Some time ago I tried to implement the \pdfadjustinterwordglue from pdfTeX in LuaTeX using a callback. Getting the callback to work is pretty simple, but there is another major problem in the way which I was so far not able to solve. The culprit is in the way parameters for spacing are set, which is for example

\knbscode\fontr=1100


This is a regular TeX assignment and this can be implemented in Lua using

\def\knbscode{\directlua{

local fid = token.scan_int()
local chr = token.scan_int()
local str = token.scan_string()

knbscode[fid] = knbscode[fid] or {}
knbscode[fid][chr] = knbscode[fid][chr] or {}
local int = string.gsub(str,"=","")
knbscode[fid][chr][field] = tonumber(int)

}\fontid}


That is to say, I look ahead for an integer, which is the font ID, another integer which is a character, and “string”, which according the LuaTeX manual is

returns a string given between {}, as \macro or as sequence of characters with catcode 11 or 12

The mindful reader might already notice a problem here because \knbscode\fontr=1100abc should parse by the normal TeX rules but will fail in this case. Luckily the usage of register assignment is sort of regular in TeX and such a situation rarely occurs.

The next big problem is that registers are not write-only. They can also be read using:

\the\knbscode\fontr


Of course, in this case, \knbscode should still scan for the font ID and the character but not for the assignment. My idea to solve this was to overload \the to peek at the next token and if it is \knbscode, delegate to another function. On the Lua end this looks like this

local t = lua.get_functions_table()
t[1] = function()
local next = token.get_next()
if next.csname == "knbscode" then
token.put_next(token.create("theknbscode"))
elseif next.csname == "stbscode" then
token.put_next(token.create("thestbscode"))
elseif next.csname == "shbscode" then
token.put_next(token.create("theshbscode"))
else
token.put_next{ token.create("normalthe"), next }
end
end


and on the TeX end I define \the to refer to this Lua function

\luadef\the1


such that it can expand within a single step. Except that it doesn't. My new \the does not expand within a single step because instead of putting the number from the register in the input stream I put a token which has to be expanded again.

Maybe this approach is just wrong altogether, so the question is relatively general:

How do I define a new TeX register with arguments in LuaTeX?

I actually have asked the same question some time ago on the LuaTeX mailing list with a simplified example, so I just got the answer: “well, use a normal register, duh” (https://tug.org/pipermail/luatex/2019-January/007040.html).

Below I have a full MWE with which you can play around.

\documentclass{article}
\usepackage{luacode}

\begin{luacode}

local subtypes = node.subtypes("glue")

local knbscode = {}

local t = lua.get_functions_table()
t[1] = function()
local next = token.get_next()
if next.csname == "knbscode" then
token.put_next(token.create("theknbscode"))
elseif next.csname == "stbscode" then
token.put_next(token.create("thestbscode"))
elseif next.csname == "shbscode" then
token.put_next(token.create("theshbscode"))
else
token.put_next{ token.create("normalthe"), next }
end
end

function set_knbscode(field)
local fid = token.scan_int()
local chr = token.scan_int()
local str = token.scan_string()

knbscode[fid] = knbscode[fid] or {}
knbscode[fid][chr] = knbscode[fid][chr] or {}
local int = string.gsub(str,"=","")
knbscode[fid][chr][field] = tonumber(int)
end

function get_knbscode(field)
local fid = token.scan_int()
local chr = token.scan_int()

knbscode[fid] = knbscode[fid] or {}
knbscode[fid][chr] = knbscode[fid][chr] or {}
tex.sprint([[\numexpr]] .. (knbscode[fid][chr][field] or 0) .. [[\relax]])
end

for space in node.traverse_id(node.id("glue"), head) do
if subtypes[space.subtype] == "spaceskip" then
local prev = node.prev(space)
if prev.id == node.id("glyph") then
local knbs = knbscode[prev.font]
if knbs and knbs[prev.char] then
local f = font.getfont(prev.font)

local width = knbs[prev.char].width or 0
local stretch = knbs[prev.char].stretch or 0
local shrink = knbs[prev.char].shrink or 0

local glue = node.new(node.id("glue"))
glue.width = width*em/1000
glue.stretch = stretch*em/1000
glue.shrink = shrink*em/1000

end
end
end
end
end
end


\end{luacode}

\protected\def\knbscode{\directlua{set_knbscode("width")}\fontid}
\protected\def\stbscode{\directlua{set_knbscode("stretch")}\fontid}
\protected\def\shbscode{\directlua{set_knbscode("shrink")}\fontid}

\let\normalthe\the

\def\theknbscode{\normalthe\directlua{get_knbscode("width")}\fontid}
\def\thestbscode{\normalthe\directlua{get_knbscode("stretch")}\fontid}
\def\theshbscode{\normalthe\directlua{get_knbscode("shrink")}\fontid}

% Bogus values, just for demonstration
\knbscode\fontr=1100
\stbscode\fontr=10
\shbscode\fontr=10

\knbscode\fontr 1100
\stbscode\fontr 10
\shbscode\fontr 10

\begin{document}

\the\knbscode\fontr
\the\stbscode\fontr
\the\shbscode\fontr

\input lorem

\end{document}

• Apart from the theoretical point if it is possible: I don't think that it is really needed to try to emulate the low-level interface. Offer a sensible interface to set and get the values and ask microtype to use it. – Ulrike Fischer Jun 30 '19 at 9:50

tl;dr: I do not think defining TeX-like registers from Lua is possible, but you can get a bit closer.

Fixing \the is easy: You only have to trigger one level of expansion form Lua. You could try token.expand but that function has a unconventional understanding which token to expand. So it's easier to rely on the standard TeX tool to get one level of expansion: \expandafter: You can insert \expandafter\relax into TeX's input stream, then read the \relax back with expansion.

But there are other problems. Let's assume that we want a "g" to use the same value as a "f". In pdfTeX we could say

\knbscode\fontg=\knbscode\fontf


This wouldn't work with emulated registers like this. This is because of two problems:

Setting \knbscode does only accept strings Lua con convert into numbers and not general TeX numbers and \knbscode works with \the, but still isn't an internal TeX number which can be used in all places where TeX accepts numbers. The first problem can be fixed: To read an optional = followed by an integer, use

token.scan_keyword'=' -- Scans an optional "="
local int = token.scan_int()


The second problem is harder to fix. If you want your register-like Lua value to be usable at all places which accept TeX numbers, you would need some way to determine from Lua if TeX tries to read a register or not. This information isn't available, so I do not think that perfectly emulating a register is possible.

Of course you can say that it's nice be be at least similar, but sometimes having a slightly different interface for the Lua version is better than to have subtly, much harder to detect, differences in usage.

The suggested changes lead to this version:

\documentclass{article}
\usepackage{luacode}

\begin{luacode}

local subtypes = node.subtypes("glue")

local knbscode = {}

-- Let's mmake sure that no one messes with our primitives:
local function frozentok(name)
local tok = token.create(name)
end
local the, expandafter, relax = frozentok'the', frozentok'expandafter', frozentok'relax'
local t = lua.get_functions_table()
t[1] = function()
local next = token.get_next()
if next.csname == "knbscode" then
token.put_next(token.create("theknbscode"))
elseif next.csname == "stbscode" then
token.put_next(token.create("thestbscode"))
elseif next.csname == "shbscode" then
token.put_next(token.create("theshbscode"))
else
token.put_next{ next }
end
token.put_next{ expandafter, relax, the }
token.scan_token() -- The scanned token will always be the relax we just wrote, but expandafter does it's magic in the progress.
end

function set_knbscode(field)
local fid = token.scan_int()
local chr = token.scan_int()
token.scan_keyword'='
local int = token.scan_int()

knbscode[fid] = knbscode[fid] or {}
knbscode[fid][chr] = knbscode[fid][chr] or {}
knbscode[fid][chr][field] = int
end

function get_knbscode(field)
local fid = token.scan_int()
local chr = token.scan_int()

knbscode[fid] = knbscode[fid] or {}
knbscode[fid][chr] = knbscode[fid][chr] or {}
tex.sprint([[\numexpr]] .. (knbscode[fid][chr][field] or 0) .. [[\relax]])
end

for space in node.traverse_id(node.id("glue"), head) do
if subtypes[space.subtype] == "spaceskip" then
local prev = node.prev(space)
if prev.id == node.id("glyph") then
local knbs = knbscode[prev.font]
if knbs and knbs[prev.char] then
local f = font.getfont(prev.font)

local width = knbs[prev.char].width or 0
local stretch = knbs[prev.char].stretch or 0
local shrink = knbs[prev.char].shrink or 0

local glue = node.new(node.id("glue"))
glue.width = width*em/1000
glue.stretch = stretch*em/1000
glue.shrink = shrink*em/1000

end
end
end
end
end
end

\end{luacode}

\protected\def\knbscode{\directlua{set_knbscode("width")}\fontid}
\protected\def\stbscode{\directlua{set_knbscode("stretch")}\fontid}
\protected\def\shbscode{\directlua{set_knbscode("shrink")}\fontid}

\def\theknbscode{\directlua{get_knbscode("width")}\fontid}
\def\thestbscode{\directlua{get_knbscode("stretch")}\fontid}
\def\theshbscode{\directlua{get_knbscode("shrink")}\fontid}

% Bogus values, just for demonstration
\knbscode\fontr=1100
\stbscode\fontr=10
\shbscode\fontr=10

\knbscode\fontr 1100
\stbscode\fontr 10
\shbscode\fontr 10

\begin{document}

\the\knbscode\fontr
\the\stbscode\fontr
\the\shbscode\fontr

\input lorem

\end{document}

• I've had a look in the source. It seems that to support this in the engine, we'd need Lua hooks into the case assign_font_int_cmd: in prefixed_command and scan_something_internal. – Henri Menke Jul 1 '19 at 5:09
• @HenriMenke You certainly have to patch scan_something_internal but instead of reusing the font commands you probably would add new commands. Then you might not even need to use prefixed_command: The only prefix that applies to registers is \global and Lua registers in most cases will not know about local/global settings(Like \knbscode`). – Marcel Krüger Jul 1 '19 at 9:16
• It should be a pretty straight forward patch, the question is if there is any sufficiently good use case to warrent including it into the engine. – Marcel Krüger Jul 1 '19 at 9:18
• @MarcelKrüger What criteria are used for adding something to the LuaTeX engine? – Faheem Mitha Aug 15 '19 at 6:25
• @FaheemMitha In general I never found a real pattern in what they accept and don't accept, but currently LuaTeX is pretty much frozen, so new features are generally rejected. That doesn't mean that there can't be any exceptions though, so you can try... – Marcel Krüger Aug 15 '19 at 8:17