To be absolutely clear before I begin, this is a question about LuaTeX and so "node" in this question means LuaTeX's notion of a node, not any other TeX package's notion.
Ultimately, I want to detect when TeX changes group and record that information in the node list so that when I come to process that node list later on (via
pre_linebreak_filter) then I can figure out the group level that existed when each node was created.
My existing method of doing this was to use an
attribute which was set to
tex.currentgrouplevel. However, as attributes are static, I needed to reset that every time TeX changed group. There doesn't seem to be a callback for that specific event, so instead I used
token_filter to set it every time a token was read. It's not very efficient, but I only wanted it for short segments at a time so I would install the callback when I wanted it and remove it again when done.
Specifically, my lua code for the callback was:
local function setlevel () tex.attribute = tex.currentgrouplevel return token.get_next() end
where I'd already set up the attribute in TeX.
However, the current version of LuaTeX has removed the
token_filter callback so this no longer works! The recommendation appears to be to use the functionality of the
token library, specifically the scanner. However, the examples of this that I have seen imply that it has to be called explicitly. So in order to examine each token as it was read to see if it is makes a group change, I'd have to keep reinvoking the scanner and it would appear that this can only be done by inserting tokens into the input stream. This gets tricky because some tokens may need processing by TeX's mouth and stomach and insert stuff back in to the input stream.
An alternative strategy would be to adjust the node creation code. However, my experiments (admittedly with an earlier version of LuaTeX) with changing the
node.new function show that this is not called when LuaTeX creates a node for itself but only when user-written code is run (I guess this makes sense for implementation reasons).
So, is there a way to achieve what I want with the current version of LuaTeX? Namely, that each created node knows the TeX group level at which it was created.
(In the "Similar Questions" was Tracing group-level and position in LuaTeX which as a question would be a duplicate, but the accepted answer to that suffers from the same issue as it uses