# Will LaTeX3 have proper support for hooks? Does LuaTeX?

Tex has powerful support for extensions triggered by events happen in generating output, by running one of six token lists appropriately:

 \everycr \everydisplay \everyhbox \everymath \everypar \everyvbox


They're used with effect in such places as Latex classes, but they are much less useful than they could be, because the \everyfoo={\t\o ks} assignment simply wipes the current value, making it rather tricky to use them by authors of packages and documents. Emacs' Elisp has a widely understood technology for registering and deregistering code to run at events, namely hooks.

The LaTeX3 Project's expl3 library offers supports for managing token lists, and LuaTeX augments TeX's event handling mechanism with a set of Lua callbacks. Are these the beginning of proper support for event-driven TeX code, as usable as hooks? Is this a priority for developers on these projects?

-
 Shouldn't this be tagged 'token-registers' not 'token-lists'? – Joseph Wright♦ Aug 19 '10 at 19:10 @Joseph: I think token list is the more general concept. Looking at The TeXbook: Knuth (p212) actually calls \everycr, &c, "token list parameters", to contrast them with \toks0..., which he calls "token list registers". – Charles Stewart Aug 19 '10 at 20:14 I wonder if the more usual 'toks' might be better in any case! – Joseph Wright♦ Aug 19 '10 at 20:31 @Joseph: Not intuitive to me - that seems to me to be about \toks0... and not \everycr, &c. – Charles Stewart Aug 19 '10 at 20:38 My point is that \everycr works like any other token register, for example the \everycr{tokens} business. Perhaps this is because I'm a TeX programmer, and so used to having to think in terms of toks and macros as the two ways to store tokens in TeX. – Joseph Wright♦ Aug 19 '10 at 20:40

LuaTeX does not offer these hooks out of the box, but you can make them yourself. The LuaLaTeX people do this (thanks!). See http://github.com/mpg/luatexbase (and there: luatexbase-mcb.dtx).

-
This is great: it does exactly what I think it should! Is there similar code for Context? – Charles Stewart Aug 19 '10 at 8:52
ConTeXt has had support for token list hooks since (almost) day one, even in traditional TeX engines. And yes, it also has callback support in LuaTeX, and that has been there for a while already as well. – Taco Hoekwater Aug 21 '10 at 8:36
I am not 100% sure that the callbacks behave as emacs hooks, where you can say "add this function to the beginning / end of the callback and remove this and that function". – dıʞsdoʇ Aug 21 '10 at 15:37
@Patrick: LuaTeX itself doesn't provide such functionality, but it is precisely the goal of luatexbase-mcb (mcb means "multiple callback") to provide it. It allows to register many functions in most callbacks, with a priority system. – mpg Oct 28 '10 at 13:31
@mpg, this is what I tried to express... I really like this interface, nice job! – dıʞsdoʇ Oct 28 '10 at 17:34
show 1 more comment

Sure, it would be great if LaTeX3 provided hooks for this sort of thing—not using the primitives directly, but rather providing an interface to a reliable way to use these token registers.

Unfortunately, ripping out the current LaTeX2e machinery for, say, giving a good way to access \everypar simply isn't possible without breaking backwards compatibility with lots and lots of current packages. Experimental LaTeX3 packages such as galley are written to start thinking about these ideas, but we're quite a way away from anything that can be used for general purposes.

-
 So you see Latex3 supporting both class files written the Latex2e way, and the new, improved Latex3 way. Will there be some mechanism for package writers to see what the class supports, like, say, a hookable-events flag? – Charles Stewart Aug 19 '10 at 11:40 Will is referring to 'l3in2e', i.e. loading bits of LaTeX3 as LaTeX2e packages. The deal with these is that they should not interfere with anything else, so for example you can load expl3 and it will not alter the behaviour of other packages. A LaTeX3-only kernel is a different matter. – Joseph Wright♦ Aug 19 '10 at 19:06 I should also add that in LaTeX3 we'll probably use macros for the storage, not token registers. See l3tl for how this is done (we use an e-TeX approach to allow us to store everything safely). – Joseph Wright♦ Aug 19 '10 at 19:08

Expanding on the LaTeX3 side of things, I guess this is a place where I'd expect to provide a different input syntax to LaTeX2e. As the question points out, the current implementation uses the token register syntax

\everyfoo{tokens}


which overwrites whatever was in \everyfoo unless you do

\everyfoo\expandafter{\the\everyfoo tokens}


which is not exactly obvious. So a better approach would be something like the current \AtBeginDocument, something like:

\AtEveryFoo{tokens}


where this is additive. The implementation of such an approach would be easy enough.

Proof of concept A short piece of concept code that might be an approach for LaTeX3. The idea here is that the TeX primitive is only used by the kernel, with two separate hooks provided. First, settings that the kernel needs are applied, then ones from programmers and finally those from the user. In the demo I've only considered the \everymath concept. I've provided an interface for the user to both add to and clear the list, plus an alias to the LaTeX2e \everymath interface. (In a pure LaTeX3 approach I'd drop this TeX-like interface, or at least mark it depreciated.)

\documentclass{article}
\usepackage{xparse}
\ExplSyntaxOn
\makeatletter
\tl_new:N \l_every_math_programmers_tl
\tl_new:N \l_every_math_users_tl
\toks_set:Nn \tex_everymath:D {
\check@mathfonts % LaTeX2e's requirement
\tl_use:N \l_every_math_programmers_tl
\tl_use:N \l_every_math_users_tl
}
\NewDocumentCommand \InsertForEvery { m +m } {
\tl_put_right:cn { l_every_ #1 _users_tl } {#2}
}
\NewDocumentCommand \ClearForEvery { m } {
\tl_clear:c { l_every_ #1 _users_tl }
}
\RenewDocumentCommand \everymath { +m } { \InsertForEvery { math } {#1} }
\ExplSyntaxOff
\begin{document}

\InsertForEvery{math}{\mathrm{Look!\,}}
$$y = mx + c$$

\ClearForEvery{math}
$$y = mx + c$$

\end{document}


I should add that a real implementation would need error-checking and so on.

-
This would be an improvement. Both Elisp hooks and the luatexbase-mcb.sty Patrick mentioned go beyond this and allow you to both register and deregister handlers - neither being done in Tex, though. You'd need a syntax like \AtEveryFooDo{\handle}{handler tokens} and \AtEveryFooDont{\handle}. – Charles Stewart Aug 21 '10 at 7:41
I take it that \AtEveryFooDont{\handle} would be equivalent to the current \everyfoo{}? (I can write up a properly written-up piece of proof-of-concept code if you like. Might be one for the LaTeX-L list, as it could be considered a bit specialist for here at the moment.) – Joseph Wright Aug 21 '10 at 7:59
I enjoyed taking the time to get a feel for how Latex3 is to be coded against. So your code has three handlers: the baked in Latex2e requirement, a math_programmers layer, which I guess is for classes, and a math_users layer, which gives authors a chance to hack. The syntax I suggested was meant to allow for arbitrarily many layers, each layer's handle being tagged by a handle, allowing the layer to be reset. Your code gives clear precedence between the layers, a free for all on my suggestion, but it doesn't help package writers avoid stepping on each others' toes. – Charles Stewart Aug 21 '10 at 20:53
I did say 'proof of concept'. For programmers, you might provide a specific function or document that you should use \tl_put_right:Nn to add to \l_every_math_programmers_tl. Ultimately you can always mess things up as a programmer :-) – Joseph Wright Aug 21 '10 at 21:15
(Just to note that \tl_put_right:Nn is essentially the same as \tl_put_right:cn but expects a control sequence rather than a csname as the frist argument.) – Joseph Wright Aug 21 '10 at 21:31