# Backslashes in macros defined in lua code

TeX file:

\directlua{require "foo"}
\show\foo
\bye


and the contents of foo.lua:

token.set_macro("foo","\\a\\b\\u\\z")


Here is the result:

> \foo=macro:
l.2 \show\foo


Why do some backslashes turn into BAD while others work like expected?

I think it might be a bug, but ask here first in case I have a fundamental misunderstanding.

• You get BAD as \a is not defined in plain, while \b is. If you add \def\a{blub} it changes. This looks like a bug to me (and I have no idea how to get \\ working). Dec 10 '19 at 16:05
• @UlrikeFischer Actually the requirement is slightly weaker: The command sequences do not have to be defined, but they must have appeared at some point. E.g. adding \def\gobble#1{}\gobble{\a\z} would "fix" it too. That's basically what the LuaTeX manual is talking about in "10.6.6 Nota bene". Dec 10 '19 at 16:22
• @MarcelKrüger yes, I got this. But what about commands like \\  or \!? Whatever I do they end up as BAD!. Dec 10 '19 at 16:30
• @UlrikeFischer It seems like the parser has a problem with non-letter control sequences. Dec 10 '19 at 17:11

The token library pokes into tex internals with almost no checks.

if you do

\def\wibble{\a\z}

\directlua{require "foo"}
\show\foo
\bye


then you get

> \foo=macro:
->\a \b \u \z .


But if \a hasn't been seen by tex at all there is no hash table entry related to \a and nothing to set as the token in the replacement text for \foo

• What can one do if one want \\  in the token list? In latex it is defined, but it still gives BAD when used in the set_macro argument. Dec 10 '19 at 16:22
• @UlrikeFischer Specifying luatexbase.registernumber("CatcodeTableOther") as you showed me in chat does put backslash tokens (with catcode other, I am sure) in the list. I wanted control sequences, however … Dec 10 '19 at 16:25
• Thanks for the explanation. It appears, however, that I also get BAD tokens in lualatex if I include "\\\\", which is what I originally wanted to do. And that makes no sense, as \\  is known to latex. (I'll experiment some more.) Dec 10 '19 at 16:28
• @HaraldHanche-Olsen you might need to ask Hans... (I couldn't get it to work) Dec 10 '19 at 16:30
• Okay, I might. But now I have to go solve the original problem by moving it out of the lua realm into latex. Dec 10 '19 at 16:39

Too long for a comment.

It is unlikely that this will be made working, see this comment in the source of lnewtokenlib.c:

            /* this is a relating simple converter; if more is needed one can just use     */
/* tex.print with a regular \def or \gdef and feed the string into the regular */
/* scanner;                                                                    */

• Thanks! Yes, that approach did occur to me last night. One could wish there were some warnings in the manual, though. Dec 11 '19 at 8:34