31

I often see code of the following form in package implementations (this example is from the LaTeX3 sources):

\begingroup\expandafter\expandafter\expandafter\endgroup
\expandafter\ifx\csname directlua\endcsname\relax
\else
  …
\fi

The first line, containing three \expandafters, confuses me. I can only follow this far:

  • \begingroup starts a group
  • The chain of \expandafters causes \csname directlua\endcsname to be converted to a control sequence
  • After this point, the state is that we're in a group and \expandafter\endgroup\ifx[directlua]\relax… remain to be examined by the macro processor ([directlua] denotes a control sequence)
  • Now the last \expandafter is processed and \ifx is expanded, then \endgroup ends the group. The TeXbook says this on the topic:

    When an \if… is expanded, TeX reads ahead as far as necessary to determine whether the condition is true or false; and if false, it skips ahead (keeping track of \if…\fi nesting) until finding the \else, \or, or \fi that ends the skipped text. Similarly, when \else, \or, or \fi is expanded, TeX reads to the end of any text that ought to be skipped. The “expansion” of a conditional is empty.

    This would suggest that the arguments to \if… are evaluated inside the group. But what about code inside the conditional's branches?

If the purpose of the code in question is indeed to evaluate the \if… inside a group, why is it better than just inserting the conditional between \begingroup and \endgroup?

3 Answers 3

34

Let's look step by step

\begingroup\expandafter\expandafter\expandafter\endgroup
\expandafter\ifx\csname directlua\endcsname\relax
  A
\else
  B
\fi

This becomes

(\begingroup)\expandafter\endgroup
\ifx\directlua\relax
  A
\else
  B
\fi

The \begingroup has already been digested, so I leave it in parentheses just to remember a group has been opened. Another step, now, where we have to distinguish between cases.

Case 1: \directlua is not defined, so the token produced by\csname directlua\endcsname is equivalent to \relax.

(\begingroup)\endgroup A\else B\fi

Now \endgroup is digested and this removes the assignment of the meaning \relax to \directlua. A is examined, the expansion of \else B\fi is empty.

Case 2: \directlua is defined.

(\begingroup)\endgroup B\fi

Again \endgroup is digested, but does not restore anything. The expansion of \fi is empty.

Why not doing this inside a group? The key point is that at the end \directlua is not defined if it wasn't at the start of the process. The same would be true if the code is

\begingroup\expandafter\ifx\csname directlua\endcsname\relax A\else B\fi\endgroup

However the purpose of A and B is doing some assignments. In this case A would probably be \luatexfalse, after having said before \newif\ifluatex, and B would be \luatextrue. The triple \expandafter inside the group dispenses from a global assignment, following the good practice that assignments to a variable should be always global or always local (so long as it's possible). Of course in this case a global assignment would not be that important, in other cases it might have consequences on the save stack.

The suggested alternative

{\expandafter}\expandafter\ifx\csname directlua\endcsname\undefined
  A
\else
  B
\fi

(with \undefined, not \relax) is less attractive, because it relies on a certain token to be undefined. One could object that the code we're analyzing assumes \relax has its primitive meaning, but some assumptions need to be made.

If e-TeX can be assumed, the simpler test

\ifdefined\directlua
  A
\else
  B
\fi

is even fully expandable.

12
  • 2
    The key point here is to avoid defining \directlua at all outside of the test. This approach keeps the hash table clean.
    – Joseph Wright
    Commented Jul 29, 2011 at 8:47
  • 1
    @Joseph: I think I said something along those lines on comp.text.tex, and got corrected. Control sequences are put in the hash table as soon as they are seen in any kind of token list, and nothing is ever removed from the hash table. The eq table, however, is indeed changed by assignments, and with this contruction is left unchanged. I think that the eTeX manual mentions that \ifdefined and \ifcsname don't add to the hash table. --- all this said, I know very little on that topic. Commented Jul 30, 2011 at 1:55
  • 2
    @Bruno Yes, with \csname...\endcsname the hash table is always touched. The point is just not to have \directlua in the equivalence table, which is the most that can be done in pure TeX.
    – egreg
    Commented Jul 30, 2011 at 8:12
  • 1
    @einpoklum I'm afraid it doesn't. It should be {\expandafter}\expandafter\ifx\csname xxx\endcsname\undefined a\else b\fi and relies on a certain token to be undefined.
    – egreg
    Commented Jan 12, 2016 at 0:11
  • 1
    @user202729 Relying on \relax being the primitive is safer. Redefining \relax would badly break everything. Of course, the test would fail if the user has previously defined \directlua to be \relax and that’s why e-TeX has \ifdefined and \ifcsname.
    – egreg
    Commented Jul 13, 2022 at 7:41
18

The reason here is that \csname ...\endcsname will define ... as a macro equal to \relax should it not already exists. This feature is used with \ifx which compares it to \relax. This test is true if ... wasn't defined before (or was \let to \relax).

However, it isn't good practice to define macros even to \relax just for testing their existence. e-TeX provides \ifcsname ...\endcsname for this. Without e-TeX the a group can be used together with \expandafter to process both the \csname and the \ifx inside it to keep the macro definition local.

Note that when an \if... is true TeX simply goes on with processing the following tokens. It remembers looking for the closing \fi which is simply removed or an \else branch which should be skipped. If the test is false everything till \else is immediately skipped and TeX again remembers to look for an closing \fi. Therefore all the \expandafter trickery works very well. The \ifx is expanded and TeX already has chosen which branch it will execute. Then the \endgroup is insert and that branch is executed.

The benefit for wrapping the whole expression inside a groups is very clear: the actual content can define/change local settings!

Note that that in TeX if-statements and groups are independent (which is not the case in almost any other programming language). You can therefore also write the following to keep the \csname ...\endcsname statement local:

\begingroup
\expandafter\ifx\csname directlua\endcsname\relax
    \endgroup
    …
\else
    \endgroup
    …
\fi

Only one \endgroup is ever executed here.

4
  • 3
    There's also \ifdefined, so \ifdefined\directlua is more direct.
    – egreg
    Commented Jul 29, 2011 at 9:05
  • 1
    Missing \expandafter ? Commented Jul 29, 2011 at 11:33
  • @Martin Scharrer: it is: \expandafter\ifx\csname directlua\endcsname\relax (or \ifx\directlua\undefined). Commented Jul 30, 2011 at 1:57
  • @Bruno: Yes of course. I was locking at the \begingroup instead :-) Commented Jul 30, 2011 at 9:36
2

Some additional notes/demonstration about the "hash table" and the "equivalence table" etc.. as mentioned in some comments above.

Hash table

The hash table (or hash_extra, it's complicated) maps each control sequence name to a number (for fast manipulation of tokens, etc.).

In LuaTeX it's possible to check whether a token is in the hash table using token.get_next() or token.create().

In the example below, initially \undefineda is not in the hash table, but after it's "seen" (even if it's never defined) it's in the hash table. (also you can see that \ifdefined alone does not create new hash table entry even if the token is "read" by TeX)

%! TEX program = lualatex
\documentclass{article}
\usepackage{luacode}
\begin{document}

\begin{luacode*}
print("\n\n\n\n")

function f()
    print("f: csname =", token.get_next().csname)
end
\end{luacode*}

\ExplSyntaxOn

\directlua{f()} \undefineda  % here the output is empty, so \undefineda is not in the hash table

\ifdefined\undefineda \directlua{print "defined"} \else \directlua{print "undefined"} \fi
\directlua{f()} \undefineda  % special case: \ifdefined (alone) does not put the token into the hash table

\iffalse \undefineda \fi
\directlua{f()} \undefineda  % special case 2: \iffalse ... \fi does not put the token into the hash table

\use_none:n {\undefineda}    % the token appears, but \use_none:n "throws it away" immediately
\directlua{f()} \undefineda  % now undefineda is in the hash table

\ExplSyntaxOff

\begin{luacode*}
print("\n\n\n\n")
\end{luacode*}

\end{document}

Relevant output:

f: csname =
undefined
f: csname =
f: csname =
f: csname =     undefineda

Equivalence table

The "equivalence table" is an implementation details of TeX, in texdoc tex it's called "The table of equivalents", part 17, section 220; and in the source code it's the variable eqtb.

As the documentation states, it holds the current "equivalents" of things.

Note that (implementation details) this table is not called "macro definition table", as it holds more than just macro definitions, as

  • control sequences/active characters can be defined to be something other than macro (for example:

    • \chardef \% = `% which makes \% equivalent (not "expands to"! It remains unexpandable) to \char `% when executed;
    • \let \c_math_toggle_token = $;
    • etc.)
  • the value of e.g. the baselineskip, are also put in this table.

In short, "not to have \directlua in the equivalence table" is just a complicated way of saying "undefine \directlua", if I understood correctly.

Note on \let \directlua \undefined and the save stack

In this particular case, if you execute \begingroup, do \csname directlua \endcsname then execute \endgroup it's similar to if you do \let \directlua \undefined after a csname-endcsname -- the only difference (as far as I know) is in the "save stack" i.e. if you do \let \directlua \undefined alone it will leave an item in the save stack:

%! TEX program = lualatex
\documentclass{article}
\begin{document}

\ExplSyntaxOn

\begingroup

\tracingrestores=1~

\wlog{========~first~example}

\begingroup


\begingroup \expandafter \endgroup \expandafter \use_none:n \csname testtest \endcsname

\wlog{//there's~a~restore~above~because~of~the~endgroup,~but~no~restoration~below}

\endgroup

\wlog{========~second~example}

\begingroup

\expandafter \use_none:n \csname testtest \endcsname
\let \testtest \undefined

\wlog{//note~that~there's~still~a~restoration~below}

\endgroup

\endgroup

\ExplSyntaxOff

\end{document}

The relevant output of this program is:

======== first example
{restoring \testtest=undefined}
//there's a restore above because of the endgroup, but no restoration below
======== second example
//note that there's still a restoration below
{restoring \testtest=undefined}
{restoring \tracingrestores=0}

side note, \use_none:n is abused to gobble a single token here, but not a big problem

Addition

As shown in the comments under egreg's answer I was a bit puzzled why they don't just do \ifx\directlua\undefined.

My conjecture is that, the LaTeX3 team knows that \csname...\endcsname (and various other constructs using that) automatically defines the control sequence to be \relax when it was originally undefined, think that's error-prone, and decide to hide the behavior from the user and treat \relax and undefined the same way.

In particular, in this case, the code will consider \directlua "nonexistent" even if the user accidentally does (something that internally does) \csname directlua \endcsname somewhere before. Which I think is rather unlikely.

Compare this with how \cs_if_exist:NTF or \cs_if_exist:cTF works. While the \ifdefined and \ifcsname primitives are true on \relax and false on undefined, both of the commands above are hard coded to be false on \relax.

(worth noting that, unlike many other commands that take c-type argument which are implemented in terms of \exp_args:Nc or \::c, this command does not implicitly define the control sequence to \relax when it was undefined because it's special-case implemented using \ifcsname; nevertheless this behavior (as well as the fact that most-but-not-all c-type argument implicitly converts undefined to \relax, or in its LaTeX3 name, \scan_stop:) is as far as I can see never documented anywhere in interface3.pdf)

That having said, my opinion (not really related to this question however) is that this can never be completely abstracted away from the user, as an undefined control sequence will create an error when expanded/executed while a \relax control sequence will not.

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .