# Test whether token list is empty

Is it possible to test whether a token list register is empty without expanding it?

-

Would something like this work?

\def\ifempty#1{%
\edef\ContentsOfList{\the#1}%
\edef\CompareTo{}%
\ifx\ContentsOfList\CompareTo}


You could then use it like this:

\toks0={\blah\bloh a b c}
\toks2={}

\ifempty{\toks0} empty\else nonempty\fi

\ifempty{\toks2} empty\else nonempty\fi

-
And \the inside \edef inserts the token list without expanding it. Yes, it works, I should have come up with that by myself. –  Charles Stewart Sep 9 '10 at 17:59
This does have the disadvantage that it isn't expandable. I.e., you couldn't use it like \edef\foo{\ifempty{\toks0} \foo\else\bar} –  TH. Sep 9 '10 at 18:02
@TH: To solve this without assignment do something like \expandafter\helper\the#1 \sentinel\anothersentinel and then make \helper do whatever it takes, namely pick something out of the expanded #1. This probably means \helper has to expand to \expandafter\anotherhelper\string, and now it's getting complicated. And perhaps can't be reliably done, however clever we are with sentinel. –  Jonathan Fine Sep 9 '10 at 20:50
@Jan: We expect \ifx\a\b \ifempty{#1} \fi \fi to expand to nothing if the \ifx is false. But your definition does not have this property. –  Jonathan Fine Sep 9 '10 at 20:56
Continuing JF's last comment, I do not recommend defining conditionals in this way. It will blow up in your face when you least expect it, usually when you try and nest this conditional within another. –  Will Robertson Sep 9 '10 at 23:03

With LuaTeX you can do this:

\directlua{
print("not emtpy")
end
}


But since I never use token list: don't use this code to build nuclear plants.

-
I'm curious why this got a down vote, does it not work? –  TH. Sep 10 '10 at 5:43
If it doesn't work, I'd like to get a comment about what is wrong. My tests were all okay. –  topskip Sep 10 '10 at 7:02
I was rash. Using Lua to solve a TeX problem seems a bit batty to me (as in, why not skip using toks entirely and just do everything in Lua?). But at least the Lua code here is nice and straightforward, unlike the TeX code required. –  Will Robertson Sep 10 '10 at 7:02
We definitely need more answers of this kind. Even if it is solvable in TeX, a solution that mere humans can understand is highly desirable. (heck, even I understood it, unlike the other black expansion magic). –  Khaled Hosny Sep 10 '10 at 7:20

Similar to Jan Hlavacek's answer, but expandable:

\def\certainlynotintoks{\certainlynotintoks}
\def\iftoksempty#1{\expandafter\ifx\expandafter
\certainlynotintoks\the#1 \certainlynotintoks}

\toks0{\undefined will not be expanded}
\toks2{}
\iftoksempty{\toks0} empty\else not empty\fi \par
\iftoksempty{\toks2} empty\else not empty\fi


This requires that \certainlynotintoks does not appear as the first token in the token register. Both this and Jan's answer do expand the token register, but only once. I don't believe there's a way to avoid that.

Edit: Taking Will's (absolutely correct) comment into consideration, change the definition of \iftoksempty to

\makeatletter
\def\iftoksempty#1{\expandafter\ifx\expandafter
\certainlynotintoks\the#1 \certainlynotintoks \expandafter\@firstoftwo
\else \expandafter\@secondoftwo\fi}


Then you can use \iftoksempty{\toks0}{empty}{not empty}.

-
In fact, \the\toksreg simply inserts the token list without any expansion at all. –  Charles Stewart Sep 9 '10 at 18:04
Yes, that's what I meant by expansion. \the\toksreg expands to whatever that token register contains. –  TH. Sep 9 '10 at 18:06
Again, hiding \ifx in a macro has dire consequences. It breaks nesting because TeX can't match its trailing \else...\fi with a leading \iftrue/\iffalse-equivalent conditional. –  Will Robertson Sep 9 '10 at 23:07
Yes, you're correct, of course. –  TH. Sep 10 '10 at 4:10

At time of writing the other TeX based answers on this page are flawed in that they hide a conditional \ifx inside a macro but still use \else/\fi at the "top" level. This will mean that things break unexpectedly when used inside other conditionals.

The LaTeX3 programming language expl3 contains a module for doing stuff with token registers:

\usepackage{expl3}
...
\ExplSyntaxOn
\toks_if_empty:NTF \mytoks {true} {false}
\ExplSyntaxOff


It essentially does internally what the other answers here are suggesting, but it uses expansion to grab its arguments so the branching is robust (and you don't have \fi lying around to get in your way).

Update: So what does this approach do that is superior to other methods? Consider the style of solution first offered in answer to this question:

\def\IfEmpty#1{%
\edef\1{\the#1}
\ifx\1\empty
}
...
\IfEmpty\foo T\else F\fi


This doesn't behave nicely when nested, because TeX scans ahead when discarding unfollowed branches of a conditional. Consider

\ifx\bar\baz
\IfEmpty\foo T\else F\fi % <- uh oh
\else
E
\fi


If \bar = \baz, then the second branch is discarded and the first branch is executed. So far so good. If \bar\baz, then the first branch is discarded by reading ahead until the first unmatched \else — and this is the one in the line labelled "uh oh" above. So you could collapse the expansion of the above snippet in this case to:

\iffalse\else F\fi % <- uh oh
\else
E
\fi


and hence the cause of the ‘Extra \else’ error message in this case.

So this form for conditionals doesn't work so well. Next try. You can also write this style of code like this:

\def\IfEmpty#1#2#3{%
\edef\1{\the#1}
\ifx\1\empty
#2%
\else
#3%
\fi
}


This avoids the problems of nesting as in the previous trial solution, but it's prone to another problem: #2 and #3 have trailing material behind them, namely \else and \fi. This is a problem if you want to write something like

\def\processfoo#1{...something with #1...}
\IfEmpty\foo{\error}{\processfoo} {arg}


because the #1 passed to \processfoo will be \fi instead of the desired {arg}. The conditional in this case is better written as

\def\IfEmpty#1#2#3{%
\edef\1{\the#1}
\ifx\1\empty
\expandafter\@firstoftwo
\else
\expandafter\@secondoftwo
\fi
{#2}{#3}
}


so overcome this problem. This is how expl3 conditionals work, and it's why we're writing TF at the end of all their names to indicate "true" and "false" branches. (Or just T or just F if you only want one of them.)

Incidentally, there are expandable tests for checking for emptiness, which is why I suggest using the expl3 approach for this test. Expandability is not always required, of course, but code that is fully expandable tends to be more reliable and it's always nice to have for cases such as

\typeout{ \toks_if_empty:NT \foo {Warning:~\string\foo\space is~ empty} }

-
Did my updated answer fix the problems? If so, is there a way to do it that doesn't involves a sentinel? –  TH. Sep 10 '10 at 5:42
Yes, it did. That's essentially the definition of the expl3 version, although the "emptiness" test in expl3 is slightly more robust. I'm not exactly sure what you mean specifically and exactly by "sentinel", but I think the answer is no. –  Will Robertson Sep 10 '10 at 7:06
I like such examples in expl3 syntax (and also the one in luatex) very much. It help to get used to their look and feel. E.g. I was quite confused about the \space behind \foo. The idea that \foo swallows a ~ like it swallows normally a space is - well - curious. –  Ulrike Fischer Sep 10 '10 at 8:11
@Ulrike: inside \ExplSyntaxOn, tilde has the catcode of a space because space has the catcode of "ignored". If that's not to your liking, you can use \ExplSyntaxNamesOn, which is the same as \ExplSyntaxOn but leaves spaces as-is. We should perhaps be promoting this more for use in general package code, since I guess the space-ignore-rule is probably somewhat inconvenient for integrating some expl3 code into regular LaTeX code. –  Will Robertson Sep 10 '10 at 10:33
@TH. There is a way which doesn't involve a sentinel (sort of), using \detokenize. It begins like \expandafter\ifx\expandafter\notsentinel\detokenize\expandafter{\the#1}\notsent‌​inel. Here, \notsentinel is used as a sentinel, but thanks to \detokenize there is not risk that it accidentally matches the contents of the toks register tested. –  mpg Oct 28 '10 at 13:44

A rather retro way of avoiding the problems Will talks about is to use flags. Define:

\newif\iftest
\def\testtoksempty#1{\edef\1{\the#1}
\ifx \1\empty \testtrue % from plain.tex: \def\empty{}
\else\testfalse \fi}


Then use:

\testtoksempty\toksreg
\iftest toksreg is empty \else toksreg's got something \fi


I tend to avoid any kind of overlap between \if conditional and macro expansion because of my incomplete, fearful grasp of the issues Will nicely explained (and which I now understand a bit better).
This is clunky, ASM-like way to code, and the use of a fixed test flag means that Boolean expressions are going to be fiddly: you may be driven to multiple, nearly identical macros - \tessttoksempty, \tesssttoksempty, etc.
But it's robust and easy to debug.

-

Ulrich Diez regularly posts on comp.text.tex some code along the following lines (slightly modified by me for toks):

\newcommand\@ifempty@toks[1]{%
\ifcat\relax\detokenize\expandafter{\the#1}\relax
\expandafter\@firstoftwo
\else
\expandafter\@secondoftwo
\fi
}


Or, without using e-TeX,

\newcommand\@ifempty@toks[1]{%
\ifcase\iffalse{{{\fi\expandafter\@ifempty@@toks\the#1}1}1}0
\expandafter\@firstoftwo
\else
\expandafter\@secondoftwo
\fi}
\newcommand{\@ifempty@@toks}
{\expandafter\@gobble\expandafter{\expandafter{%
\ifcase}\expandafter}\expandafter\fi\string}

-
The non e-TeX version is an excellent example of using \ifcase to 'bury' the extra braces. It is missing two \expandafter's however: one before '' and one after it (in the definition of \@ifempty@@toks). I think it would also be nice if standard \def' were used instead of \LaTeX lingo. Thanks for the idea, anyway! –  alexsh Mar 10 at 2:56
Another comment is that to make it possible to use this macro inside alignments, } should be replaced by 1\expandafter} (see The TeXbook, p. 385 for the reasons and the discussion of the balance and master counters). In any case, this is a beautiful macro! –  alexsh Mar 10 at 18:50
@alexsh The two \expandafter you suggest adding would do nothing. When TeX looks for a number after \ifcase, it reads the next two tokens, which denote the character code of }. It turns out that such a number can be followed by an optional space token, so TeX continues expanding what follows (namely \expandafter}\expandafter...) before using the number (character code of }) as the argument of \ifcase. –  Bruno Le Floch Mar 12 at 13:25
As far as I can tell without testing, your second comment is correct: to make the macro useable inside alignments, one should write \ifcase1\expandafter}\expandafter}\expandafter\fi`. –  Bruno Le Floch Mar 12 at 13:27
I stand corrected. I just assumed that <internal integer> and <character constant> both disallowed an optional trailing space. Only the former one does, however, so the expansion will continue ... –  alexsh Mar 12 at 19:52