Test whether token list is empty

Question

Is it possible to test whether a token list register is empty without expanding it?

Answer 1

Would something like this work?

\def\ifempty#1{%
\edef\ContentsOfList{\the#1}%
\edef\CompareTo{}%
\ifx\ContentsOfList\CompareTo}

You could then use it like this:

\toks0={\blah\bloh a b c}
\toks2={}

\ifempty{\toks0} empty\else nonempty\fi

\ifempty{\toks2} empty\else nonempty\fi

Answer 2

At time of writing the other TeX based answers on this page are flawed in that they hide a conditional \ifx inside a macro but still use \else/\fi at the "top" level. This will mean that things break unexpectedly when used inside other conditionals.

The LaTeX3 programming language expl3 contains a module for doing stuff with token registers:

\usepackage{expl3}
...
\ExplSyntaxOn
\toks_if_empty:NTF \mytoks {true} {false}
\ExplSyntaxOff

It essentially does internally what the other answers here are suggesting, but it uses expansion to grab its arguments so the branching is robust (and you don't have \fi lying around to get in your way).

Update: So what does this approach do that is superior to other methods? Consider the style of solution first offered in answer to this question:

\def\IfEmpty#1{%
  \edef\1{\the#1}
  \ifx\1\empty
}
...
\IfEmpty\foo T\else F\fi

This doesn't behave nicely when nested, because TeX scans ahead when discarding unfollowed branches of a conditional. Consider

\ifx\bar\baz
  \IfEmpty\foo T\else F\fi % <- uh oh
\else
  E
\fi

If \bar = \baz, then the second branch is discarded and the first branch is executed. So far so good. If \bar ≠ \baz, then the first branch is discarded by reading ahead until the first unmatched \else — and this is the one in the line labelled "uh oh" above. So you could collapse the expansion of the above snippet in this case to:

\iffalse\else F\fi % <- uh oh
\else
  E
\fi

and hence the cause of the ‘Extra \else’ error message in this case.

So this form for conditionals doesn't work so well. Next try. You can also write this style of code like this:

\def\IfEmpty#1#2#3{%
  \edef\1{\the#1}
  \ifx\1\empty
    #2%
  \else
    #3%
  \fi
}

This avoids the problems of nesting as in the previous trial solution, but it's prone to another problem: #2 and #3 have trailing material behind them, namely \else and \fi. This is a problem if you want to write something like

\def\processfoo#1{...something with #1...}
\IfEmpty\foo{\error}{\processfoo} {arg}

because the #1 passed to \processfoo will be \fi instead of the desired {arg}. The conditional in this case is better written as

\def\IfEmpty#1#2#3{%
  \edef\1{\the#1}
  \ifx\1\empty
    \expandafter\@firstoftwo
  \else
    \expandafter\@secondoftwo
  \fi
  {#2}{#3}
}

so overcome this problem. This is how expl3 conditionals work, and it's why we're writing TF at the end of all their names to indicate "true" and "false" branches. (Or just T or just F if you only want one of them.)

Incidentally, there are expandable tests for checking for emptiness, which is why I suggest using the expl3 approach for this test. Expandability is not always required, of course, but code that is fully expandable tends to be more reliable and it's always nice to have for cases such as

\typeout{ \toks_if_empty:NT \foo {Warning:~\string\foo\space is~ empty} }

Answer 3

With LuaTeX you can do this:

\directlua{
if string.len(tex.toks["headline"]) > 0 then
    print("not emtpy")
end
}

But since I never use token list: don't use this code to build nuclear plants.

Answer 4

Ulrich Diez regularly posts on comp.text.tex some code along the following lines (slightly modified by me for toks):

\newcommand\@ifempty@toks[1]{%
  \ifcat\relax\detokenize\expandafter{\the#1}\relax
    \expandafter\@firstoftwo
  \else
    \expandafter\@secondoftwo
  \fi
}

Or, without using e-TeX,

\newcommand\@ifempty@toks[1]{%
  \ifcase\iffalse{{{\fi\expandafter\@ifempty@@toks\the#1}1}1}0
    \expandafter\@firstoftwo
  \else
    \expandafter\@secondoftwo
  \fi}
\newcommand{\@ifempty@@toks}
  {\expandafter\@gobble\expandafter{\expandafter{%
        \ifcase`}\expandafter}\expandafter\fi\string}

Answer 5

Similar to Jan Hlavacek's answer, but expandable:

\def\certainlynotintoks{\certainlynotintoks}
\def\iftoksempty#1{\expandafter\ifx\expandafter
    \certainlynotintoks\the#1 \certainlynotintoks}

\toks0{\undefined will not be expanded}
\toks2{}
\iftoksempty{\toks0} empty\else not empty\fi \par
\iftoksempty{\toks2} empty\else not empty\fi

This requires that \certainlynotintoks does not appear as the first token in the token register. Both this and Jan's answer do expand the token register, but only once. I don't believe there's a way to avoid that.

Edit: Taking Will's (absolutely correct) comment into consideration, change the definition of \iftoksempty to

\makeatletter
\def\iftoksempty#1{\expandafter\ifx\expandafter
    \certainlynotintoks\the#1 \certainlynotintoks \expandafter\@firstoftwo
    \else \expandafter\@secondoftwo\fi}

Then you can use \iftoksempty{\toks0}{empty}{not empty}.

Answer 6

A rather retro way of avoiding the problems Will talks about is to use flags. Define:

\newif\iftest 
\def\testtoksempty#1{\edef\1{\the#1}
  \ifx \1\empty \testtrue % from plain.tex: \def\empty{}
  \else\testfalse \fi}

Then use:

\testtoksempty\toksreg
\iftest toksreg is empty \else toksreg's got something \fi

I tend to avoid any kind of overlap between \if conditional and macro expansion because of my incomplete, fearful grasp of the issues Will nicely explained (and which I now understand a bit better).
This is clunky, ASM-like way to code, and the use of a fixed test flag means that Boolean expressions are going to be fiddly: you may be driven to multiple, nearly identical macros - \tessttoksempty, \tesssttoksempty, etc.
But it's robust and easy to debug.