How to know whether or not we need \expandafter?

Question

The following is quoted from Peter Grill's answer on question LaTeX dynamic macro definition.

The \expandafter is requried as the \newcommand needs to be dealt with after the \csname.

As a newbie in TeX programming, how to know whether or not I need \expandafter?

Answer 1

Let's look at three examples from latex.ltx (the LaTeX kernel).

\@namedef

\def\@namedef#1{\expandafter\def\csname #1\endcsname}

This is entirely analogous to the \expandafter\newcommand that's the main object of the question. With \@namedef{foo} we get

\expandafter\def\csname foo\endcsname

and the control sequence \foo is built before \def comes into action: the expansion of \csname is the symbolic token whose name is what comes after it up to the matching \endcsname (with complete expansion, but analyzing this would take too far).

\loop

\def\loop#1\repeat{\def\iterate{#1\relax\expandafter\iterate\fi}%
  \iterate \let\iterate\relax}

We use this in the form

\loop
  <code A>
\ifz<condition>
  <code B>
\repeat

where \ifz is one of the TeX conditionals and <condition> is the test that, if true, will make the loop continue. This becomes

\def\iterate{<code A>\ifz<condition><code B>\relax\expandafter\iterate\fi}
\iterate \let\iterate\relax

The first line is an assignment: it's executed and removed from the input token list, leaving

\iterate \let\iterate\relax

Now \iterate is expanded, so we get

<code A>\ifz<condition><code B>\relax\expandafter\iterate\fi
\let\iterate\relax

TeX expands/executes <code A> and then evaluates the conditional. If it's false, everything up to \fi is skipped, so what remains is

\fi\let\iterate\relax

The expansion of \fi is empty; the next assignment is executed, ending the loop. If it's true, then we obtain

<code B>\relax\expandafter\iterate\fi
\let\iterate\relax

Now <code B> is expanded/executed; \relax does nothing and we have the magic

\expandafter\iterate\fi
\let\iterate\relax

Here \expandafter expands \fi, which has empty expansion: so TeX finds

\iterate \let\iterate\relax

and restarts. If the \expandafter were omitted, the \fi tokens would accumulate, possibly filling the memory on long loops.

\@onlypreamble

Several commands in the kernel are defined following the scheme

\def\@foo{...}
\@onlypreamble\@foo

which means that \@foo cannot be used past \begin{document}, where it would trigger the "Can be used only in the preamble" error. Let's see what \@onlypreamble does:

\def\@preamblecmds{}
\def\@onlypreamble#1{%
  \expandafter\gdef\expandafter\@preamblecmds\expandafter{%
       \@preamblecmds\do#1}}

First \@preamblecmds is initialized to empty. If we say

\@onlypreamble\@foo

we get

\expandafter\gdef\expandafter\@preamblecmds\expandafter{\@preamblecmds\do\@foo}}

The first \expandafter expands the second, which expands the third; the last one, in turn, expands \@preamblecmds. If this was the first call of the macro \@onlypreamble, we'd get

\gdef\@preamblecmds{\do\@foo}

To make the problem more interesting, let's call now

\@onlypreamble\@bar

Again we get

\expandafter\gdef\expandafter\@preamblecmds\expandafter{\@preamblecmds\do\@bar}}

but now the expansion of \@preamblecmds is \do\@foo: so finally we get

\gdef\@preamblecmds{\do\@foo\do\@bar}

and so on for every call of \@preamblecmds. When LaTeX works on \begin{document} it says

\gdef\do#1{\global\let #1\@notprerr}\@preamblecmds

The \do command is defined to make its argument equivalent to \@notprerr (that triggers the above mentioned error); then \@preamblecmds is expanded, so that

\do\@foo\do\@bar

(and many other tokens of the same type) will be executed.

In \@onlypreamble the \expandafter is needed to get the expansion of \@preamblecmds in the replacement text before redefining \@preamblecmds: doing

\gdef\@preamblecmds{\@preamblecmds\do\@foo}

would lead to a disaster, because the final replacement text of \@preamblecmds would be

\@preamblecmds\do\@foo\do\@bar

(with many other \do... combinations). And when the expansion of \@preamblecmds is performed, TeX would find \@preamblecmds and expand it, finding \@preamblecmds which it would expand, finding… Infinite loop. Oops!

\@ifundefined

The LaTeX kernel provides \@ifundefined that checks whether the control sequence having as name its argument is defined or not (actually, undefined means either really undefined or equivalent to \relax).

\def\@ifundefined#1{%
  \expandafter\ifx\csname#1\endcsname\relax
    \expandafter\@firstoftwo
  \else
    \expandafter\@secondoftwo
  \fi}

The \expandafter\@firstoftwo and \expandafter\@secondoftwo are explained elsewhere on the site; here the \expandafter application is similar to the first one: the usage is \@ifundefined{foo}{true}{false} and the control sequence token \foo is built before \ifx comes into action.

What I want to underline here is that generally using \expandafter\ifx is wrong: for instance, after

\def\foo{foo}

the test

\expandafter\ifx\foo\foo

would return false. So don't use \expandafter unless you know it's necessary.

Answer 2

The short answer to this is that you need \expandafter whenever you need something expanded before the previous token (helpful, I know!). In Peter's example, the code was

\expandafter\newcommand \csname name#1\endcsname{\emph{#2}}

(where #1 and #2 were arguments to some surrounding macro). The reason you want \expandafter here is because the entire \csname construction is not itself a control sequence, and yet \newcommand expects the first thing after it to be a control sequence that will name the new command. Without \expandafter, it will grab \csname itself as the name of the new command, and then it will error out because it is not actually undefined.

The philosophy here is that the \csname is "preparation" for using \newcommand, and the \newcommand itself cannot be expanded meaningfully until the following input has been suitably prepared. The purpose of \expandafter is to reserve it while acting on some subsequent directive to transform the actual input into the desired one.

Answer 3

Not to compete with @egreg's answer but to answer your comment on when not to use \expandafter.

consider

 \def\foo{hello}

which defines \foo to be hello. So that is safe and good.

\expandafter\def\foo{hello}

may not be wrong (but it may be, and often is).

If it's the common case that \foo is not yet defined then you will get the error

! Undefined control sequence.
l.1   \expandafter\def\foo
                          {hello}

As the expandafter tries to expand \foo before the definition.

If on the other hand \foo already had the definition \def\foo{world} then

\expandafter\def\foo{hello}

would give the error

! Missing control sequence inserted.
<inserted text> 
                \inaccessible 
<to be read again> 
                   w
\foo ->w
        orld
l.3   \expandafter\def\foo
                          {hello}

as it is equivalent to \def world{hello}

and finally if \foo already was defined by \def\foo{\bar} then you would get no error but it would define \bar not \foo which is good if that's what is intended, but may not be what is intended if you just add \expandafter "for luck".

\def\foo{\bar}

\expandafter\def\foo{hello}

\show\foo
\show\bar

produces

> \foo=macro:
->\bar .
l.5 \show\foo

? 
> \bar=macro:
->hello.
l.6 \show\bar