181

I'm quite happy hacking TeX macros and cobbling together bits and pieces from different style files to suit my own ends, but I have a suspicion that my resulting hacks are not quite as elegant as they could be. In particular, with regard to when to expand and when not to expand macros.

A common occurrence for me is defining a meta-command that defines a whole slew of sub-commands. So the names of the sub-commands will contain parameters depending on the parameter passed to the meta-command, and the contents of the sub-commands will also vary a little depending on what the meta-command got. Often it won't be a direct substitution but rather a "if #1 is a do \this else do \that", but \this and \that need expansion at define time, not call time.

Here's a very simple example just involving direct substitution:

\def\cohtheory#1{
\expandafter\newcommand\expandafter{\csname #1func\endcsname}[1][*]{%
\MakeUppercase{#1}^{##1}}
}

Sometimes I worry that my commands are more complicated than they need be. For example, if I want to call a command with two arguments and the arguments expand before the command, here's how I've coded it:

\expandafter
  \expandafter
  \expandafter
  \command\expandafter
            \expandafter
            \expandafter
              {\expandafter
                \argone
                \expandafter
              }\expandafter
              {\argtwo}

So I'm looking for guidance on when and how to control expansion in defining macros. I strongly suspect that such cannot be given in a simple answer, so to make this a focussed question, let me phrase it thus:

Where's a good reference for writing TeX macros that includes advice on how to best deal with how to handle expansions?

Of course, if anyone can formulate some advice in short answer, I'd be only to happy to read it.

(Note: this was partially motivated by juannavarroperez's adaptation of my answer to this question where over 20 \expandafters got condensed down to just 1!)

  • 26
    That’s a typical example of cargo cult programming. I think I’m allowed to make this observation without being insulting since my TeX code looks exactly like this, i.e. I have the exact same problem. :-( All this to say that I was about to ask the exact same question. – Konrad Rudolph Jul 28 '10 at 10:54
  • 4
    Andrew, just because I saw your example above: I recently discovered the LaTeX2e macro \@expandtwoargs\command{<arg1>}{<arg2>} which expands the two arguments using \edef before feeding it to \command. – Martin Scharrer Feb 15 '11 at 17:38
  • 1
    @MartinScharrer That's neat! Unfortunately, there does not seem to be an equivalent for three (or more?) parameters. :/ – Raphael Nov 30 '12 at 14:18
  • 3
    @Raphael: Just look at the definition of \@expandtwoargs and define a macro for three arguments. Should not be any trouble. – Martin Scharrer Dec 1 '12 at 8:46
  • 3
    Relevant: A tutorial on \expandafter – Werner Oct 16 '15 at 23:34
193

Expansion is a complicated area of TeX programming. I'll try to explain the key primitives involved first, then try to come up with some examples.

The \expandafter primitive expands the token after the next one. So

\expandafter\def\csname an-awkward-name\endcsname

will expand \csname before \def. So after one expansion the above turns into

\def\an-awkward-name

which will then do its thing. Life becomes more complex when you want to step further ahead, and it soon becomes very hard to track what is going on.

The \edef> primitive does a full expansion of what is given as its argument (in contrast to \def, which simply stores the input). So

\def\examplea{more stuff}
\edef\exampleb{Some stuff \csname examplea\endcsname}

will expand the \csname name\endcsname to \examplea, then expand that to leave a final definition of \exampleb as 'Some stuff more stuff'.

Now, \noexpand comes in by preventing \edef from doing an expansion of the next token. So if I modify my above example to read

\def\examplea{more stuff}
\edef\exampleb{Some stuff \expandafter\noexpand\csname examplea\endcsname}

then what will happen is that the \edef will execute the \expandafter, which will turn the above effectively into

\def\examplea{more stuff}
\edef\exampleb{Some stuff \noexpand\examplea}

Now the \noexpand will operate (disappearing in the process), leaving the definition of \exampleb as 'Some stuff \examplea'.

We can use this ability to cut down on \expandafter use, but there are a couple of other things to know. First, e-TeX includes an additional primitive \unexpanded, which will prevent expansion of multiple tokens. Secondly, there are various special cases where you don't need quite so many \expandafter statements. A classic example is from within \csname, as this will do expansion anyway. So you'll see things like

\csname name\expandafter\endcsname\token

which will expand \token before \name.

Back to your example. In the first one, there isn't much to do: as the entire point is to have a dynamic name (#1), doing an \edef at point-of-definition doesn't really make sense. The closest one can get is something like

\edef\cohtheory{%
  \noexpand\newcommand\expandafter\noexpand\csname foofunc\endcsname[1][*]{%
  \noexpand\MakeUppercase{foo}^{##1}}%
}

What will happen here is that \newcommand and \MakeUppercase will be protected from expansion, and the \csname will only expand once. (Tokens which don't have an expansion don't need protection, which is why things like '[1]' are simply included as is.) Of course, this is something of a 'toy' as all it does is create a fixed \foofunc.

For your second example, you could instead to

\begingroup
  \edef\temp{%
    \endgroup
    \noexpand\command
    {\unexpanded\expandafter{\argone}}%
    {\unexpanded\expandafter{\argtwo}}%
  }
\temp

I'm using a couple of extra ideas here. First, the group is used so that \temp is not altered anywhere other than where I'm using it. The \endgroup primitive will do nothing inside the \edef, and so will still be there to close the group when \temp is used. Secondly, \unexpanded works like a toks, and so will respect the \expandafter after it but before the {. This cuts down on an unnecessary \expandafter.

There are more wrinkles to this, and often there are several equally-efficient and clear methods. You are best off posting specific examples, and seeking advice on how they might be achieved.

  • The solution I went for is a cross of your two approaches: \def\cohtheory#1{\def\name{\csname#1func\endcsname} \edef\temp{\noexpand\gdef\name{...}}\temp}. Thanks for your time! – Michaël Jan 16 '18 at 16:00
26

One difference of \expandafter and \edef is their behaviour towards protected macros.

eTeX provides the prefix \protected which can be used before \def and friends to define a protected, i.e. "robust" macro which doesn't expand inside an \edef context (like in \write). However, \expandafter does expand such a macro.

See the following example (works with eTeX and LaTeX):

\protected\def\pempty{}

\edef\withedef{x\pempty x}
\expandafter\def\expandafter\withexpandafter\expandafter{\expandafter x\pempty x}

\tt
\meaning\pempty.

\meaning\withedef.

\meaning\withexpandafter.

Gives:

\protected macro:->.
macro:->x\pempty x.
macro:->xx.

There are also the situations where TeX is expanding tokens, like after & and \cr inside a \halign. Here TeX stops when finding a protected macro without expanding it. However, if TeX is expanding tokens while in number reading mode like for \ifnum protected macros are also expanded.

  • 4
    The details of exactly when protected macros are not expanded are in the e-TeX manual. – Joseph Wright Feb 26 '11 at 20:06
10

For the particular problem of creating control sequences dynamically, I suggest you use something like

\def\csarg #1{%
    \begingroup
    \expandafter
    \endgroup
    \expandafter #1\csname
}

You use it like

\csarg\mycommandbuilder <whatever> \endcsname

You can even use it like this

\csarg\mycommandbuilder <whatever> \expandafter\endcsname
    \csname <whatever2> \expandafter\endcsname
    \csname <whatever3> \endcsname

However, take care with the spaces. The above code constructs control sequences that have spaces in their names!

10

Sometimes having a few helper macros makes the code much more readable. For example, in ConTeXt, typically your first macro will be written as

\def\cohtheory#1%
   {\setvalue{#1func}{\dodoubleargument\docohtheory[#1]}}

\def\docohtheory[#1][#2]%
   {#1^{\ifsecondargument #2 \else * \fi}}

where \setvalue is roughly equivalent to the expandafter newcommand bit. (ignore the difference in the manner in which optional arguments are handled).

The second macro will typically be written as

\expanded{\command{\argone}{\argtwo}}

where \expanded fully expands its arguments. \expanded is defined roughly in the same manner as Joseph's \temp macro, but uses \xdef instead of \edef.

  • @ShreevatsaR Oops, I just wanted to correct the syntax highlighting (go to the Markdown source for more information). Will correct it right now – JouleV Apr 22 at 7:20
  • @JouleV \expanded is now a primitive in all engines; the name is very badly chosen. – egreg Apr 22 at 10:14
2

To make this answers complete, one has to know, that TeX uses scopes, that are regions, where the defined macros are know. If you leave that region, the macro (and of course its content) will be destroyed. Lets make an example:

\def\a{foo}
{
  \def\a{bar}
  The actual content of \verb|\a| is \a
}
The actual value of \verb|\a| is \a

will produce

First example output

The reason is due to the fact, that the braces {} form a new scope. Within this scope, the macro \a is local, so to say: it is a quite new macro which stores different content compared to the still existing macro with the same name but resting in a different scope. Therefore, the first call of \a results in the content "bar". Afterwards the scope area ends, the local variable is destroyed and the macro, that belongs to the surrounding scope is "restored". The second call of \a from the outer scope will present the content, that was initially stored in \a: "foo".

If you need to work on the same value in different scopes, you have to use a global variable. Here is another example, to make the point clear:

%% Second example with \b
{
  \global\def\b{baz}
  %% Do some things on \b
}
The actual value of \verb|\b|, defined in a not more existing scope is
still `\b'.

And the result:

Second example output

Instead of the above more literal code, I could also have used the shorter form with \gdef:

%% Second example with \b
{
  \gdef\b{baz}
  %% Do some things on \b
}
The actual value of \verb|\b|, defined in a not more existing scope is
still `\b'.

The result is still the same:

Second example output

Please see the other (real good!) answers about the difference between the normal \def command, and its companion, the expanding version \edef. You can, of course, play the game with expanding macros in the local scope or in the global scope:

%% Third example with \edef and \global\edef
%% Define the variable \foo to show, where and why this is happening.
%% First: the value before the new scope
\def\foo{before}
\def\a{\foo}
\edef\b{\foo}
\def\c{}
\def\d{}
{
  %% Second: within the scope
  \def\foo{within}
  \def\c{\foo}
  \edef\d{\foo}
  \global\def\e{\foo}
  \global\edef\f{\foo}
}
%% Third: after the scope was left, now again in the global scope
\def\foo{after}

Results 
\begin{tabular}{@{} cc @{}}
  \toprule
  \multicolumn{1}{@{} H}{Variable} & \multicolumn{1}{H @{}}{Content} \\
  \midrule
  a & \a \\
  b & \b \\
  c & \c \\
  d & \d \\
  e & \e \\
  f & \f \\
  \bottomrule
\end{tabular}

It should not be surprising, that the variables \c and \d are still empty, as they have not been defined \global, when they were manipulated in the scope:

Third example with \global and \edef

Of course, again I could have saved typing labour, by replacing \global\def with \gdef (see above) and \global\edef with \xdef. So this is an example with all four kinds of def: \def, \edef, \gdef and \xdef.

  • Though it explains well about the scopes I think this is the wrong question for this answer because this is strictly about expansion not scoping. – percusse Feb 11 '17 at 11:21
  • @percusse Thanks for your kind words. I had asked the question, what are the differences between the four ?\defs, as I had not found an answer in TeX.SE. My question was marked duplicate with respect to this question, and I must agree, that to some kind of extent, this is true. In the above answers, \gdef and \xdef are missing. In a more broader view, scopes also affect where something is expanded and where not, see my last example with variables \c and \d. But if you have a strict view, you are right. – Jan Feb 11 '17 at 11:26
  • 1
    That's a wrong duplicate closing reason in my opinion – percusse Feb 11 '17 at 11:43

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.