Take the 2-minute tour ×
TeX - LaTeX Stack Exchange is a question and answer site for users of TeX, LaTeX, ConTeXt, and related typesetting systems. It's 100% free, no registration required.

I'm relatively new to (La)TeX, but I have a reasonable amount of experience with computational programming languages. Until I began learning (La)TeX, I'd never heard of "expansion"...

So, what is "expansion"? To be clear, I'm not asking how to use commands like \expandafter and its ilk.

Perhaps I'm simply unfamiliar with the name of the term. Does "expansion" have an analog in computational languages? From what I've read about it (which has mostly been explanations of commands), it seems like it might be akin to "evaluation" until terminal values are reached, but I'm not sure.

share|improve this question
6  
It is similar to one step of rewriting in grammars, or the replacement of a (C-like) macro with its definition –  Bordaigorl Feb 6 at 1:10
    
    
@MartinSchröder However, inlining typically doesn't change the effect of the functions, only the efficiency. In TeX, this is substantially different. I've tried to find a good wiki article on the TeX's expansions, but I wasn't able to. Should we create one? :) –  tohecz Feb 7 at 14:03
    
@tohecz: Yes, you are right. The expansion in shells is similar, but TeX takes it a step further. –  Martin Schröder Feb 7 at 14:06
    
Welcome to TeX.SX! You can have a look at our starter guide to familiarize yourself further with our format. –  Martin Schröder Feb 7 at 14:06

4 Answers 4

up vote 40 down vote accepted

Expansion is when: You take a thing that is expandable and ... well ... you expand it.

For example, if you define \newcommand\Aa{\Bb\relax}, then:

  1. \Aa is (once) expandable, and it's expansion is \Bb\relax.

  2. \Aa is not fully expandable, because its 1st expansion contains \relax, which is unexpandable.

Shortly, expansion is the process of substituting macro contents in place of macro call. What TeX-core (and whence LaTeX) does is that it:

  • Expands whatever expandable it sees;
  • Processes whatever un-expandable it sees.

So, for instance a TeX-definition macro \def is unexpandable. So TeX processes it, in the following way (ingnoring any specialities): take the first thing after \def -- this is the name of the macro being defined. Take whatever follows until { -- that are the paramemters, and take the following one block {...} and that's what the new macro expands to. In this process, TeX of course doesn't expand the things it processes unless it has to, but that's not the case of \def.

On the other hand, when TeX-core sees in LaTeX \newcommand\greeting[1]{Hello #1!}, then it starts expanding \newcommand (which is expandable to some extent), and after many crazy things (and with me not being quite precise), it expands the sequence \newcommand\greeting[1]{Hello #1!} into \def\greeting#1{Hello #1!}. We already know from before that \def is unexpandable, and when this is processed, it will define a macro called \greeting which takes one parameter and which expands to Hello #1! wherever used.

So now that we have our \greeting macro, let's use it! So we write \greeting{glassbjs}. TeX-core sees it, and knowing that \greeting is expandable and takes one argument, it expands the sequence \greeting{glassbjs} into Hello glassbjs!. Since printable characters (letters, spaces, punctuation, ...) are by default unexpandable, TeX-core tries to process them, which means that it converts them into the glyphs from the current font (again I'm not precise) and it adds them to something that is called "the current horizontal list". And at that moment, TeX-core is happy about it and it moves forward.

If you look for a similarity in other languages, expansion is closest to pre-processor substitutions like C's #define f(x) something-here. So the above thing would be in pretty similar to (in C++):

#define greeting(x) cout << "Hello " << x << "!"

However, expansion has some strange rules that do not apply for pre-processing in C. The two main differences are:

  1. Expansion is not pre-processing, it goes along with procession. Which means that:

  2. You can re-define things, and the definition used is the one that is valid in the current scope; in this sense, all macros are much more similar to variables than to functions; they are sensitive to changes, to grouping etc.

share|improve this answer
    
May be worth noting: (1) \newcommand isn't actually fully expandable, but is a macro so gets replaced by a definition, then there is some execution, then you do end up effectively with \def; (2) The difference in TeX expansion compared with C-preprocessing is that TeX replacements happen in parallel with execution and so can change, while in C it's a one shot 'search and replace'. –  Joseph Wright Feb 6 at 7:00
4  
“Full expandability” usually means being able to expand (recursively) up to unexpandable tokens. Otherwise only expansion producing an empty list of tokens would be fully expandable according to your definition. –  egreg Feb 6 at 8:30

Expansion by example, from the TeX Book (p 200, answer on p 328):

EXERCISE 20.2: What is the expansion of \puzzle, given the following definitions?

\def\a{\b}
\def\b{A\def\a{B\def\a{C\def\a{\b}}}}
\def\puzzle{\a\a\a\a\a}

Answer: ABCAB. (The first \a expands into A\def\a{B...}; this redefines \a, so the second \a expands into B..., etc.) At least, that's what happens if \puzzle is encountered when TeX is building a list. But if \puzzle is expanded in an \edef or \message or something like that, we will see later that the interior \def commands are not performed while the expansion is taking place, and the control sequences following \def are expanded; so the result is an infinite string

A\def A\def A\def A\def A\def A\def A\def A\def A...

which causes TeX to abort because the program's input stack is finite. This example points out that a control sequence (e.g., \b) need not be defined when it appears in the replacement text of a definition. The example also shows that TeX doesn't expand a macro until it needs to.

We can see the expansion when using \tracingmacros=1 (see LaTeX \tracing commands list?) and examining the .log of the following MWE:

\documentclass{article}
\def\a{\b}
\def\b{A\def\a{B\def\a{C\def\a{\b}}}}
\def\puzzle{\a\a\a\a\a}
\begin{document}
{\tracingmacros=1

\puzzle}
\end{document}

With some comments:

\puzzle ->\a \a \a \a \a % Expansion of \puzzle contains five \a's

\a ->\b % Expansion of first \a

\b ->A\def \a {B\def \a {C\def \a {\b }}} % Expansion of \b (adds A to output)

\a ->B\def \a {C\def \a {\b }} % Expansion of second \a (adds B to output)

\a ->C\def \a {\b } % Expansion of third \a (adds C to output)

\a ->\b % Expansions of fourth \a

\b ->A\def \a {B\def \a {C\def \a {\b }}} % Expansion of \b (adds A to output)

\a ->B\def \a {C\def \a {\b }} % Expansion of fifth \a (adds B to output)

The final redefinition of \a (to C\def\a{\b}) is never "used".

share|improve this answer
    
I've always liked this example; in a very concise way Knuth illustrates some important characteristics of expansion. –  Gonzalo Medina Feb 6 at 1:50
1  
@Gonzalo Knuth has a penchant for super-dense instructive examples. In his "Art of Computer Programming" he has at least one exercise on the MIX programming language that he proudly informs the reader contains deliberate errors (unspecified) for them to uncover and unravel for further edification upon the relevant concepts. –  Ryan Reich Feb 6 at 2:38
4  
@GonzaloMedina knuth’s “super dense” examples (quoting Ryan here) are, i think, a reason why tex was slow to “take off” — they're fine for people experienced in maths, but seem to repel the ‘ordinary’ humanities person. –  wasteofspace Feb 6 at 11:46

Expansion is simply half of how TeX works, so this is not going to be an easy answer.

First of all, TeX operates on tokens (see What is a token?). A token can be either expandable or unexpandable; if it's unexpandable, it obviously never expands, otherwise it can expand. What an expandable token expands to depends on how it has been made part of the language.

Some primitive (that is, undefined) tokens are expandable, some aren't. Among the expandable primitive one finds \expandafter, \if and all other conditionals; this was predictable, as these primitive are used to control the expansion process itself; the expansion of an expandable primitive can be void, but it can trigger some other action. All macros defined with \def (along with \gdef, \edef or \xdef) are expandable and their expansion is precisely the replacement text.

So, assume we have

\def\test{some text}

or, in LaTeX, \newcommand\test{some text} which is pretty similar. Under normal operations, if TeX finds

Here is \test

it will stop at \test, examine it and decide that it is expandable (TeX uses the current meaning of \test), so it replaces it by some text and goes on starting from “s”. Since this token is unexpandable, it's handed over to another phase of processing and TeX goes on with “o” and so on.

Now, suppose that we have

\def\test{\foo{o}me text}
\def\foo#1{s#1}

an input such as

Here is \test

would make TeX hand over tokens up to finding \test, which it would replace with

\foo{o}me text

and \foo would now be expanded to so, after having absorbed its argument.

I said “normal operations”, because in some cases expansion is inhibited:

  1. when TeX is looking for macro arguments;

  2. when TeX is doing a definition with \def or \gdef, with regard to the macro to be defined, the parameter text and the replacement text;

  3. when TeX is doing a definition with \edef or \xdef, but this only concerns the macro to be defined and the parameter text, not the replacement text;

  4. when TeX is examining the token to be defined after \let, \chardef, \mathchardef and all other \...def commands;

  5. when TeX is examining the “right hand side” of a \let;

  6. when TeX is absorbing the tokens for \write, \uppercase, \lowercase, \detokenize, \unexpanded or an assignment to a token register.

Sometimes we talk about "full expansion". What's that? When we do

\edef\baz{<tokens>}

the <tokens> are subject to expansion, the token list is subject to expansion and so on, until only unexpandable tokens remain. After this, the definition is performed, using the token list so obtained as replacement text. Thus, with the definition above,

\edef\baz{\test}

would expand \test to \foo{o}me text and this to some text, resulting in the same as if we said \def\baz{some text}. Why should we want to do in this way? Because TeX uses the current meaning of the tokens; if we said

\def\baz{\test}

and later change the definition of \foo (or of \test, of course), the macro \baz would expand to something else; with \edef we're sure it expands to some text.

However, \edef doesn't perform assignments. So if we do

\count255=1
\edef\baz{\advance\count255 by 1 \number\count255 }

using \baz would print 1, not 2. This is because \advance and \count are not expandable; character tokens aren't expandable either; \number is expandable and its expansion is the decimal representation of the <number> following it. So this \edef is just equivalent to

\def\baz{\advance\count255 by 1 1}

The same happens when, ultimately, TeX expands the token list in a \write: the expansion is performed "all the way", just like in \edef. Expansion of tokens can be inhibited by preceding each one with \noexpand or enclosing them as the argument to \unexpanded.

Also tokens appearing after \csname and before the matching \endcsname are subject to full expansion; in this case only character tokens must remain, so a \relax is illegal here, while it wouldn't be in \edef or \write.

Complicated? Yes.

share|improve this answer
    
Very good explanation about the expansion rule. It is strange that no one comment on this answer before. Expansion looks like visit a tree like structure. You go down from the root to the leaf. When you want to a "full expansion", you just find one child of the root, and go down to the leaf (like a left recursive search), all the leaves are the output of the expansion. Once the first child of the root is handled, it goes to the second sibling. Like a top-down recursive parser or tree visitor, right? –  ollydbg23 May 5 at 14:23
    
@ollydbg23 Well, macros can have arguments, which maybe make the thing more complicated. –  egreg May 5 at 15:28

@glassbjs, others,

I would have just dropped in a comment at this point, since the question has been "answered" already, rather than post another — I hate how on each StackExchange site you can't comment until you've accumulated enough rep on that particular site — but since this is really the only way I can respond, I'll try to answer your question slightly more directly (i.e. from the "computational languages" perspective).

So, what is "expansion"?

It is the process by which an otherwise static variable assignment is transformed by substitution.

Does "expansion" have an analog in computational languages? ...it seems like it might be akin to "evaluation" until terminal values are reached, but I'm not sure.

Yes, it does. It's not even an analog per se, its the same concept and terminology.
And no, its not like eval, not quite.

Precisely, its this: in a shell script, strings with "double quotes" are expanded. Those in 'single quotes' are not. That's why if you echo "some $(echo thing)", the output is some thing: the command inside the $() is executed first. After that, its called a "fully expanded" string, and the outer echo is performed on that. On the other hand, echo 'some $(echo thing)' outputs some $(echo thing), because its not expanded. Its also the difference between $@ and $*.

share|improve this answer
1  
Hi! Well, once you reach 200 rep on any site, this problem will disappear (c.f. "association bonus"). –  tohecz Feb 7 at 10:24
    
Good to know, thanks. Oh the dance I'll do that day comes. :) –  Geoff Nixon Feb 7 at 10:25
    
"c.f." should be "cf.". –  KeithB Feb 12 at 10:25
1  
Actually, I think it's either, depending on whether you take it to be a Latin abbreviation or an English one (confer v. carry forward). Curiously, my dictionary lists both, and additionally, "CF"; but on Wiktionary, that last form is apparently more likely to abbreviate "Charlie Foxtrot" or "cluster fuck". –  Geoff Nixon Feb 12 at 13:18

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.