As far as I have understand the tokenization in LaTeX, x_\text y will be split up into

  1. x
  2. _
  3. \text
  4. y

Now the _ will consume/bind to the next token. However, \text takes an argument as well. If _ consumes \text, the latter does not have an argument.

When I do something like \frac 1 \sqrt 2, it does not work in contrast to the first one, and this is the behavior that I would except.

Why does the first example work at all?

  • @egreg Yes, you're right of course! I deleted my comment. It is expanded before it is sent to "TeX's stomach" according to TeXbook. Can you shed some light on this? – Henri Menke May 4 '14 at 19:42
  • @HenriMenke As you see from my answer, it's a bit more complex. ;-) – egreg May 4 '14 at 19:47

You have to know that _ (in math mode, of course, and with normal setting of category codes) will expand the next token in order to see if a brace (explicit or implicit) follows. This is common for primitives that want a <general text> as their argument (so also \uppercase, for instance).

Thus _ will try expanding the following token, because it's not { or \bgroup. The first level expansion of \text is


(where denotes a space in the name of the control sequence). Since we are in a normal situation (typesetting), \protect is equivalent to \relax. Here another feature comes in: when a <general text> is being looked for, spaces and \relax tokens are ignored (they're called a <filler> in the TeXbook), so this \protect disappears. Now \text• is not a brace, so TeX continues expanding it; the first level expansion is


OK, we're in math mode, so we're left with


Hmm, \expandafter is not a brace, expand it! This causes the expansion of \else, therefore just \text@ remains. Here we are, almost! The definition of \text@ in amstext.sty is

  {\textdef@\textstyle\f@size{\firstchoice@false #1}}%
  {\textdef@\textstyle\sf@size{\firstchoice@false #1}}%
  {\textdef@\textstyle \ssf@size{\firstchoice@false #1}}%

so, since TeX wants to expand it, the argument is looked for; y is a token which is not a brace that delimits an argument, so it is the argument.

The expansion is then


with y in place of #1. Hurray! The open brace has been found! Go on and typeset the subscript.

Final word: always use braces around the subscript and arguments; your input will be clearer. So


and nothing else.

  • 1
    Assuming of course that _ is not an active token actually absorbing an argument! – Joseph Wright May 4 '14 at 19:47
  • @JosephWright I've added that I assume _ has the usual category code. – egreg May 4 '14 at 19:48
  • 1
    Thanks for blowing my mind. I only had a hunch that TeX would have a “Alice in Wonderland” experience in store for me so far … – Martin Ueding May 4 '14 at 19:51
  • 1
    @queueoverflow Did you like the <filler> part? – egreg May 4 '14 at 19:52
  • So far, everything that has an @ lets me think about it being internal and that I do not need to think about it, if I just want to use it. Kind of like assembly language. I just had no idea that there are so many layers of expansion to it. – Martin Ueding May 4 '14 at 19:59

\text is defined in amstext.sty by means of \text@ where

\text@ #1->{\mathchoice{...}\check@mathfonts}

which adds braces around its expansion text. Thus the braces from the expansion of \text@ will enclose all stuff in the index.

Compare for instance:

\def\bad#1{\rm #1}
\def\good#1{{\rm #1}}
% $x_\bad y$% <- Missing { inserted
$x_\good y$
  • So \text is handled before _ is? – Martin Ueding May 4 '14 at 19:45
  • Okay, the answer from @egreg makes it clear now. – Martin Ueding May 4 '14 at 19:51

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.