# Why does x_\text y work?

As far as I have understand the tokenization in LaTeX, x_\text y will be split up into

1. x
2. _
3. \text
4. y

Now the _ will consume/bind to the next token. However, \text takes an argument as well. If _ consumes \text, the latter does not have an argument.

When I do something like \frac 1 \sqrt 2, it does not work in contrast to the first one, and this is the behavior that I would except.

Why does the first example work at all?

-
@egreg Yes, you're right of course! I deleted my comment. It is expanded before it is sent to "TeX's stomach" according to TeXbook. Can you shed some light on this? –  Henri Menke May 4 at 19:42
@HenriMenke As you see from my answer, it's a bit more complex. ;-) –  egreg May 4 at 19:47

You have to know that _ (in math mode, of course, and with normal setting of category codes) will expand the next token in order to see if a brace (explicit or implicit) follows. This is common for primitives that want a <general text> as their argument (so also \uppercase, for instance).

Thus _ will try expanding the following token, because it's not { or \bgroup. The first level expansion of \text is

\protect\text•


(where • denotes a space in the name of the control sequence). Since we are in a normal situation (typesetting), \protect is equivalent to \relax. Here another feature comes in: when a <general text> is being looked for, spaces and \relax tokens are ignored (they're called a <filler> in the TeXbook), so this \protect disappears. Now \text• is not a brace, so TeX continues expanding it; the first level expansion is

\ifmmode\expandafter\text@\else\expandafter\mbox\fi


OK, we're in math mode, so we're left with

\expandafter\text@\else\expandafter\mbox\fi


Hmm, \expandafter is not a brace, expand it! This causes the expansion of \else, therefore just \text@ remains. Here we are, almost! The definition of \text@ in amstext.sty is

\def\text@#1{{\mathchoice
{\textdef@\displaystyle\f@size{#1}}%
{\textdef@\textstyle\f@size{\firstchoice@false #1}}%
{\textdef@\textstyle\sf@size{\firstchoice@false #1}}%
{\textdef@\textstyle \ssf@size{\firstchoice@false #1}}%
\check@mathfonts
}%
}


so, since TeX wants to expand it, the argument is looked for; y is a token which is not a brace that delimits an argument, so it is the argument.

The expansion is then

{\mathchoice...}


with y in place of #1. Hurray! The open brace has been found! Go on and typeset the subscript.

Final word: always use braces around the subscript and arguments; your input will be clearer. So

$x_{\text{y}}$


and nothing else.

-
Assuming of course that _ is not an active token actually absorbing an argument! –  Joseph Wright May 4 at 19:47
@JosephWright I've added that I assume _ has the usual category code. –  egreg May 4 at 19:48
Thanks for blowing my mind. I only had a hunch that TeX would have a “Alice in Wonderland” experience in store for me so far … –  queueoverflow May 4 at 19:51
@queueoverflow Did you like the <filler> part? –  egreg May 4 at 19:52
So far, everything that has an @ lets me think about it being internal and that I do not need to think about it, if I just want to use it. Kind of like assembly language. I just had no idea that there are so many layers of expansion to it. –  queueoverflow May 4 at 19:59

\text is defined in amstext.sty by means of \text@ where

\text@ #1->{\mathchoice{...}\check@mathfonts}


which adds braces around its expansion text. Thus the braces from the expansion of \text@ will enclose all stuff in the index.

Compare for instance:

\def\bad#1{\rm #1}
\def\good#1{{\rm #1}}
% $x_\bad y$% <- Missing { inserted
$x_\good y$
\bye

-
So \text is handled before _ is? –  queueoverflow May 4 at 19:45
Okay, the answer from @egreg makes it clear now. –  queueoverflow May 4 at 19:51