Testing "sequentiality" of \expandafter (plain Tex) - weird results

Trying to have a better understanding/controlling over \expandafter, I wrote this code.

\count100 = 0
\count101 = 1
\def\c{%
\the \count100
%
\everypar{\count100 = 0\relax}

% the ending dot-brace sequence remembers me the commands executed on each line

\expandafter\c\c\c \hfil $...$

\expandafter{\c\c}\c \hfil $\{..\}.$ % I think it's
%                                      useless for
%                                      this situation

\expandafter\c{\c}\c \hfil $.\{.\}.$

\expandafter\c{\c\c} \hfil $.\{..\}$


This outputs the sequence of numbers:

012

313

411

212

After a long time thing about such result and after a look on Internet, I've concluded: that makes no sense to me.

Could someone help me to understand such output?

• I'm just curious about what output did you expect... Jan 25 at 17:20

Basically you have \everypar reset the count-register 100 to the value 0 and the command \c deliver \the\count100, i.e., the current value of the count-register 100 and increment that count-register.

I think the tricky thing is not \expandafter but the mix of the circumstances that

• expansion is not typesetting and therefore does not change the typesetting-mode.
• scoping is not typesetting and therefore does not change the typesetting-mode.
• \everypar is invoked at certain moments in time for "injecting"/inserting tokens into the token-stream. The moments in time are when TeX encounters a token which triggers switching to (unrestricted) horizontal mode—(unrestricted) horizontal mode is the typesetting-mode where TeX also does the breaking of text across lines for you—, and beginning gathering material from which, when gathering is done, a paragraph is to be crafted whose text is broken across lines as nicely as possible. At those moments in time, before injecting those tokens, vertical \parskip-glue is added to the vertical list where the boxes forming the lines of the paragraph are to be appended, and horizontal \parindent-indentation is inserted at the left end/the begin of the box which contains the first line of the paragraph. (That was a simplified description of the happenings.)
• the moment in time when TeX switches to horizontal mode might be a moment where previously things like opening up a local scope and/or obtaining (not typesetting yet) tokens via \the-expansion or whatsoever expansion-trickery and/or whatsoever other things that did not change the typesetting-mode took place.

\expandafter\c\c\c \hfil $...$

\count100 is initialized to 0. TeX is in vertical mode. We have three sequences of delivering the value of \count100 and afterwards incrementing \count100: When processing \the\count100 coming from the first \c, you get the token 0. Processing that token causes TeX to typeset it and to hereby switch to horizontal mode and thus to carry out \everypar, which in turn resets \count100 to 0. Then the pending \advance\count100 by\count101\relax, also coming from the first \c, is carried out so that \count100 has value 1. The second and the third \c each yield typesetting and incremeting the value of \count100, so that you get 1 and 2 and in the end \count100 has value 3.
(\expandafter just yields that the toplevel-expansion of the second \c is delivered before expanding the first \c but that does not matter: the first \c does not use tokens belonging to the toplevel-expansion of the second \c as arguments.)

So in the end you have 012 and \count100 has value 3. When "0" was processed, TeX switched to horizontal mode for typesetting a paragraph and hereby reset a register whose value was 0 to the value 0. When the empty line was encountered, the paragraph was finished and TeX switched back to vertical mode.

\expandafter{\c\c}\c \hfil $\{..\}.$ %

Here \expandafter yields that toplevel-expansion of \c is delivered before TeX sees the curly brace which opens up the local scope. In this case it doesn't doesn't make a difference for the final result. In any case TeX is still in vertical mode and opens up a local scope. Inside that local scope \the\count100 coming from the first \c delivers/prints the token 3 as that is the current value of \count100. When TeX processes that token, TeX starts a new paragraph by switching to horizontal mode. Thus \everypar is executed and count-register 100 is reset to 0. The subsequent \advance \count100 by \count101\relax, also coming from the first \c, increments count-register 100 by 1 so that now it has value 1. The subsequent/second \c prints that 1 and increments count-register 100 by 1 so that now it has value 2. The closing brace closes the local scope: Now the value of count-register 100 is back to 3. The third \c prints the value "3" and increments the register so that now it has value 4. The subsequent empty line finishes the paragraph and TeX goes back to vertical mode.

\expandafter\c{\c}\c \hfil $.\{.\}.$

Here \expandafter causes TeX to look at { (which is not expandable) before looking at the first \c, thus \expandafter has no effect.

The first \c delivers: \the \count100 \advance \count100 by \count101\relax.
As the count-register 100 has value 4 \the\count100, coming from the first \c yields 4. For typesetting this token TeX switches to horizontal mode whereby \everypar is carried out so that \count100 is reset to 0. The subsequent \advance\count100 by \count101\relax, also coming from the first \c, increments \count100 so that it now has value 1.
Now the local scope is opened. Inside the local scope, due to the second \c the current value of \count100 is delivered, which is 1 and \count100 is incremented and now has value 2. Then the local scope is closed and the value of \count100 is reset to what it was before opening up the local scope, i.e., is reset to 1. The 3rd \c yields typesetting that value 1 and incrementing \count100 so that \count100 now has the value 2.
The subsequent empty line causes TeX to finish the paragraph and to switch back to vertical mode.

\expandafter\c{\c\c} \hfil $..\{.\}$

Again \expandafter causes TeX to look at { (which is not expandable) before looking at the first \c, thus \expandafter has no effect.

The first \c causes TeX to deliver \the \count100 \advance \count100 by \count101\relax. As count-register 100 has value 2, \the\count100 yields the token 2. When this token gets processed, TeX switches to horizontal mode and hereby carries out \everypar. Thus the count-register is reset to the value 0. Then the subsequent \advance \count100 by \count101\relax is carried out, yielding that the count-register 100 has value 1. Then a local scope is opened up by {.Then, due to the second \c, the value of count-register 100, i.e., the token 1 is delivered and count-register 100 is incremented so that its value is "2". Then, due to the third \c, the value of count-register 100, i.e., the token 2 is delivered and count-register 100 is incremented so that its value is "3". Then the local scope is closed by } and the value of count-register 100 is back at 1.

• Man... Unbelievable how TeX can be so complex. You shot the matter in head. Reading the initial part of the third paragraph (after the "list of circumstances"), I could understand. Again, TeX overwhelms me. Thank you a lot to show me what was going on! Jan 17 at 22:54

The first thing to realize is that the reset associated with \everypar does not take hold until you leave vertical mode. So to make things easier, let us modify \c definition to initially leave vertical mode. Everything else stays the same.

The second thing to realize is that each \expandafter accomplishes exactly nothing (will explain later), so for purposes of understanding the output, they can be ignored.

\documentclass{article}
\begin{document}
\count100 = 0
\count101 = 1
\def\c{\leavevmode%
\the \count100
%
\everypar{\count100 = 0\relax}

% the ending dot-brace sequence remembers me the commands executed on each line

\expandafter\c\c\c \hfil $...$

\expandafter{\c\c}\c \hfil $\{..\}.$ % I think it's
%                                      useless for
%                                      this situation

\expandafter\c{\c}\c \hfil $.\{.\}.$

\expandafter\c{\c\c} \hfil $..\{.\}$

\end{document}


With this set, the first invocation of \c in each paragraph will start with \count100 equal to 0. In the first case, \c\c\c, it behaves as expected, with an initial reset to 0, and an increment of 1 with each invocation.

The second case {\c\c}\c resets to 0 inside a group, accounting for the initial 01 in the output. However, when the group ends, the reset is confined inside the group, so that the third \c increments from what was left on the prior line, which was a 2 incrementing to 3.

The third line \c{\c}\c has a similar behavior. The count is reset to 0, it increments to 1 just prior to entering the group. But, upon group exit, it only recalls the pre-group value of 1, and so the final \c again prints 1, before incrementing.

In the last example, the invocations produce a 012. However, the value of \count100, upon exiting the group, after the third invocation, is still 1, which is what the counter was incremented to just before entering the group.

Now let us understand why the \expandafter accomplishes nothing. Acting upon \c, it merely substitutes the definition of \c for the token \c. However, \the\count100 is still not executed (or even the \leavevmode). Thus, it has no effect on pre-expanding anything inside of \c. In some cases, you aren't even expanding \c, but trying to expand {, which does nothing.

Finally, let us return to the OP's original case, which omitted the \leavevmode (\expandafter is still useless). Now, the count reset associated with \everypar is not executed until vertical mode is left, which doesn't occur until the initial \the\count100 is encountered in the first invocation of \c.

So, the first line acts as expected, only because \count100 is initially equal to 0.

For the 2nd line, the value of \count100 left at the end of the prior paragraph is in force for \the\count100 and then reset to 0. Thus, the 31 of the second line reflects this. But, as before, when the group ends, the final \c invocation only remembers the 3 from the end of the prior paragraph.

For the 3rd line, the 4 reflects the value at the end of the prior paragraph, which is reset to 0 after the 4 is typeset. Thus, the second \c produces a 1. However, the last \c also produces a 1, because the increment of the count inside the group is lost in the scope.

Similarly, for the last paragraph, the initial 2 is left over from the end of the prior paragraph, and then the count is reset to 0. Therefore, the 2nd and 3rd invocations yield a 12.

If I understood what it was you were trying to accomplish with the \expandafters, I might be able to help, but it is not clear to me what you are hoping to have happen.

• First: thank you too much. During the reading your answer, a lot of question appears to me (and others certainly will do until a read it completely). Making questions in parts comes well, I believe. Let me do it so. Is it relevant using \relax in \everypar's last token for finishing a number? You do it... Coincidentally, this doubt comes about these days. Jan 17 at 22:34
• @DanielBandeira Regarding the question about paragraph being complete, TeX is, well, TeX. Yes the paragraph is complete, but vertical mode (movement between paragraphs) and horizontal mode (movement within a paragraph) are unique concepts peculiar to TeX. And this result is, even to me, not something I would have expected but, apparently, the \everypar commands are not actually executed until entry into horizontal mode is detected. Thus, \the\count100 is evaluated before the reset instituted in the \everypar, even if it is not yet printed. Jan 17 at 23:40
• Tokenizing the end-of-line-character denoting the end of an empty line yields the token \par. \par is the directive to typeset the paragraph for which TeX is currently gathering material and to switch back to vertical mode. Finishing gathering material for a paragraph and typesetting that paragraph does not necessarily imply beginning gathering material for another paragraph. There might be directives to go to math-mode or to restricted horizontal-mode or to finish the job etc. \everypar is executed when encountering tokens that trigger TeX into gathering material for another paragraph. Jan 18 at 0:04
• @DanielBandeira TeX has typesetting-modes: In vertical mode each box produced by TeX is placed below the box produced beforehand. In restricted horizontal mode each box produced by TeX is placed at the right of the box produced beforehand. No matter what box it is. Whether the box is a \hbox or a \vbox only indicates how the material within the box is arranged, not how the box itself which contains that material is arranged. In (unrestricted) horizontal mode TeX gathers material of a paragraph and arranges horizontal boxes (lines of text) that are placed vertically below each other. Jan 18 at 0:27
• @DanielBandeira ... \everypar just a certain moment of time "inject/inserts" a set of tokens into the token-stream. Later stages of processing don't "know" how/why those tokens made it into the place of the token-stream where they are encountered. Jan 18 at 13:23

First and upmost: your \expandafter does nothing at all.

If you're hoping that it causes \count100 to be stepped before the first \c instruction is reexamined, your expectation is wrong. Indeed, after expanding \expandafter, TeX will see the token list (line breaks just for readability)

\c
\c


and it will proceed to expand \c according to its definition. So you'll get

\the\count100 \advance\count100 by \count101\relax
\c


So you get

• 0 is printed;
• \count100 is stepped to 1;
• 1 is printed;
• \count100 is stepped to 2;
• 2 is printed;
• \count100 is stepped to 3.

Finally, \hfil$..$ is processed. Now comes a \par that ends the paragraph. OK, you now have

\expandafter{\c\c}\c \hfil text


(whatever comes after \hfil is essentially immaterial). Also in this case \expandafter does nothing sensible and TeX ends up seeing

{\the\count100 \advance\count100 by \count101\relax\c}\c \hfil text


OK, a group is opened, TeX expands \the and realizes that this causes a character token to be processed. This character token is 3, that is, the value of \count100 at the moment. Since this would start a paragraph, the contents of \everypar is delivered after providing the indentation box; the counter is reset to 0, but \the\count100 has already been expanded, so the result is

• \count100 is reset to 0;
• 3 is printed;
• \count100 is stepped to 1;
• 1 is printed;
• \count100 is stepped to 2;
• the group ends and \count100 gets back the value it had before the group started
• 3 is printed;
• \count100 is stepped to 4.

Now you can go on and explain the next output, I've already shown that 012 and 313 do make sense. Likely not what you expected, but…

1. \everypar is used after something triggered the start of a paragraph;
2. \expandafter causes just one expansion step;
3. assignments are anyway not performed during macro expansion.

Point 1 is really important. You may try to explain what you get with

\count100 = 0
\count101 = 1
\def\c{%
\the \count100
%
\everypar{\count100 = 0\relax}

% the ending dot-brace sequence remembers me the commands executed on each line

\noindent\c\c\c \quad $...$

\noindent{\c\c}\c \quad $\{..\}.$

\noindent\c{\c}\c \quad $.\{.\}.$

\noindent\c{\c\c} \quad $..\{.\}$


I removed the \expandafter tokens because they do nothing sensible; \noindent will cause \everypar tokens to be delivered and \quad makes for a clearer output.

• Your code addaptation is interesting to this question. Thank you! Jan 17 at 23:22