# TeX's expansion rules: a case study with a token-list register

I'm trying to figure out TeX's rules of expansion. Consider the following plain TeX manuscript:

\toks1{hello}%
\toks10{world}%
\def\mac{0}%
\the\toks1\noexpand\mac%
\bye


When this manuscript is processed with pdftex, the following pdf results:

and the log file contains the following warning:

Underfull \hbox (badness 10000) has occurred while \output is active
[]\tenrm world

\hbox(8.5+0.0)x469.75499
.\vbox(8.5+0.0)x0.0
.\tenrm w
.\kern-0.27779
.\tenrm o
.\tenrm r
.etc.

[1{/usr/local/texlive/2017/texmf-var/fonts/map/pdftex/updmap/pdftex.map}] )<./c
mr10.pfb>


If I replace both occurrences of 0 in the manuscript by 2, so that the manuscript becomes

\toks1{hello}%
\toks12{world}%
\def\mac{2}%
\the\toks1\noexpand\mac%
\bye


the pdf contains a single line:

hello2

and no warnings in the log file.

Could someone please explain to me why the pdf document looks the way it does, as well as what the warning means, and why it is being reported. Also, please walk me through the steps TeX goes through in processing my manuscript. Thanks.

• warning is about something else. But what is the problem here? toks is expecting a number until it finds something that doesn't expand to a number and that's 1 – percusse Aug 22 '17 at 9:33
• @percusse: Sorry, I don't follow your line of thought. Do you mean to say that in the line that starts with \the, TeX first processes \the\toks1 and after it finishes it processes the rest of the manuscript? But if this were the case, the output would have been simply "hello", and not what we see. – Evan Aad Aug 22 '17 at 9:37
• What is confusing you about the output? – percusse Aug 22 '17 at 9:39
• @percusse: Firstly, the fact that both "hello" and "world" are printed. I could kinda wrap my mind around any of the following outputs: 1. "hello", 2. "hello0", 3. "world". Secondly the whitespace: two lines (or is it three?) and the second one is indented. – Evan Aad Aug 22 '17 at 9:41
• Explained here tex.stackexchange.com/questions/138/… – percusse Aug 22 '17 at 10:22

I'll skip over the three assignments and go straight to

\the\toks1\noexpand\mac


The \the primitive inserts the content of a register: here a \toks. After the \toks we need a number, so TeX goes looking for one: it finds a literal 1 then a \noexpand, which temporarily turns \mac into \relax and so terminates the search for a number. \toks1 contains hello so that gets inserted (and starts a paragraph), then TeX re-reads \mac which is now 'back to normal': a macro expanding to 0. So the paragraph you see is hello0. (There is of course a paragraph indent as one would expect at the start of a paragraph.)

So why does world get inserted? That happens as you've picked \toks10: alter the number and it vanishes. In plain TeX, \headline is \toks10 so gets inserted as part of the page finalisation: it's got nothing to do with the specific input you have (other than that you've set it!). Of course, this is why we almost always use the allocation system for registers.

Here, \tracingall is probably your friend: the example is short and the trace is clear.

• Thanks. You wrote: "alter the number and it vanishes". If I replace both occurrences of 0 in the manuscript by 1, then the output is: "world". But according to your explanation, it should be: "world1", no? – Evan Aad Aug 22 '17 at 9:53
• Or, rather, "hello1". At any rate, not "world". – Evan Aad Aug 22 '17 at 9:58
• I get as expected hello1 if I replace the 0. – Ulrike Fischer Aug 22 '17 at 9:59
• @UlrikeFischer: I don't. Even if I add \font\myfont=cmtt14\myfont% as the first line. – Evan Aad Aug 22 '17 at 10:01
• @EvanAad In the output routine, there is \line{\vbox to8.5\p@{}\the\headline}\vss} and with \toks10={world}, which is the same as \headline{world}, there is no glue to fill up \line (which is \hbox to\hsize). – egreg Aug 22 '17 at 10:27