# Parsing oddity: when is a space not a space

Saying

``````\count89=14 \the\count89~{\count89=11 \the\count89~}\the\count89
``````

produces `14 11 14`, as one would expect. But, get rid of the space in the middle clause:

``````\count89=14 \the\count89~{\count89=11\the\count89~}\the\count89
``````

and you get `14 14`. That makes sense too since the parser must eat up all of `11\the\count89` without a space to terminate the input.

So far, so good, but what does the middle clause actually do? Try

``````\count89=14 \the\count89~{\count89=11\the\count89 \the\count89~}\the\count89
``````

and you get `14 14`. Note the additional space in the middle clause. Yet there is no output. The parser must be eating up everything to the closing brace and assigning it to `\count89`. I know that `\the\count89` evaluates to a series of tokens that are assigned the `other` category code. I thought that this might somehow put the parser into an "eat everything" state until the closing brace, but that's not quite true. If you say

``````\count89=14 \the\count89~{\count89=11\the\count89~\the\count89~}\the\count89
``````

(note the additional `~`), then the output is `14 1114 14`, which is what one might expect.

What's going on here? Why does `~` serve to terminate the input value, but a plain space does not?

• I don’t have constructive comments on this question but… “Space Oddity: When is A Space not A Space” would have been a great title :) Feb 17 at 10:32

From

``````\count89=14 \the\count89~{\count89=11 \the\count89~}\the\count89
``````

you get `14 11 14` and that's expected, as you say. Let's look at

``````\count89=14 \the\count89~{\count89=11\the\count89~}\the\count89
``````

After `11` you have no space and TeX continues expansion until finding `~`. This is an active character, so it behaves like a macro and its expansion starts with `\penalty`. Now the number TeX has found is `1114` and it sets `\count89` to that value. The brace makes it to forget the assignment and you get

``````14  14
``````

with two spaces for the two `~`. The search for a `<number>` in an assignment goes on with macro expansion until something that cannot be interpreted as a digit is found.

If a space token is found during this expansion process, it stops the search for digits and is gobbled. But `~` expands to `\penalty10000\ ` and so this remains.

##### Some more explanations

A space token in the above sense is an explicit character with category code 10. When TeX is looking for digits to perform an assignment, it will do macro expansion (including expandable primitives) and it will stop, as said, when, after expansion, a nondigit is found. If the stop is caused by a space token, it will be ignored; otherwise the token causing the stop is read again after the assignment has been completed.

So, in your second example, `\the\count89~` becomes `14\penalty10000\ ` and the assignment becomes `\count89=1114`, but `\penalty10000\ ` will be processed in the typesetting phase. The same output would be obtained with

``````{\count89=11\the\count89\ }
``````

because `\ ` is not a space token. It produces a space in the output, but that's a completely unrelated business.

• What you and @David Carlisle say is clear, as far as it goes -- although I don't understand what you mean about `\penalty`. But it doesn't answer the real question: why is `~` treated differently than space in the last two situations? Feb 16 at 15:41
• @RandallFairman `~` is a macro (active character) expanding to `\penalty10000\ `. I reworded a bit. Note that the expansion of `~` will be then processed: no gobbling is done. Only a space token is gobbled (if present and used to terminate the search for digits). Feb 16 at 16:08
• It makes sense that `~` expands to `\penalty10000` to indicate "don't break here," as in `Mr.~Smith` but how does that become what acts like a "more powerful delimiter" than space? It's like the parser shifts gears when it sees `\count89=11\the\count89` and decides to eat spaces, but not a `\penalty10000' for some reason. What reason? Category codes? Feb 16 at 16:22
• I notice that `~` is actually `\penalty10000\ ` with a final `\ `. So my question could be rephrased as, "why is `\ ` more powerful than an ordinary space in this context?" Feb 16 at 16:26
• @RandallFairman An assignment to a count register gobbles a space token, not anything that can become space in output. For instance, neither `\ ` would be gobbled, because it isn't a space token. Feb 16 at 16:26

In the inner `\count89 ` the space is absorbed while reading the 89 so that tex seees count register 89

but that means the space is not there when the outer assignment is looking for the value

`\count89=11\the\count89 \the\count89`