# Why last character in TeX's input buffer must not be blank?

In §31 of tex.web it is said:

Trailing blanks are removed from the line; thus, either |last==first| (in which case the line was entirely blank) or |buffer[last-1]!=' '|.

Input should be the same when trailing blanks are present as when they are not, due to the fact that blanks are invisible at the end of the line. From this I can deduce that if trailing blanks were not removed, TeX would behave differently.

Why is it necessary to remove trailing blanks? Does anybody know which section in tex.web would work differently if trailing blank would be present in input line? Or some example input?

• Please see the accepted answer to this question: tex.stackexchange.com/questions/7453/… I think it is related to that about which you inquire. – Steven B. Segletes Apr 14 '20 at 10:17
• Those examples use %, which can be used to leave space in end of line, but it removes \endlinechar. Which undesired effects would arise if space in end of line was kept and \endlinechar was preserved? – Igor Liferenko Apr 14 '20 at 11:00
• I think, in such a case, you would end up with two space tokens, which is not the desired default. – Steven B. Segletes Apr 14 '20 at 11:09
• @StevenB.Segletes Well, actually what troubles me is that in §36 nothing is said about removing trailing blanks. So, they are safe in the end of command line? – Igor Liferenko Apr 14 '20 at 11:36
• Consider also the case when the catcode of space is not the usual 10 (e.g. when it is an active character). (For possibly relevant context, also the comment by DRF quoted here, about "IBM's OS360 and VM/CMS".) – ShreevatsaR Apr 14 '20 at 11:36

I believe you gave yourself a good reason why it is not completely unreasonable to ignore trailing blanks: since most people can't see them, having different behaviors depending on their presence could be very confusing (note that I do see trailing blanks, because I have (setq-default show-trailing-whitespace t) in my Emacs configuration). There may be other reasons that I don't know—I only wrote this answer in reply to your comment here.

So, regarding your request for sample input that would behave differently if trailing blanks weren't ignored, I propose the following (which belongs to the category envisaged by ShreevatsaR: catcode different from 10 for the ASCII space character):

\def\visiblespace{{\tt\char32 }}
\obeyspaces\let =\visiblespace
abc def  ghi
\par
\bye


where I've left three spaces after ghi (which are unfortunately invisible here). The output with my unmodified TeX engine is:

I would expect three “visible spaces” after ghi with your modified engine that doesn't ignore trailing blanks.

Here are two other examples, this time with the standard category code for the ASCII space (10):

1. {\endlinechar=X
abc}
d\par
\bye


There is one trailing space after abc}. Subtlety: when \endlinechar is assigned, the line where this happens has already been entirely read by TeX (this is before tokenization starts for the line); that line is thus terminated according to the \endlinechar setting that was in force at the end of the previous line. Ditto after the closing brace: the X has already been appended as line terminator before TeX started to tokenize abc.

2. {\let\par=X\obeylines%
abc
}d\par
\bye


There is one trailing space after abc.

In both cases, an unmodified TeX engine outputs:

I expect that your modified engine prints abc Xd in both cases.

• I get two visible spaces after ghi. – Igor Liferenko Apr 16 '20 at 11:51
• You're right, I mistakenly only put two trailing spaces in the code. This is now fixed; you should see three of them. – frougon Apr 16 '20 at 12:04
• yes, it's OK now – Igor Liferenko Apr 16 '20 at 12:42
• I've added two examples where spaces have their standard catcode (10). Can you confirm my expectations when the TeX engine doesn't strip trailing spaces? – frougon Apr 16 '20 at 13:22
• yes, it produces abc Xd in both cases – Igor Liferenko Apr 17 '20 at 9:38

in TeXLive 2018 the interpretation of "stripping blanks" was corrected to just strip spaces and not tabs, so you can see the effect by comparing texlive 2017 with any later release (texlive 2020 here)

consider the plain TeX


\catcode9\active\def    {X}

one two three

one two
three

\bye


This has two tab characters (U+0009) this site will strip them so I will show them as T here:


\catcode9\active\defT{X}

one two three

one twoT
three

\bye
`

in texlive2017 the tabs are stripped and you get

in TeXlive2020 you get

• I'm not sure how this answers the question. – egreg Apr 14 '20 at 14:20
• @egreg ??? it is an example input that produces different output if trailing blanks are stripped. That is exactly the question is it not? – David Carlisle Apr 14 '20 at 14:21
• No, it's not. The question is: what would happen if TeX didn't strip blanke? The TAB issue is not relevant, as the question refers to the “original TeX”. – egreg Apr 14 '20 at 14:26
• @frougon TeX originally didn't strip tabs at the end of lines. The feature was introduced at some point in time in TeX Live implementations, but now removed. TeX strips characters with character code 32, tokenization has not yet come into play at that stage. – egreg Apr 14 '20 at 14:51
• @egreg the point of giving an example with tabs is you can see the difference using actual implementations rather than an example with space where you have to imagine an implementation that didn't strip, – David Carlisle Apr 14 '20 at 15:33