# Why does LaTeX tell me that my external text file has a last line which contains the token \par?

Why does LaTeX (TeX Live 2020, Debian) tell me that SomeLines.txt has a 6th line which contains the token \par?

I know with \endlinechar being 13 and the catcode of char 13 (carriage return) being 5 you get \par for an empty line because a carriage-return character is appended to the empty line and that carriage-return character is processed while the carriage-return character is of category 5(end of line) and TeX's reading-apparatus is in state N.

So I suppose TeX's routine for reading a line of input "thinks" there is an empty line/an empty record after Line 5 and appends the endline-character, whose category is 5(end of line). When processing that character, TeX's reading-apparatus is in state N leading to the appended endline-character yielding the control-word-token \par.

If so:

• Why does TeX think that there is an empty line/an empty record after Line 5?
• Do the routines for reading a line of input of all TeX distributions think so?
\begin{filecontents*}{SomeLines.txt}
Line1
Line2
Line3
Line4
Line5
\end{filecontents*}

\newcount\SomeLinesCount
\def\CountLinesLoop{%
\else
\message{^^JLine \number\SomeLinesCount="\Thisline"}%
\expandafter\CountLinesLoop\fi
}
\SomeLinesCount=0
\begingroup%
% Changes of the value of \endlinechar are reflected in the line "Line6=...".
% If \endlinechar denotes the number of the code-point of a
% character of category 5(end of line), the state of the reading-
% apparatus is reflected, too, which apparently is N when the
% character appended to line/record 6 is processed.
%\endlinechar=\A  % -> Line 6="A"
%\endlinechar=\B  % -> Line 6="B"
\CountLinesLoop%
\endgroup%
\message{^^JTotal amount of lines: \number\SomeLinesCount}%
\stop


Console output:

This is pdfTeX, Version 3.14159265-2.6-1.40.21 (TeX Live 2020) (preloaded format=pdflatex)
restricted \write18 enabled.
entering extended mode
(./test.tex
LaTeX2e <2020-10-01> patch level 4
L3 programming layer <2021-02-18>

LaTeX Warning: Writing file ./SomeLines.txt'.

Line 1="Line1 "
Line 2="Line2 "
Line 3="Line3 "
Line 4="Line4 "
Line 5="Line5 "
Line 6="\par "
Total amount of lines: 6 )
No pages of output.
Transcript written on test.log.


I assume:

The condition for terminating the loop is not "if the last record-terminator of the file is reached".

The condition for terminating the loop is \ifeof, i.e., "if the end of the file is reached".

There are five lines/records (none of them containing unmatched characters of category 1(begin group) so that in this scenario a single \read-operation processes a single line/record) and thus five "record-terminators" and one "end of file" belonging to SomeLines.txt, which makes a total of six reading-iterations until the end of the file is reached. In the 6th iteration the "line/record" is considered empty because there is nothing between the last "record-terminator" and the "end of file".
TeX's handling of empty lines and the coming into being of the token \par is described above.

Is this interpretation correct?

In his answer David Carlisle pointed out:

tex.web 9507 says

@ An empty line is appended at the end of a |read_file|. @^empty
line at end of file@>


Seems this means an empty line/record is appended.
Seems this is done on purpose.

Thus subsequent questions are:

Why? What purpose?

Did I overlook the documentation of this "feature" in the TeXbook?

Is this one of the many features for which you need to study tex.web to become aware?

The TeXbook says in Chapter 20: Definitions (also called Macros):

To get input from an open file, you say
\read⟨number⟩to⟨control sequence⟩
and the control sequence is defined to be a parameterless macro whose replacement text is the contents of the next line read from the designated file. This line is converted to a token list, using the procedure of Chapter 8, based on the current category codes.
Additional lines are read, if necessary, until an equal number of left and right braces has been found. An empty line is implicitly appended to the end of a file that is being \read.

• tex is really record based and doesn't assume the input has eol characters at all, the last line whether terminated by eol or eof in the file is normalised to the same tex input Nov 2 '21 at 20:43
• if you set \endlinechar to -1 the final "fake line" is shown as empty, but I'm not sure if that answers your question Nov 2 '21 at 20:59
• @DavidCarlisle I edited my question and called the thing "record terminator". The question could be rephrased: Why dies TeX assume the presence of an empty record after the record "Line5"? Nov 2 '21 at 21:01
• @DavidCarlisle I know. I kept \endlinechar's value 13 because appending \endlinechar is another action that takes place on record-basis and the resulting \par shows that there must have been an empty record where the endline-character was appended to. (Otherwise the reading-apparatus would not have been in state N leading to the endline-character yielding \par.) If TeX did not assume an empty record, it would not have appended the endline-character... Nov 2 '21 at 21:06
• yes as I show below, tex just explicitly (and somewhat arbitrarily) acts as if there is a blank line at the end, Probably some pre-historic normalisation of interesting file systems of the day Nov 2 '21 at 21:09

When reading files tex acts essentially the same way whether or not there is an end of line character at the end of the file. The final line is taken as a line and \endlinechar is added to the end of each line if this is set to a legal character value.
@ An empty line is appended at the end of a |read_file|.

so if \endlinechar is the normal 13 and character 13 has catcode 5 then as usual that ends up being reported as \par. If you set \endlinechar=-1 then the final line before the \ifeof` is true will be reported as empty.