3

In web2cTeX (TeX Live) I observed this behavior:

(after each of the following perl commands run: "tex xyz.tex; xdvi xyz.dvi")

  1. Each \r and \n separately produces one line:

    perl -e 'print "\\obeylines a\rb\\bye"' > xyz.tex
    perl -e 'print "\\obeylines a\nb\\bye"' > xyz.tex
  1. \r and \n combined in various ways produce two lines (thus - a paragraph), as expected, except one case:

    perl -e 'print "a\r\rb\\bye"' > xyz.tex
    perl -e 'print "a\r\nb\\bye"' > xyz.tex # why?
    perl -e 'print "a\n\rb\\bye"' > xyz.tex
    perl -e 'print "a\n\nb\\bye"' > xyz.tex

--
\r = ASCII '15
\n = ASCII '12

IMHO, underlying C library (if it has anything to do with it at all) must not treat \r\n as an end-of-line, because I'm on Linux, not Windows.

So please somebody clarify why \r\n is not treated separately.

  • Line ends are normalised: both Linux and Windows line ends should be treated identically so sources are portable. – Joseph Wright Jan 21 '15 at 9:29
  • Welcome to TeX.SX! The developers of Web2C decided for this behavior because it's common that files made on Windows (which uses CR-LF) are ported on a different system. Requiring the user to change the end-of-record delimiters would hinder portability. – egreg Jan 21 '15 at 9:30
3

There are three steps when inputting the line by TeX.

  • The line is read from the file using system dependent readln procedure.
  • The end of line mark (if exists) is removed, the spaces immediately before it (if exist) are removed too and the rest is saved to line buffer.
  • The line buffer is appended by \endlinechar value, if it is nonnegative (ASCII 13 alias ^^M by default). Note that ^^M has catcode 5 by default and this is the reason why the ends of lines behave in TeX as behave.

The system dependent readln procedure is implemnted in web2c TeX in src/texk/web2c/lib/eofeoln.c file where is:

void
readln (FILE *f)
{
    int c;
    while ((c = getc (f)) != '\n' && c != '\r' && c != EOF)
        ;
    if (c == '\r' && (c = getc (f)) != '\n' && c != EOF)
        ungetc (c, f);
}

Note, that the special case CRLF is handled here.

2

TeX relies on a “change file” for system dependent adjustments. One of these adjustments is making TeX aware of what method the operating system uses for marking the end of a record (a line in a text file, speaking less formally).

Different operating systems had very different ideas about this: it could be

  1. CR+LF (for instance, MS-DOS)
  2. CR (for instance, Mac OS before version 10)
  3. LF (for instance, Unix)
  4. nothing, for systems that used fixed length records

In the first implementations of TeX, this was taken literally; for instance OzTeX used CR as record terminator and files coming from DOS systems without conversion could be compiled wrongly, because some DOS editors used LF+CR as record terminator.

In order to enhance portability between different file systems, the developers of Web2C decided to be “record terminator agnostic”. When a file is opened by TeX, its first lines are examined and the record terminator used is determined; then it's changed into LF and passed to TeX for further processing.

If you do

perl -e 'print "\\obeylines a\rb\r\\input zzz \\bye"' > xyz.tex
perl -e 'print "\\obeylines c\nd\\bye"' > zzz.tex

the output of pdftex xyz would be

a
b
c
d

but also the output of pdftex '\catcode`^^L=12 \input xyz' wouldn't be different (note that plain.tex does

\catcode`\^^L=\active \outer\def^^L{\par}

so ^^L would be interpreted as outer \par; by setting its catcode to 12 we tell it's printable, but nothing appears even if \n is used as record terminator in zzz.tex.

So one doesn't have to know what operating system a file comes from: inputting it will do “the right thing”.

Any Web2C based TeX distribution will thus behave the same when fed the same file.

What source file does this? I don't know, but I'm not going to read the complete sources.

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.