# Which TeX procedure removes trailing tabs?

From reading the TeX source, it seems that the `input_ln` procedure is responsible for removing trailing spaces from lines upon input (before category codes are considered). A very short excerpt from the code there is

``````if buffer[last-1]<>" " then last_nonblank:=last;
end;
``````

The test appears to target specifically spaces, and not tab characters. However, doing some tests, I see that TeX also removes tabs (regardless of catcodes). For instance, running

``````echo -e "\\\\catcode9=12\\\\show\\t%" | tex
``````

correctly shows `the character <tab>`, but omitting the `%` there is no output.

So... Which part of TeX removes trailing tabs?

Here's how it goes in texlive (and presumably other web2c distros):

1. The changefile tex.ch removes the Knuth definition of `input_ln`;
2. The underscore gets lost in the conversion of web to c;
3. The resulting C code `#includes` texmfmp.h which does

``````#define inputln(stream, flag) input_line (stream)
``````
4. The function `input_line()` gets defined in texmfmp.c where the trailing whitespace is cut thusly:

``````/* Trim trailing whitespace.  */
while (last > first && ISBLANK (buffer[last - 1]))
--last;
``````
5. `ISBLANK()` is defined in c-ctype.h as

``````#define ISBLANK(c) (isascii (c) && isblank (c))
``````

where `isblank()` means (at least for usual locale settings) "is space or tab".

I'd say this behaviour could be a "bug", in that it deviates from the TeXbook: in chapter 8, page 46, Knuth writes:

TeX deletes any `<space>` characters (number 32) that occur at the right end of an input line. Then it inserts a `<return>` character (number 13) at the right end of the line, except that it places nothing additional at the end of a line that you inserted with 'I' during error recovery. Note that `<return>` is considered to be an actual character that is part of the line; you can obtain special effects by changing its catcode.

Elsewhere, we learn that plain tex sets the catcode for `<tab>` to 10 (space), and makes `\<tab>` the same as `\<space>` (and `\<return>`).

On the other hand, I can imagine the potential for confusion if tabs would not be skipped, here, so it makes some sense to do it.

• Very nice explanation. Your first point implies that Knuth's `input_ln` is different from the one in TeXlive. Does Knuth's remove tabs as well? -- EDIT: well, that comment of mine was silly: I had forgotten that my question specifically mentions that Knuth's `input_ln` only removes spaces. – Bruno Le Floch Sep 18 '12 at 12:50

I think the `get_next` procedure (starting in Section 332 "Getting the next token") takes care of discarding tabs.

Section 343 "Input from external file ..." loads `cur_char` from `buffer` and sets `cur_cmd` to the catcode for `cur_char`; it then advances `loc`, the index into `buffer`, by 1. In Section 344 "Change state if necessary ...," TeX does nothing if none of the cases for the `case` statement match. Therefore, when `state` is `skip_blanks`, and `cur_cmd` is 10 (space), the `othercases` label will match and TeX will jump to the `switch` label at the beginning of Section 343. `loc` will be incremented and the tab (or space) will effectively be discarded.

Section 347 "Handle situations involving spaces, ..." puts TeX into the `skip_blanks` state, and also takes TeX out of `skip_blanks` when non-space characters are encountered.

• I think what you describe is a different step: `skip_blanks` ignores any character which has category code 10. It turns out that tabs normally have this category code, but if you do `\catcode9=12\relax` then `\show<tab>%` and `\show<tab>`, you'll see first `the character ^^I`, then nothing, since `\show` is still looking for an argument. – Bruno Le Floch Aug 15 '12 at 19:43
• I have been looking through the sources and I cannot see where tabs get eaten! Is it possible the implementation of `get` in Web2c has some tab-eating behavior? Otherwise, I think you should send in a bug report :) – Justin Bailey Aug 16 '12 at 19:57
• @BrunoLeFloch From looking at the web2c sources, it looks like `get` is just implemented with `getc` (from C's standard library), which doesn't eat tabs. – Justin Bailey Aug 17 '12 at 15:41
• the behaviour is clearly by design, so no bug report there. This is very mysterious :(. – Bruno Le Floch Aug 19 '12 at 0:39