catcode 12 spaces

Question

I would like to understand TeX's behaviour when the catcode of the space character is set to 12.

The following two example input files differ in just one position: where the first has the character 4 the second has the character (space). Both 4 and have catcode 12 in these examples. I therefore expected the typeset output of the second example to be identical to that of the first, with the only difference that the typeset '4' would be replaced by a typeset 'space'. In other words, I expected the second example to have three typeset lines, the second line consisting of five of those "open boxes" that cmtt10 uses to typeset spaces. Instead, TeX ignored the second line, even though it is not empty in the input file (it consists of five catcode 12 spaces). What is going on here?

\catcode`\ =12
\obeylines
\tt
 een  twee   drie
    4
hoedje van papier
\bye

\catcode`\ =12
\obeylines
\tt
 een  twee   drie
     
hoedje van papier
\bye

Answer 1

On page 46 of the TeXbook, third double dangerous bend, we find

TeX deletes any ⟨space⟩ characters (number 32) that occur at the right end of an input line. Then it inserts a ⟨return⟩ character (number 13) at the right end of the line, except that it places nothing additional at the end of a line that you inserted with ‘I’ during error recovery. Note that ⟨return⟩ is considered to be an actual character that is part of the line; you can obtain special effects by changing its catcode.

The first sentence explains what you get. Note that tokenization hasn't even started yet: when needed, TeX puts a record in a buffer, doing the steps outlined in the text above; only after this it starts tokenizing and, indeed, the text above specifies characters number 32.

The ⟨return⟩ character actually is the one with character code \endlinechar which under \obeylines is still code 13, but active (category code 13). This is explained in the fifth double dangerous bend on page 48. Anyway it's inserted after the deletion of the space characters, so this doesn't help in your situation.