I started reading TeX by Topic by Victor Eijkhout. He starts from describing four engines:
The input processor. This is the piece of TeX that accepts input lines from the file system of whatever computer TeX runs on, and turns them into tokens. Tokens are the internal objects of TeX: there are character tokens that constitute the typeset text, and control sequence tokens that are commands to be processed by the next two levels.
The expansion processor. Some but not all of the tokens generated in the first level—macros, conditionals, and a number of primitive TeX commands—are subject to expansion. Expansion is the process that replaces some (sequences of) tokens by other (or no) tokens.
The execution processor. Control sequences that are not expandable are executable, and this execution takes place on the third level of the TeX processor. One part of the activity here concerns changes to TeX’s internal state: assignments (including macro definitions) are typical activities in this category. The other major thing happening on this level is the construction of horizontal, vertical, and mathematical lists.
The visual processor. In the final level of processing the visual part of TeX processing is performed. Here horizontal lists are broken into paragraphs, vertical lists are broken into pages, and formulas are built out of math lists. Also the output to the dvi file takes place on this level. The algorithms working here are not accessible to the user, but they can be influenced by a number of parameters.
The author makes the following statement: “For many purposes it is most convenient, and most insightful, to consider these four levels of processing as happening after one another, each one accepting the completed output of the previous level.”
I don't understand how “each one accepting the completed output of the previous level” is possible, since the expansion processor (level 2) needs macro definitions that are assigned by the execution processor (level 3). I sent an email to the author of the book with a question how it is possible, there is no reply. Can somebody explain?
I think I can accept the book's model, if we reconsider what completed means. The author later gives an example:
Given above the call
This happens because the space was removed after
\DoAssign by the input processor. In this sense it was completed before the next stage. Instead of: “
\DoAssign gets processed,
\count42 is assigned
800, and then the input continues from the last