Does macros get applied to the source code before compilation?

Question

I wonder if this is true.

1. The latex compiler first run though the document, and search for all the macro declarations.

2. the compiler then go though the document, apply each macro as a string substitution.

3. The compiler compile the document that has applied all the macros.

If this is true, then are there ways to get the output of the second step?

Answer 1

The (La)TeX compiler works with only one pass. (Not to be confused with multiple compiler runs, which might be required to process material which is required at the begin of the document but only defined later, like the entries in the table of contents.)

Most control sequences (the \name things) are macros and hold more macros. These are simply expanded to their replacement text as soon TeX sees them. The replacement text is then processed more or less as it was part of the input file, and the containing macro are again expanded. This happens as long something non-expandable is found, e.g. some characters or a TeX primitive (like an assignment, e.g. \def) which must be executed by the compiler. After the execution the compiler keeps expanding the rest of the input, until it again reaches a non-expandable token. There are also some other steps done when a page is filled and shipped out etc. Important to note is that all steps happen in one pass, i.e. the first part of a document is expanded, executed, typeset and already flushed out to the output file before the end of the document is even read.

A better explanation, which also describes the "organs" of TeX where the different processing steps are done, is given by the freely online available book "TEX for the Impatient", p.16:

How TEX works

In order to use TEX effectively, it helps to have some idea of how TEX goes about its activity of transmuting input into output. You can imagine TEX as a kind of organism with “eyes”, “mouth”, “gullet”, “stomach”, and “intestines”. Each part of the organism transforms its input in some way and passes the transformed input to the next stage. The eyes transform an input file into a sequence of characters. The mouth transforms the sequence of characters into a sequence of tokens, where each token is either a single character or a control sequence. The gullet expands the tokens into a sequence of primitive commands, which are also tokens. The stomach carries out the operations specified by the primitive commands, producing a sequence of pages. Finally, the intestines transform each page into the form required for the .dvi file and send it there. These actions are described in more detail in Section 4 under “anatomy of TEX” (p. 46).

Answer 2

No, it doesn't work like this. There's only one pass and macro expansion is performed until unexpandable tokens remain. A macro can be recursive (and many indeed are), but also for "normal" macros there is usually more than one substitution (better, expansion). Moreover macros can even redefine other macros as part of their work, even themselves.

Answer 3

As mentioned by egreg, Martin and Bruno, the answer is no.

There are ways to show how macros expand in TeX.

In (plain) TeX, you can set \tracingcommands, \tracingmacros, \tracingoutput, etc. to trace how TeX works. See chapter 34 of TeX by Topic for more explanations. For example:

\tracingmacros=1
\tracingcommands=1
\def\foo{foo}
\def\bar{(\foo)}

\bar

\bye

You'll get the information below in the .log file:

{vertical mode: \def}
{blank space  }
{\def}
{blank space  }
{\par}

\bar ->(\foo )
{the character (}
{horizontal mode: the character (}

\foo ->foo
{\par}

\bye ->\par \vfill \supereject \end 
{vertical mode: \par}
{\vfill}
......

In LaTeX, you can also use trace package. For example:

\documentclass{article}
\usepackage{trace}
\begin{document}
\traceon
\def\foo{foo}
\def\bar{(\foo)}

\bar
\traceoff
\end{document}

You'll get these output:

{into \tracingonline=1}
{\def}
{changing \foo=undefined}
{into \foo=macro:->foo}
{blank space  }
{\def}
{changing \bar=macro:->\mathaccent "7016\relax }
{into \bar=macro:->(\foo )}
{blank space  }
{\par}

\bar ->(\foo )
{the character (}
{horizontal mode: the character (}

\foo ->foo

Answer 4

Here's the same answer as others have given, but from a different perspective.

As a new LaTeX user, it's natural to think of TeX as being a markup language consisting mostly of actual text with occasional formatting directives (\textbf, math mode $...$, and so on) and macros that stand for long bits of text that should be abbreviated. Indeed, LaTeX goes to some trouble to present this appearance to the user.

This is not how things actually are. A directive such as \mymacro might be defined in such a way as to completely transform the text appearing after it, or even change the way TeX reads its input. It is better to imagine that everything you write is a command for telling TeX how to typeset: a letter tells TeX to typeset itself, but macros (and other things even more horrible) can either tell it to typeset something else, or tell it to typeset things in a different way entirely.

From this perspective, there is no way that TeX could do a simple "substitution run" on its input: macros can mean something far more serious than simple text replacement and there is no way to tell until they are expanded in order. Furthermore, the result of that expansion may be typesetting commands like \hbox that don't mean expand into plain text at all and can never be eliminated. As an example of this, here is what the command \TeX means:

T\kern -.1667em\lower .5ex\hbox {E}\kern -.125emX

It's a "macro" as you consider it, but it doesn't just say "write the letters TeX", but rather describes how close they must be written (\kern) and at what height (\lower), so as to draw the TeX logo. That just doesn't have any meaning in a plain text file.

You might be interested in the question Can LaTeX be persuaded to produce text output?, though I'm afraid the answers are not encouraging.

Really, the answer to your question is that the "full expansion" of a TeX file is a finished document, since that is the minimal format that can meet the requirements expressed in the input. If you know that the document only contains plain text (no pictures or symbols), then pdftotext may give you a reasonable approximation to what you want.