# TeX macro idioms, or: understanding advanced macros

I often find that when I look at solutions from the experts here, the macro definitions are completely inscrutable to me. Trying to use standard techniques one would use to understand code in most programming languages often isn't helpful; for example, I find that chasing definitions using show can bottom out in something that still doesn't make sense (to me).

Now, @David Carlisle has just very kindly taught me a TeX idiom which suddenly makes a lot of macros make sense. This question is largely motivated by a desire to share that with other people at my level. It's asking for descriptions of any common idioms that:

1. occur frequently in packages and answers from experts here
2. are doing something that isn't obvious from just understanding the commands. (For example, lccode used to something that has nothing to do with lowercase characters.)

The idiom I've just learnt is going into an answer below, and should help illustrate what I mean.

• \usepackage{trace} ... \traceon :-) – Joseph Wright Dec 8 '12 at 18:34
• I think that the famous \romannumeral expansion trick fits perfectly here. Joseph, you might want to (more or less) copy-paste your blog post here;). – mbork Dec 8 '12 at 19:57
• Related: Cunning (La)TeX Tricks – Scott H. Dec 8 '12 at 21:29
• – Philippe Goutet Dec 9 '12 at 8:48

One of the more frustrating features of TeX's macro-expansion based language is that the control code and the code it controls can interact unexpectedly. This means that if you want a conditional block \ifwhatever...\fi to control some macro \macro that affects stuff after the block, you need to leapfrog \macro to finish the conditional first:

\ifwhatever
\expandafter\macro
\fi
<stuff taken as an argument by \macro>


You see, the \expandafter eats the \fi before \macro can. Using this trick, you can write some code that conditionally executes some following code. Say you have a conditional \ifwhatever, and you want to execute <code> if it is true, but not if it is false. Then you can write (here I am assuming \makeatletter is active):

\ifwhatever
\expandafter\@firstofone
\else
\expandafter\@gobble
\fi
{<code>}


This can be generalized if you want to select among two different pieces of code, which by the way is how the LaTeX-style conditionals \ifwhatever{<true code>}{<false code>} work:

\ifwhatever
\expandafter\@firstoftwo
\else
\expandafter\@secondoftwo
\fi
{<code 1>}
{<code 2>}


I wouldn't advise doing this with more than one conditional, though: if you want to execute <code> when \ifwhatever or \ifnot is true, you have to write:

\ifwhatever
\expandafter\@gobble
\else
\ifnot
\expandafter\expandafter\expandafter\@firstoftwo
\else
\expandafter\expandafter\expandafter\@gobble
\fi
\fi
{<code>}


This profusion of \expandafters is to skip \@firstoftwo or \@gobble and expand first one token, then another after it, before going back to expand the macro you actually want once there is nothing in its way.

Of course, the technique of using multiple \expandafters to skip ahead is itself an idiom. For example, you can expand the first token in a group once:

\expandafter{\macro}


or twice:

\expandafter\expandafter
\expandafter{\macro}


or three times:

\expandafter\expandafter\expandafter\expandafter
\expandafter\expandafter
\expandafter{\macro}


and you see the pattern. I may as well explain why this works. When the first \expandafter goes off, it expands in turn the third, then the first in the next row, then the first in the following row, and finally, \macro. The intervening tokens are not expanded in this "run", but just stuck back in order. The effect is that every other column of \expandafters vanishes, \macro is expanded once, and then TeX starts expanding again with what is now the first \expandater, which was the second. The configuration has become the one for expanding \macro twice, and so we end up expanding it three times.

lccode + lowercase trick

@David Carlisle's trick is used when you want to redefine the behaviour of a standard (say) ASCII character, such as - or \n. This is useful in e.g. verbatim-like environments, where you might want a newline to be processed in a non-standard way -- and I'm going to stick with that example for the rest of this post.

TeX lets you make any such character into a macro using \catcode and \active, which gives you a easy way of changing the behaviour of \n. The problem is that as soon as you've done that, then every newline in your source code will start behaving differently, which makes it rather hard to actually write anything. So you need some way of being able to use 'inactive' newlines in your source while treating any newlines typed by your user as 'active'.

As I understand it, the trick works as follows:

1. You pick an arbitrary active character which you don't need to use in your macro. ~ is a good choice because it is always active. If you need more, you make whatever characters you like active.

2. You "tell" TeX that the lowercase version of (say) ~ is \n, like this:

\lccode~=\^^M


This is of course nothing to do with actual lowercase text.

3. You wrap your entire macro in \lowercase{...}. This means that whenever you use ~, TeX converts it into \n. And because ~ is active, it converts it into an active \n. This means, in particular, that you can redefine the behaviour of \n like this:

\lowercase{
...
\def~{...}
...
}


At the same time, normal newlines in your macro (like the one before the final }) remain inactive, and so are processed by TeX in the normal fashion.

[At least, that's it as I best understand it... ]

• The explanation is correct; but one should be aware that all character tokens will be affected by \lccode which, conversely, doesn't touch symbolic tokens (roughly speaking, those that start with a backslash). – egreg Dec 8 '12 at 20:31

I suppose we should have mention of

{\ifnum0=}\fi

\ifnum0={\fi}


The brace group constructs that are endemic in packages related to tabular and related alignment constructs.

To understand these constructs it is best to start in the middle: the backtick construct { returns the character code of { which is 123 and in particular it is not 0 so

\ifnum0=}\fi


is like \iffalse\fi and expands to nothing. Similarly

\ifnum0={\fi


Thus the first construct expands to an explicit { and the second construct expands to an explicit }

there are simpler constructs that expand to explicit braces notably

{\iffalse}\fi

\iffalse{\fi}


However as we shall see they do not work in the tabular constructs the way that is needed.

Now consider a simple halign construct (just using plain TeX for simplicity) that defines centred columns and (like LaTeX) wants to use a local definition of \\ to end the row.

\def\tabc{%
\leavevmode
\vtop\bgroup\let\\\cr

\def\endtabc{%
\crcr\egroup
\egroup}

\tabc
1 & 2 & 3\\
aaa & bbb & ccc
\endtabc

\bye


This apparently works fine and produces

Then someone decides that they want to nest tables …

Replacing the 1 in the first cell by a nested 2x2 table:

\tabc
\tabc a & b \cr x & y \endtabc & 2 & 3\\
aaa & bbb & ccc
\endtabc


This fails with the error message

! Missing } inserted.
<inserted text>
}
\endtemplate

\tabc ->\leavevmode \vtop \bgroup \let \\
\cr \halign \bgroup &\quad \hfil \i...
l.14 \tabc
a & b \cr x & y \endtabc & 2 & 3The hint to understanding the error message is the \\  at the end of the line has cause \endtemplate to be read. What has happened is that the intended meaning of locally defining \\  has not happened: as soon as TeX saw the \\  (which is \let to \cr already because of the outer table, the cell of the outer table ended, the the \let then defined \unskip to be \hfil, the quad made a space, but then the cell ended in while the group started by \halign\bgroup is still open, so an error was generated.


One work-around is to nest the inner table in explicit braces:

\tabc
{\tabc a & b \cr x & y \endtabc} & 2 & 3\\
aaa & bbb & ccc
\endtabc


works as intended and produces

Sometimes you see table macros with similar requirements that nested instances be protected by explicit braces. But normally it is better to make the macros safe for nested use.

Due to the way that macro definitions are parsed, it is not possible to simply add a { to the definition of \tabc. Also the usual implicit brace \bgroup does not work here, note that the redefinition of \\ is already inside a group started by \bgroup but TeX's table cell scanner does not see that in the right way. It turns out that the construct at the start is exactly what is required. If we define the table macro as below then the nested table works as expected without the need for extra braces in the document: Once the inner \tabc is expanded in the first cell of the outer table the {\ifnum0=}\fi is expanded and TeX will then not close the cell of the outer table until the matching end group is seen, even if it sees a & or \cr token that would normally end the cell.

\def\tabc{%
{\ifnum0=}\fi
\leavevmode
\vtop\bgroup\let\\\cr

\def\endtabc{%
\crcr\egroup
\egroup
\ifnum0={\fi}}

\tabc
\tabc a & b \cr x & y \endtabc & 2 & 3\\
aaa & bbb & ccc
\endtabc

\bye

• I think you missed the promised explanation of why \iffalse won't work in this situation, which I have always wondered about. I think I understand based on what you wrote: is it because when expanding \iffalse}\fi, TeX actually "counts" the } and therefore doesn't shield the later occurrence of \\  because it exits the group for the purposes of nesting \haligns? Whereas in \ifnum}\fi, the brace is not actually read as a brace, but as its character code, so doesn't contribute to the close-brace count. – Ryan Reich Dec 9 '12 at 1:19

One of TeX's core features, delimited arguments/parameters, doesn't seem to be widely used in LaTeX user code, so some of the common idioms related to them may not be immediately clear when you encounter them in package code. The syntax and argument reading for those parameters is explained in this answer, so I'd like to concentrate on some examples of more or less common idioms using here.

Special argument delimiters

Special argument delimiters are usually implemented using delimited parameters, most notably optional parameters [...]. The typical pattern for LaTeX macros with optional arguments looks like

\def\foo{\@ifnextchar[{\foo@aux}{\foo@aux[default]}}
\def\foo@aux[#1]{Foo with parameter #1'}


where \foo tests if the following character is [ and then passes over to \foo@aux which uses delimited parameters to read the following optional argument. Arguments with delimiters like (...) or <...> work similarly.

Skipping code until end marker

A common idiom is to use delimited parameters to skip parts of the code. As explained in Ryan Reich's answer, sometimes we have to get rid of unwanted tokens like an extra \else or \fi. The answer presents one approach to deal with them.

Another one is to define helper macros that use delimited parameters to skip through the conditional up to the final \fi, and then move the corresponding parameter after the \fi that is required to end the conditional. Using the macros

\def\handletrue#1\else#2\fi{\fi#1}
\def\handlefalse#1\fi{\fi#1}


we get the following expansions series for different conditions:

    \iftrue \handletrue{\textbf}\else \handlefalse{\textit}\fi{foo}
--> \handletrue{\textbf}\else \handlefalse{\textit}\fi{foo}
--> \fi\textbf{foo}
--> \textbf{foo}

\iffalse \handletrue{\textbf}\else \handlefalse{\textit}\fi{bar}
--> \handlefalse{\textit}\fi{bar}
--> \fi\textit{bar}
--> \textit{bar}


thus correctly outputting "foo"/"bar". The lazylist package uses something similar:

\def\TeXif#1{#1\gobblefalse\else\gobbletrue\fi}
\def\gobblefalse\else\gobbletrue\fi#1#2{\fi#1}
\def\gobbletrue\fi#1#2{\fi#2}


A second use case of this idea are multiway branches (switch/case). TeX doesn't provide this control structure natively, so usually nested conditionals are used to avoid redundant tests in the code. Again, delimited parameters can be useful here. l3regex, for example, defines macros \__l3regex_break_true:w and \__l3regex_break_point:TF (renamed below for readability) which can be used to emulate a multiway branch:

\def\breaktrue#1\breakpoint#2#3{#2}
\def\breakpoint#1#2{#2}

\def\testchar{y}
\if\testchar x
An x was found.\breaktrue
\fi
\if\testchar y
An y was found.\breaktrue
\fi
\if\testchar z
An y was found.\breaktrue
\fi
\breakpoint{\fi}{No character matched.}


Whenever an \breaktrue is found, it will skip anything up to the next \breakpoint, in particular all the remaining alternative tests. If any of the tests was successful, the first parameter of \breakpoint is inserted into the token stream (here the skipped \fi for the successful test must be reinserted), otherwise a common "else branch" is used.

Splitting token lists

Delimited parameters are also often used for testing if a certain separator token occurs in a list of tokens, such that the sublists before and after that token can be handled in a special way. As an example, say we want to replace all occurences of \textbf{...} in a token list by \textit{...}. We start with a helper macro \@replacebold that uses delimited parameters to split a token list, passed to it without surrounding braces:

\def\@replacebold#1\textbf#2#3\@end{...}


\textbf here delimits any tokens that occur before the first occurence of this control sequence in the list (#1), \@end is the delimiter for all the rest. As \textbf takes a single argument, we add another parameter #2 to match on this argument. Finally, #3 matches anything after the bold part. With this decomposition we can easily replace the bold text with an italic one and handle the rest of the list recursively.

Delimited parameters are mandatory in the actual argument if used in the macro definition, so we still need to handle the case in which \textbf{...} doesn't occur in the argument at all (which is also the case at the end of recursion). For this we already pass \@replacebold a single, faked call of \textbf which will match in any case:

\def\replacebold#1{\@replacebold#1\textbf{}\@end}


If this final occurrence is reached when calling \@replacebold, #3 will be empty as \textbf{} is immediately followed by \@end. We can use this fact for our test if another recursion step is necessary. Putting all pieces together, we might define \replacebold as follows:

\def\replacebold#1{\@replacebold#1\textbf{}\@end}
\def\@replacebold#1\textbf#2#3\@end{%
#1%
% Real code should use a saner test for emptiness here
\ifx\relax#3\relax%
\expandafter\@gobble
\else
\textit{#2}%
\expandafter\@firstofone
\fi
{\@replacebold#3\@end}%
}


This idiom is used quite frequently in many variations. LaTeX's \@for loop macro, for example, uses a more sophisticated version to split a comma-separated list into its fields. You might also enjoy trying to find out how the various \quark_if_recursion functions form expl3 use this pattern for looping.

Other uses

Paragraphs as arguments: When TeX finds an end-of-line character in the input text that is immediately followed by another such character, it converts both into a \par token. This token can also be used as a delimited parameter. For example, the macro

\def\warning#1\par{%
\textbf{Attention!} {\itshape #1\par}
}


could be used like \warning Do not use ... and will then read all text up to the end of the current paragraph, i.e. the next explict \par or empty line in the input file.

Simple loops: Plain TeX defines a simple loop macro \loop ... \repeat which uses a delimiter to mark the end of the loop body:

\def\loop#1\repeat{\def\body{#1}\iterate}
\def\iterate{\body \let\next\iterate \else\let\next\relax\fi \next}
\let\repeat=\fi % this makes \loop...\if...\repeat skippable

\count0=1
\loop
... use \count0 here ...
\ifnum\count0<10

Use without parameter: Plain TeX also has a special use of delimiters in an internal macro to make sure that the command name passed to \newif actually starts with the letters i f. The macro \@if then gobbles those letters and builds a control sequence from the rest of the letters:
\def\@if#1#2{\csname\expandafter\if@\string#1#2\endcsname}
{\uccode1=i \uccode2=f \uppercase{\gdef\if@12{}}}