341

So I've read a few times about this, the “moving arguments” and all that. But what does it really mean? Could someone explain in a few words what is going on at the TeX level, what is the difference between fragile and robust commands? Furthermore, when do we need to use \protect and why?

1
  • 13
    This is a good question. I'm having difficulty formulating a succinct answer to it, so I guess I'll let someone else have a go at it. Basically, a fragile command is one that does not behave properly when expanded, but not executed. LaTeX uses macros like \protected@edef which cause \protect\foo in the replacement text to expand to itself in contrast to \edef which would (normally) cause \protect\foo to essentially expand to \relax followed by the expansion of \foo.
    – TH.
    Oct 31, 2010 at 8:27

5 Answers 5

300

The key concept here is that, when TeX handles its input, it is doing two distinct things, called expanding and executing stuff. Normally, these activities are interleaved: TeX takes a token (ie, an elementary piece of input), expands it, then executes it (if possible). Then it does so with the next token. But in certain circumstances, most notably when writing to a file, TeX only expands things without executing them (the result will most probably be (re-expanded and) executed later when TeX reads the file back). Some macros, for proper operation, rely on something being properly executed before the next token is expanded. Those are called "fragile", since they work only in the normal (interleaved) mode, but not in expansion-only contexts (such as "moving arguments" which often means writing to a file).

That's the general picture. Now let's give a "few" more details. Feel free to skip to "what to do in practice" :)

Expansion vs execution

The distinction between expansion and execution is somewhat arbitrary, but as a rule of thumb:

  • expansion changes only the input stream, ie "what TeX is going to read next";
  • execution is everything else.

For example, macros are expandable (TeX is going to read their replacement text next), \input is expandable (TeX is going to read the given file next), etc. \def is not expandable (it changes the meaning of the defined macro), \kern is not expandable (it changes the content of the current paragraph or page), etc.

How things can go wrong

Now, consider a macro \foo:

\newcommand\foo[1]{\def\arg{#1}\ifx\arg\empty T\else F\fi}

In normal context, \foo{} gives T and foo{stuff} gives F.) In normal context, TeX will try to expand \def (which does nothing) then execute it (which removes \arg{#1} from the input stream and defines \arg) then expand the next token \ifx (which removes \arg\empty and possibly everything up to, but not including, the matching \else from the input stream), etc.

In expansion-only context, TeX will try to expand \def (does nothing), then expand whatever comes next ie the \arg. At this point, anything could happen. Maybe \arg is not defined and you get a (confusing) error message. Maybe it is defined to something like abc, so \foo{} will expand to \def abc{} F. You'll not get an error when writing this to the file, but it will crash when read back. Perhaps \argis defined to \abc, then \foo{} will expand to \def\abc{} F. Then you get no error message either when writing nor at readback, but not only you get F while you're expecting T, but also \abc is redefined, which can have all kinds of consequences if this is an important macro (and good luck for tracking the bug down).

How protection works

Edited to add (not in the original question, but someone asked in a comment): so how does \protect works? Well, in normal context \protect expands to \relax which does nothing. When a LaTeX (not TeX) command is about to process one of its arguments in expansion-only mode, it changes \protect to mean something based on \noexpand, which avoids expansion of the next token, thus protecting it from being expanded-but-not-executed. (See 11.4 in source2e.pdf for full details.)

For example, with \foo as above, if you try \section{\foo{}} chaos ensues as explained above. Now if you do \section{\protect\foo{}} then when LaTeX prints the section title it's in normal (interleaved) mode, \protect expands to \relax, then \foo{} expands-and-executes normally and you get a big T in your document. Before LaTeX writes your section title to the .aux file for the table of contents, it changes \protect to \noexpand\protect\noexpand, so \protect\foo expands to \noexpand\protect\noexpand\foo and \protect\foo is written to the aux file. When that line of the aux file is moved to the toc file, LaTeX defines \protect to \noexpand, so just \foo gets written to the toc file. When the toc file is finally read in normal mode, then and only then \foo is expanded-and-executed and you get a T in your document again.

You can play with the following document, looking at the contents of the .aux and .toc files without and with \protect. Notes: (1) you want to run pdflatex manually on the file, as opposed to latexmk or your IDE which might do multiple runs at once, and (2) you will need to remove the toc file to recover after trying the non-\protected version.

\documentclass{article}
\newcommand\foo[1]{\def\arg{#1}\ifx\arg\empty T\else F\fi}
\begin{document}
\tableofcontents
\section{\foo{}} % first run writes garbage to the aux file, second crashes
%\section{\protect\foo{}} % this is fine
\end{document}

Fun fact: the unprotected version fails in a different way (as explained above) if we replace every occurrence of \arg with \lol in the definition of \foo.

Which macros are fragile

This was the easy (read: TeXnical, but well-defined) part of your question. Now, the hard part: when to use \protect? Well it depends. You cannot know whether a macro is fragile or not without looking at is implementation. For example, the \foo macro above could use an expandable trick to test for emptyness and would not be fragile. Also, some macros are "self-\protecting" (those defined with \DeclareRobustCommand for example). As Joseph mentioned, \( is fragile unless you (or another package) loaded fixltx2e. (As a rule of thumb, most mathmode macros are fragile.) Also, you cannot know whether a particular macro tries to expand-only its arguments, but you can at least be sure all moving arguments will be expanded-only at some point.

What to do in practice

So, my advice is: when you see a weird error happening in or near a moving argument (ie a piece of text that's moved to another part of the document, like a footnote (to the bottom of the page), a section title (to the table of contents), etc), try \protecting every macro in it. It solves 99% of the problems.

(This can make you a hero when applied to a colleague's article, due today and "mysteriously" crashing: look at their document for a few seconds before you see a math formula inside a \section title, say "add a \protect here", then go back to work and let them call you a wizard. Cheap trick, but works.)

15
  • 7
    I accepted this answer mostly because of the first paragraph that gives a neat and succinct explanation. The rest of the details in the following paragraphs are also quite helpful to figure out more of the details. Nov 1, 2010 at 11:48
  • 2
    I agree that the first paragraph is very clear. However i'd still want to see a simple complete example that I could feed into TeX or LaTeX so I could see how \protect works.
    – Jay
    Jun 19, 2012 at 12:20
  • 1
    I get the impression that \def\arg{#1}\ifx\arg\empty E\else F\fi (which doesn't compile) was intended to be a test of whether the argument passed to \arg was empty or not, and produce T if it is and F if it isn't...but I'm not aware of a way to use either \if or \ifx to do that, I would use \ifthenelse from the ifthen package, i.e. \def\isargempty#1{\ifthenelse{\equal{#1}{}}{T}{F}}. Aug 7, 2012 at 14:27
  • 3
    @TheodoreMurdock \def\arg{#1}\ifx\arg\empty E\else F\fi is the definition of \foo, so it's intended to be a test of whether the argument passed to \foo is empty (not \arg as you say).
    – ruds
    Feb 2, 2013 at 6:18
  • 1
    But how do I use this definition for \foo? If I try it as given, the error reads Illegal parameter number in definition of \arg. How do I make it work?
    – MickG
    Dec 30, 2013 at 14:42
108

The key concept here is expansion. I'll take as an example a hypothetical function \foo which is 'fragile', used in the argument of \section:

\section{Some text \foo[option]{argument}}

When LaTeX processes the \section macro, it does a number of things. One of those is to write the text of the section name to the .aux file. Now, the key point here is that this uses the \write primitive, effectively:

\immediate\write\@auxout{Some text \foo[option]{argument}}

The \write primitive expands its argument in the same way as \edef. However, I've said that \foo is 'fragile'. That means that trying to \edef it will either lead to an error or the wrong result. A classic case for this is any macro with an optional argument: the detection of these cannot be expanded inside an \edef. The other is where something will be numbered based on where it is in the input, which can give bad numbering in the output. See for example http://texblog.net/help/latex/fragile.html for more details on macros which are fragile (but note that the fixltx2e package sorts out some of these).

When you use \protect, it prevents TeX expanding the next token during the \write. So the text is written 'as given' to the .aux file. This of course requires that you know which functions need to be protected. As TH notes, it also needs the correct use of \protected@write or \protected@edef to work correctly. (The way that these macros work is by altering the definition of \protect to achieve the desired effects. So inside \protected@edef, the expansion of \protect is \noexpand\protect\noexpand, for example.)

The macro \DeclareRobustCommand is available in LaTeX2e. This adds some automatic protect into the macro itself, so that \protect is not needed. This again works inside a \protected@write situation.

This is good, but a better method is e-TeX's \protected system:

\protected\def\foo....

Macros defined in this way are not expanded inside an \edef or a \write at all, as the engine itself knows to leave them alone. This is the approach taken by etoolbox and xparse for defining truly robust macros. Macros which are engine-protected don't rely on LaTeX2e's mechanisms at all, so are safe inside a plain \edef.

2
  • So these two commands are equivalent (with or without redundant command brackets): \DeclareRobustCommand{\setlanguage}[1] and \protected\def{\setlanguage}[1]? Also xparse's version \DeclareDocumentCommand{\setlanguage}{O{}} is equivalent? All ignore any previously defined commands...all are "robust" (which I seem to understand as meaning "protected from expansion") Sep 7, 2015 at 6:52
  • 1
    @macmadness86 No. \DeclareRobustCommand makes LaTeX2e robust commands, which as 'safe' inside \protected@edef and so on but not inside a raw \edef. \protected\def... makes an engine-protected command, which never expands in an \edef-like context. xparse uses \protected internally so its commands are engine-protected.
    – Joseph Wright
    Sep 7, 2015 at 7:28
45

This question is perhaps best answered by an example. Consider the fragile command \title. Here are the relevant definitions from latex.ltx:

\def\title#1{\gdef\@title{#1}}
\def\@title{\@latex@error{No \noexpand\title given}\@ehc}

Now imagine that you include \title{This is the title} in the argument of a command that first executes this argument, then writes it to the aux file, or otherwise “moves” it. We can run an interactive experiment to see what would happen, using \edef to immediately see the result:

; latex
This is pdfTeX, Version 3.1415926-1.40.11 (TeX Live 2010)
 restricted \write18 enabled.
**\relax
[…]
*\title{This is the title}

*\edef\foo{\title{This is the title}}

*\show\foo
> \foo=macro:
->\gdef This is the title{This is the title}.
<*> \show\foo

As you can imagine (try it if you must), actually executing \foo will not work as you might have expected before running this experiment. The problem, of course, is that \@title got expanded.

To find out how \protect works, you could do worse than running texdoc source2e and looking at section 11.4, Robust commands and protect.

4
  • 11
    The general point here (that any macro which performs an assignment is fragile) is one that we're trying to address in the LaTeX3 work. There, the rule is simple: every macro that performs an assignment (or uses one which does) has to be e-TeX protected. That leads to two distinct classes of macro: fully expandable ones and protected ones. The idea is that this type of ambiguity should be avoided.
    – Joseph Wright
    Oct 31, 2010 at 10:11
  • @HaraldHanche-Olsen When you say "or otherwise 'moves' it", do you mean "or in other words 'moves' it"? I was just trying to understand people when they use the word "move", which seems like a strange term to use when referring to the process of writing to a file. Jul 19, 2016 at 11:17
  • 1
    @macmadness86 That was nearly six years ago – hard to say exactly what I meant at the time! But I think I did mean otherwise. The word “move” may seem strange, but recall the point of LaTeX: Typesetting stuff in a file. Most of the time, what is in the file is typeset in the same order, but sometimes, data from one part of the input file gets typeset in a totally different part than nearby data from the input file. Hence it is “moved” from its natural location. The most common way is indeed to write it to the aux file, so it can be read back in during the next run … Jul 19, 2016 at 22:10
  • … and probably saved in a command or token register, in order to be typeset later. But other times, the aux file is bypassed and data is just saved for later use. So that must be what I was thinking of when I said “otherwise”. Perhaps. Jul 19, 2016 at 22:12
7

Just for comparison: OpTeX doesn't have "fragile" and "robust" commands. The titles of sections, subsections etc. are read in verbatim mode, when catcodes of all special TeX characters are 12. It means that the data about titles are treated inside macros as they are (verbatim). When we want to print them (in the section header or in the headline) then \scantextokens primitive is used. In other words: tokenization is done when the parameter is printed, not when it is scanned.

If we want to write the parameter to the .ref file then such titles are written verbatim. When the .ref file is read again (in order to crate table of contents) then the titles are read in verbatim mode and when it is printed into the TOC line then \scantextokens is used again.

We are using \protected\def if we want to protect such macros against \edef but not because we want to use it in titles and to write it to the .ref file.

When we declare \verbchar` then we can create a title, for example:

\secc Something about the `{` character in \TeX

The title text is scanned in verbatim mode a written to the .ref file as is. It is correctly printed in the section header, in the header line, in the table of contents and in the PDF outline. Something similar in LaTeX:

\subsection {Something about the \verb|{| character in \TeX}

is impossible.

LaTeX doesn't adopt the concept mentioned here because it is burdened by the past. Moreover, it still uses very old concept with \protect macro although we have \potected\def and \detokenize for 30 years.

4

Some remarks.

Note on old-style robust commands.

Because of historical reasons (?), some commands are "old-style robust" (usually those that are written back before e-TeX is available) and some are "new-style robust" (uses e-TeX's \protected mechanism). see also this question

You can tell which one is old-style robust and which one is new-style robust for example:

$ latexdef textbf

\textbf:
macro:->\protect \textbf  


\textbf :
\long macro:#1->\ifmmode \nfss@text {\bfseries #1}\else \hmode@bgroup \text@command {#1}\bfseries \c
heck@icl #1\check@icr \expandafter \egroup \fi 

if there's \protect, then it's old-style robust.

There are only a few new-style robust commands available:

$ latexdef lfloor

\lfloor:
\protected macro:->\delimiter "4262304 

$ latexdef AddToHook

\AddToHook:
\protected macro:->\__cmd_start:nNNnnn {mo+m}\AddToHook  \AddToHook code {\__cmd_grab_m_1:w \__cmd_g
rab_D:w []\__cmd_grab_m_long:w }{}{}

you can see them with \protected.

The problem is, if some command is old-style robust then it cannot be used inside \write or \edef.

Instead, there's \protected@write and \protected@edef respectively. There's no \protected@immediate@write however, you need to define one yourself.

Commands such as \hyphenation:

$ latexdef \hyphenation

\hyphenation:
\hyphenation

are "robust" (unexpandable) because they're TeX primitive, so they're safe everywhere.

For user-defined commands, \NewDocumentCommand results in new-style robust command, (almost) everything else results in old-style robust commands.

$ latexdef -o '\DeclareRobustCommand\test{hello}' test  #% this is old-style robust because it has "\protect"

\test:
macro:->\protect \test  


\test :s?
\long macro:->hello

$ latexdef -o '\newcommand\test{hello}\MakeRobust\test' test  #% identical to the above

\test:
macro:->\protect \test  


\test :
\long macro:->hello

$ latexdef -o '\NewDocumentCommand\test{}{hello}' test  #% this is new-style robust because it has "\protected macro"

\test:
\protected macro:->\__cmd_start_expandable:nNNNNn {}\test  \test  \test code ?{}

$ latexdef -p etoolbox -o '\newrobustcmd\test{hello}' test  #% this is new-style robust

\test:
\protected\long macro:->hello

$ latexdef -o '\newcommand\test{hello}' test  #% this is not robust

\test:
\long macro:->hello

Not all commands are either fragile or robust.

There are the following kinds of commands which we don't say it's fragile or robust:

  • \verb or similar commands, which never works inside a command argument regardless of \protect.

    (remark: These commands changes the catcode of some characters. You need cprotect or similar, or just switch to equivalent non-catcode-changing commands such as \texttt)

  • fully expandable commands, that is, commands that is intended to "return a result" instead of typeset/execute something.

    This function will always work regardless of whether it's in an expansion-only or an execution context.

The definition.

(in "general overview" terms, not in terms of TeX primitives.)

The answer above defines fragile command well.

Some macros, for proper operation, rely on something being properly executed before the next token is expanded. Those are called "fragile", since they work only in the normal (interleaved) mode, but not in expansion-only contexts (such as "moving arguments" which often means writing to a file).

In other words, a command is fragile if

  • it works in typesetting context
  • it does not work as intended when used in an expansion-only context.

A robust command, instead

  • works in typesetting context
  • does not get expanded at all in expansion-only context.

(remember that fully expandable commands are neither fragile or robust.)

Every fragile commands can be made robust.

So in principle, making a command robust is not very hard -- just delay expanding it until it's in an execution context.

(for comparison, not every unexpandable commands can be made expandable.)

If you get a "mysterious error" with some LaTeX built-in command, it's because e.g. the LaTeX team didn't make it robust, most likely because of historical reason.

(a few decades ago it takes some additional memory bytes to robustify commands, and machines have limited memory. Nowadays most commands are already made robust, so this should hopefully be rare.)

If it's your own command, use \DeclareRobustCommand or \NewDocumentCommand instead of \def or \newcommand would hopefully fix the issue.

Or if it's defined with some other libraries such as tcolorbox's \newtcbox where \NewDocumentCommand is not an option, there's \MakeRobust (see an example in the answer linked above. Documentation no idea where, but it's a LaTeX built-in), or etoolbox's \robustify.

Why must \section fully expand its argument?

Or: wouldn't it be simpler if \section just write the content to the file unexpanded, then execute it later?

Previously, I think it wasn't possible. (at least without some additional programming effort, and programming in TeX is difficult.)

Nowadays, in e-TeX it could \detokenize the argument before writing to the file, which would expand nothing; but...

as egreg pointed out this would make e.g. the value of user-defined counters incorrect unless the user explicitly inject the expanded value in, and this one is not very easy.

Also quote Cyrus's comment below that answer

TeX gives users the ability to expand, but if users don't want it then they can choose not to expand using \protect or \DeclareRobustCommand. The default is to expand and so requires extra work if users want it otherwise.

1
  • by the way documentation of \MakeRobust is in source2e.pdf.
    – user202729
    Aug 9, 2022 at 2:49

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .