# Why does TeX remove braces around delimited arguments?

On pp. 203–204 of the TeXbook, Knuth describes how TeX absorbs macros' arguments: delimited go until the first appearance of the delimiter, and undelimited go until the next token, in both cases respecting brace groups. But then he says:

In both cases ,if the argument found in this way has the form '{<nested tokens>}', where <nested tokens> stands for any sequence of tokens that is properly nested with respect to braces, the outermost braces enclosing the argument are removed and the <nested tokens> will remain.

This is a very annoying feature for delimited arguments! For example, in this question, there is no good way to deal with a macro \macro#1\pgfeov where #1 may either be a single braced argument or a pair of arguments, because the braces are removed by the argument scanner and then the result (if not double-braced) looks like multiple tokens and so is caught (by an inner macro) as more than one argument even though I put braces around it.

I consulted the TeXbook in the hopes that some kind of explanation would be given, but there is not even an exercise on the subject. Can anyone suggest why this is a useful feature?

Edit: I want to clarify that I am not talking about "arguments" in the imperative-language sense where one tries to treat macros as functions, passing its inputs as, say, \macro[#1]<#2>(#3,#4). I mean them in the sense described above: token lists absorbed by TeX by parsing text.

I also want to add that I am less interested in workarounds than in the answer to the question in the title: why (philosophically) does TeX work like this? For example, I know that the reason that spaces are ignored after control words is that if they were not, there would be no way to omit a space after one, since the scanner will keep piling on letters to the macro name until it no longer can. So there is a benefit to this rule; it makes the language more powerful.

-
maybe i am failing to see what you mean. But how would you then pass a single argument which is created of several nested groups? That would require nested arguments to be defined in a nested token command list, would it not? Am i way of with your question? As i see it the user needs to know if the macro requires a single token argument (possibly in a group). – zeroth Jan 2 '12 at 21:17
@zeroth: If you want a macro that requires a single-token argument, you define it \def\macro#1{...}. I'm asking about macros of the form \def\macro#1\pgfeov{...}, where #1 is anything, possibly multiple groups, that goes until the token \pgfeov. My complaint is that almost all the time, #1 is taken to be the literal text between \macro and \pgfeov; however, if #1 consists of a single brace group, those braces are removed. – Ryan Reich Jan 2 '12 at 21:26
ahh, now i see what you mean! Thanks for the clarification! I also see what your frustration is! But i can also see that i for one would not expect it to be handled exceptionally different from the ordinary argument passing. – zeroth Jan 2 '12 at 21:35
@zeroth: As I see it, the only reason you need to brace-group an undelimited argument is because there is no other way to catch multiple tokens. Essentially, balanced braces play the role of the delimiters. When you are scanning in a mode that catches unlimited amounts of text, why should they still be special, except to the extent that groups should be preserved? – Ryan Reich Jan 2 '12 at 21:44
@RyanReich I agree with you that this feature is annoying. Losing braces is typically avoided with a variation on the theme \def\macro{\macro@\empty} and \def\macro@#1\pgfeov{\expandafter\macro@@\expandafter{#1}}. One case where removing braces could be useful is optional arguments in LaTeX2e, where you can "hide ] behind braces", e.g., \section[{Some [bracketed] text}]{Title}, without ending up with spurious braces in the argument later. – Bruno Le Floch Jan 5 '12 at 1:10

One thing to note is that TeX's only means of nesting arguments are braces. You can define a macro \def\whatever[#1]{...} but when you call it as \whatever[oh[well]], things go down the drain awfully. Calling it as \whatever[{oh[well]}] however works swimmingly, and \whatever never notices it has been taken for a ride by slipping a ] into its argument. So the braces can be used as a means of hiding occurences of the closing delimiter from TeX without actually affecting the intended argument.

It also means that whenever you call a macro using delimited arguments with a non-literal argument (more exactly, an argument not completely under your own control, as it often happens when you write a macro package to be used by others), you should always add a layer of braces around each delimited argument, like \whatever[{#1}] or similar. There is no other way to ensure that arguments will not get chopped up into something different because they themselves may contain a closing bracket.

-
But this isn't always possible: for example, you might use \def\macro#1 #2{...} to look for spaces, and call it inside some other macro as \macro#1...no way to insert braces. – Ryan Reich Jan 7 '12 at 11:19
Huh? \macro{#1} ... works just fine. Why would you not be able to insert braces here? – user9588 Jan 7 '12 at 12:40
And actually, so will \macro{#1} {...} in case you were concerned about the second, non-delimited argument. Leading spaces are skipped when looking for an undelimited argument, but in this use case, the space has already served its purpose of delimiting the first argument before this skipping could have ensued. – user9588 Jan 7 '12 at 13:15
That is exactly my point. I want to use delimited macros to do what they are obviously intended to do: text processing. Except for this one strange rule, they do it as expected. Aside from the word "arguments" in the title, which has the technical meaning of "thing that gets put in #1 by TeX", why would you think I meant otherwise? – Ryan Reich Jan 8 '12 at 20:10
If your premise is that the real purpose of delimited arguments is not delimiting arguments but text processing, you obviously won't get an answer both in concord with the underlying philosophy as well as your preconceptions. The ability to transparently pass an argument that includes argument delimiters itself by stripping one level of delimiters per call is not unique to TeX. You'll find it, for example, ubiquitous when working with the m4 macro processor. – user9588 Jan 9 '12 at 15:55

I am adding a second answer here because now I am answering a different, implied question: How do I faithfully reproduce an argument possibly brace-enclosed in the form \macro#1\pgfeov? Like putting it as an argument to \message? Well, make sure it is not quite brace-enclosed.

\def\macro{\macroii \empty}
\def\macroii#1\pgfeov{\message{#1}}


In non-expandable contexts, you can work with other stuff that is easy to remove, possibly an empty argument {}. If you were planning to pass this on as delimited argument, you could just use the called command instead of \empty.

-

Although I appreciate David's answers they seem not to grapple with the design question I asked. Therefore I will attempt to answer this myself using the understanding I've gained from our discussions.

Let's start from the premise that delimited arguments are meant to do exactly what they appear to do: grab arguments but ignore certain surrounding and intervening text. It doesn't matter whether the goal is to actually perform text processing or simply use unusual bracing conventions. I claim that:

1. It is necessary to have a mechanism to prevent the argument scanner from finding certain expressions that would otherwise constitute a delimited argument;

2. It is logical that this mechanism is to wrap the affected text in braces;

3. It is consistent with undelimited arguments to remove those braces;

4. It is furthermore simplest to do it that way, in that the workaround required to defeat the action in point 3 is easier than the workaround required to produce it if the braces aren't removed.

About point 1: of course such a mechanism is required, or else text that might appear as a delimiter could never appear in an argument (and I know that Knuth is irked by that sort of thing because of his objections to certain verbatim-printing techniques in Appendix D of the TeXbook).

About point 2: as David says in his first answer, there's no other established mechanism in TeX to group things (braces are the grouping characters). It is, furthermore, how we group undelimited arguments as well, so this is an issue of consistency.

About point 3: the key point here is that undelimited arguments are not actually treated any differently than delimited arguments! The description makes the delimited ones sound special, but actually, in the question, I summarized the behavior of argument-grabbing in a perfectly neutral way: arguments are grabbed up until the next delimiter, which is then removed. Undelimited arguments have empty delimiters, that's all.

About point 4: okay, just having a logical reason to do something is not a practical justification for doing it, and my question describes a logical reason for not doing it. It would help to have a logical reason not to do the alternative. So suppose braces were not removed from delimited arguments; then you would be forced by point 1 to add them occasionally and have to get rid of them. How would you do that? There are three cases to take care of (#1 = the argument):

(a): #1 = abc...
(b): #1 = {abc...}
(c): #1 = {abc}{...}


Bear in mind that abc... can contain nested groups, so your solution has to respect nesting. Therefore, you either have to implement a brace-parser within TeX, or you have to use some primitive that respects nesting. For the solution to be expandable you have to use \def or \edef. So: you need to \define a macro capable of stripping the outer braces from (b) but no braces from (c), and not altering (a). Here is a macro that handles (b) and (c), but not (a), using some simple utility macros from LaTeX:

\def\stripbraces#1#2\@nil{%
% #1 is undelimited, so has its braces removed.  If there is no #2, that is good.
\ifx\relax#2%
\expandafter\@firstoftwo
% #2 never has its braces removed, even if it's a single group.
\else
\expandafter\@secondoftwo
\fi
{#1}{{#1}#2}%
}


You use it as \stripbraces...\@nil and it removes the outer braces from .... This is under the assumption that delimited arguments don't have their braces removed. Alas, if the text had two or more tokens but no braces to begin, then its first token gets braced.

Now, it is possible to use \futurelet (i.e. \@ifnextchar) to figure out if #1 begins with a brace, but that's not expandable. Here is an expandable version inspired by my trickery in this answer, as summarized in this one:

\def\findbrace#1#{%
% The brace is immediately after the macro
\ifx\relax#1%
\expandafter\@firstoftwo
% The next token is not a brace
\else
\expandafter\@secondoftwo
\fi
{\stripbraces}{\remove@nil#1}%
}
\def\remove@nil#1\@nil{#1\@gobble}


Used, of course, as \findbrace...\@nil{}, and you have to modify \stripbraces also to \@gobble the dummy braces you put after \@nil. These two macros together will catch (a), (b), and (c) as they should.

The alternative to this workaround is the one described in David's second answer: just prepend, say, \empty to your delimited text and then it will never be surrounded by braces, yet has the same expansion. This is considerably less tricky, to say the least.

I suppose the combination of logical consistency and avoiding extreme trickiness is a good reason to take the route that TeX follows in the real world, though I would like to observe that, since my workaround does exist, the alternative wouldn't have been unacceptable (though possibly unworkable until someone figured it out). And it's not like TeX is exactly averse to trickiness as it is, but by that logic, the present situation with David's workaround is fine too. So I didn't say that.

-
Note that the workaround with \empty is fully robust, regardless of what weird category codes are in place, but your \findbrace (which is missing some \relax I believe) is not. You should instead use \ifcase\expandafter\@gobble\expandafter{\expandafter{\string#1.}1}0 to test if the first character is a brace, then \ifcase\expandafter\@gobble\expandafter{\expandafter{\romannumeral\expandafter\‌​expandafter\expandafter\z@\expandafter\string\@gobble#1}1}1}0 to test for emptyness of \@gobble#1. Robustness requires significantly harder code than what you propose. – Bruno Le Floch Jan 10 '12 at 6:22
@Bruno: I take your point, but I call the Pareto principle. – Ryan Reich Jan 10 '12 at 6:48
Also known as 80-20 law. My attempts to make it apply here were indistinguishable from satire, so I would be interested in your take on it. – user9588 Jan 10 '12 at 14:58

In both cases, if the argument found in this way has the form '{}', where stands for any sequence of tokens that is properly nested with respect to braces, the outermost braces enclosing the argument are removed and the will remain.

[...]

I also want to add that I am less interested in workarounds than in the answer to the question in the title: why (philosophically) does TeX work like this?

When TeX processes delimiteds arguments, delimiters "hidden" inside curly braces will not be taken into account. Sometimes this is desired while keeping the surrounding braces is not desired. E.g., in situations where the set of tokens forming one delimiter is a subset of those tokens that form another delimiter.

E.g.,

\def\macroA#1delimited{The argument #1 twice.}
\def\macroB#1delimitedBB{#1 was delimited}

\macroA{\macroB ARGdelimitedBB}delimited
\bye


yields:

The argument \macroB ARGdelimitedBB twice.


yields:

The argument ARG was delimited twice.


If braces surrounding the argument of \macroA were omitted in the source code, then there wouldn't be a matching delimiter for the argument of \macroB any more.

If braces were left in place during expansion, the result would contain a probably unwanted brace group:

The argument {ARG was delimited} twice.


With delimited arguments you can prevent removal of surrounding braces by adding some token in front of the argument, e.g. a dot, and later removing that:

\def\gobbledot.{}%
\def\macroA#1\delimited{\expandafter\macroB\gobbledot#1}%
\def\macroB#1{The argument is wrapped into parentheses: (#1)}

\macroA.{ARG}\delimited
\bye


yields

The argument is wrapped into parentheses: (ARG)


If no trailing dot were inserted and laterwards removed, braces would have been removed and only the first token of \macroA's argument would have been treated as \macroB's argument:

\def\macroA#1\delimited{\macroB#1}%
\def\macroB#1{The argument is wrapped into parentheses: (#1)}

\macroA{ARG}\delimited
\bye


yields:

The argument is wrapped into parentheses: (A)RG

-