# Trying to understand the definition of \newcommand (complex handling of parameter tokens)

I'm still trying to understand how exactly tex parses documents. This time I'm trying to understand all the parameter-tokens-handling in \newcommand and friends, which looks like black magic to me - the crux here seems to happen in \@yargdef and \@yargd@f, which can be found in latex.ltx lines 868 and 877 respectively. The aim is obviously to create the parameter string that will ultimately be following a \def\foo, but I don't understand how this works.

To explain where I think my problem is, I'll have to explain my thinking a little, unfortunately. So: If I do \newcommand\foo[2]{\bar}, then as far as I can tell, this ultimately (after various checks for whether \foo is already defined and checking for optional arguments etc.pp.) expands to \@argdef\foo[2]{\bar}.

I'll go step by step how I read the definitions:

• \@argdef is defined via:
\long\def\@argdef#1[#2]#3{%
\@ifdefinable #1{\@yargdef#1\@ne{#2}{#3}}}

So we end up with \@yargdef\foo\@ne{2} (leaving {\bar}).
• \@yargdefis defined thus:
\long \def \@yargdef #1#2#3{%
\ifx#2\tw@
\def\reserved@b##11{[####1]}%
\else
\let\reserved@b\@gobble
\fi
\expandafter
\@yargd@f \expandafter{\number #3}#1%
}

which defines \reserved@b as \@gobble and ultimately expands to \@yargd@f{2}\foo (leaving {\bar}).
• Now \@yargd@f is the weird part. It is defined via:
\long \def \@yargd@f#1#2{%
\def \reserved@a ##1#1##2##{%
\expandafter\def\expandafter#2\reserved@b ##1#1%
}%
\l@ngrel@x \reserved@a 0##1##2##3##4##5##6##7##8##9###1%
}

So the expansion in my case should be (if I'm not mistaken?):
\def\reserved@a #12#2#{%
\expandafter\def\expandafter\foo\reserved@b #12%
}%
\l@ngrel@x \reserved@a 0#1#2#3#4#5#6#7#8#9#2

So in the application of \reserved@a, the first argument should be 0#1# and the second argument should be... empty, I guess, since the immediate following token is already #...? Leaving 3#4#5#6#7#8#9#2{\bar} as the next tokens?

This must be where my thinking goes wrong, because if I pretend that string before the {\bar} isn't there, the rest makes sense: \reserved@a then expands to \expandafter\def\expandafter\foo\reserved@b 0#1#2 - \reserved@b is defined as \@gobble which just eats the first token it finds, so the expansion of that should be \def\foo#1#2 - which would be followed by the {\bar}.

So apparently I'm interpreting the argument-string of \reserved@a wrong? What exactly is the second argument of \reserved@a in its application, if not empty? The only possible other explanation I can think of is that it eats to the next #-token, but even then, it would only eat the #3 part and leave a whole string of parameter tokens...?

am I expanding 0##1##2##3##4##5##6##7##8##9###1 or the ##1#1##2## in the definition of \reserved@a wrong?

Your reasoning is correct except for what the #{ parameter does.

When TeX sees a # in the <parameter text> of a macro (as in \def\macro<parameter text>{<replacement text>}), the following token can be either a digit in the range 1–9, or a {. When the parameter is "the dreaded weird #{ parameter", then TeX behaves as if the delimiter of the previous argument was a { (since you can't use a { in the <parameter text>).

In this definition (remember the {\bar} is still in the input stream):

\def\reserved@a #12#2#{%
\expandafter\def\expandafter\foo\reserved@b #12%
}%
\l@ngrel@x \reserved@a 0#1#2#3#4#5#6#7#8#9#2{\bar }


when \reserved@a expands, #1 will be, as you said, 0#1#, and #2 will be everything up to the next {, and not empty, so #3#4#5#6#7#8#9#2, delimited by the { in {\bar }. This definition of \reserved@a effectively throws away the remaining parameter tokens. So your "pretend[ing] that string before the {\bar} isn't there" was actually correct. The rest of the code executes as you assumed.

A quote from The TeXbook, about the #{ parameter:

A special extension is allowed to these rules: If the very last character of the <parameter text> is #, so that this # is immediately followed by {, TeX will behave as if the { had been inserted at the right end of both the parameter text and the replacement text. For example, if you say '\def\a#1#{\hbox to #1}', the subsequent text '\a3pt{x}' will expand to '\hbox to 3pt{x}', because the argument of \a is delimited by a left brace.

• Other links about #{: Macros with # as the last parameter and “Grab to #{” macro arguments – campa Feb 24 at 13:58
• @campa Those are good references, indeed. Thanks! – Phelype Oleinik Feb 24 at 14:00
• Oooooh, I had no idea that #{ was special - that was my problem. Thanks a lot :) – Dennis Müller Feb 24 at 14:21
• @DennisMüller You're welcome :-) Yes, that is the only exception to the "# must be followed by 1--9" rule. See egreg's answer to my question here, and perhaps §476 in TeX-the-program (texdoc tex) to see where does that exception come from. – Phelype Oleinik Feb 24 at 14:30

I've already dealt with the problem in my book on LaTeX programming. My example is slightly different, but it's easier to copy. ;-) Suppose we have

\newcommand{\xyz}[2]{ab#1cd#2ef}


This becomes

\@star@or@long\new@command[2]{ab#1cd#2ef}


As there is no * after \new@command, this does \let\l@ngrel@x=\long and \@star@or@long disappears. Now \new@command is expanded, yielding

\@testopt{\@newcommand\xyz}0[2]{ab#1cd#2ef}


This becomes



The purpose is to add [0] if the number of arguments is not given. Since there is a [, we get

\@newcommand\xyz[2]{ab#1cd#2ef}


Now comes the lookup for a further optional argument, which I skip, and we obtain

\@argdef\xyz[2]{ab#1cd#2ef}


The token \xyz is checked for definability; if it is not definable we get the error message, otherwise

\@yargdef\xyz\@ne{2}{ab#1cd#2ef}


and this is where the real fun starts:

\ifx\@ne\tw@
\def\reserved@b#11{[##1]}
\else
\let\reserved@b\@gobble
\fi
\expandafter\@yargd@f\expandafter{\number2}\xyz{ab#1cd#2ef}


This defines \reserved@b to be \@gobble and leads to

\@yargd@f{2}\xyz{ab#1cd#2ef}


Recall the definition of \@yarg@def:

\long\def\@yargd@f#1#2{%
\def\reserved@a##1#1##2##{%
\expandafter\def\expandafter#2\reserved@b##1#1}%
\l@ngrel@x\reserved@a0##1##2##3##4##5##6##7##8##9###1%
}


The arguments are #1=2 and #2=\xyz, so we get (double ## are here reduced to a single #)

\def\reserved@a#12#2#{\expandafter\def\expandafter\xyz\reserved@b#12}
\l@ngrel@x\reserved@a0#1#2#3#4#5#6#7#8#9##1{ab#1cd#2ef}


The first argument to \reserved@a is delimited by 2, the second argument by {. This is a particular feature of TeX: the final argument can be delimited by the opening brace of the replacement text.

Thus the first argument is, in our case, 0#1#. Since \l@ngrel@x is \long it triggers the expansion of \reserved@a with the stated arguments:

\long\expandafter\def\expandafter\xyz\reserved@b0#1#2{ab#1cd#2ef}


Since \reserved@b is \@gobble, we end up with

\long\def\xyz{ab#1cd#2ef}