26

On pp. 203–204 of the TeXbook, Knuth describes how TeX absorbs macros' arguments: delimited go until the first appearance of the delimiter, and undelimited go until the next token, in both cases respecting brace groups. But then he says:

In both cases ,if the argument found in this way has the form '{<nested tokens>}', where <nested tokens> stands for any sequence of tokens that is properly nested with respect to braces, the outermost braces enclosing the argument are removed and the <nested tokens> will remain.

This is a very annoying feature for delimited arguments! For example, in this question, there is no good way to deal with a macro \macro#1\pgfeov where #1 may either be a single braced argument or a pair of arguments, because the braces are removed by the argument scanner and then the result (if not double-braced) looks like multiple tokens and so is caught (by an inner macro) as more than one argument even though I put braces around it.

I consulted the TeXbook in the hopes that some kind of explanation would be given, but there is not even an exercise on the subject. Can anyone suggest why this is a useful feature?

Edit: I want to clarify that I am not talking about "arguments" in the imperative-language sense where one tries to treat macros as functions, passing its inputs as, say, \macro[#1]<#2>(#3,#4). I mean them in the sense described above: token lists absorbed by TeX by parsing text.

I also want to add that I am less interested in workarounds than in the answer to the question in the title: why (philosophically) does TeX work like this? For example, I know that the reason that spaces are ignored after control words is that if they were not, there would be no way to omit a space after one, since the scanner will keep piling on letters to the macro name until it no longer can. So there is a benefit to this rule; it makes the language more powerful.

8
  • maybe i am failing to see what you mean. But how would you then pass a single argument which is created of several nested groups? That would require nested arguments to be defined in a nested token command list, would it not? Am i way of with your question? As i see it the user needs to know if the macro requires a single token argument (possibly in a group).
    – nickpapior
    Jan 2, 2012 at 21:17
  • 4
    @zeroth: If you want a macro that requires a single-token argument, you define it \def\macro#1{...}. I'm asking about macros of the form \def\macro#1\pgfeov{...}, where #1 is anything, possibly multiple groups, that goes until the token \pgfeov. My complaint is that almost all the time, #1 is taken to be the literal text between \macro and \pgfeov; however, if #1 consists of a single brace group, those braces are removed.
    – Ryan Reich
    Jan 2, 2012 at 21:26
  • ahh, now i see what you mean! Thanks for the clarification! I also see what your frustration is! But i can also see that i for one would not expect it to be handled exceptionally different from the ordinary argument passing.
    – nickpapior
    Jan 2, 2012 at 21:35
  • @zeroth: As I see it, the only reason you need to brace-group an undelimited argument is because there is no other way to catch multiple tokens. Essentially, balanced braces play the role of the delimiters. When you are scanning in a mode that catches unlimited amounts of text, why should they still be special, except to the extent that groups should be preserved?
    – Ryan Reich
    Jan 2, 2012 at 21:44
  • 3
    @RyanReich I agree with you that this feature is annoying. Losing braces is typically avoided with a variation on the theme \def\macro{\macro@\empty} and \def\macro@#1\pgfeov{\expandafter\macro@@\expandafter{#1}}. One case where removing braces could be useful is optional arguments in LaTeX2e, where you can "hide ] behind braces", e.g., \section[{Some [bracketed] text}]{Title}, without ending up with spurious braces in the argument later. Jan 5, 2012 at 1:10

5 Answers 5

25

One thing to note is that TeX's only means of nesting arguments are braces. You can define a macro \def\whatever[#1]{...} but when you call it as \whatever[oh[well]], things go down the drain awfully. Calling it as \whatever[{oh[well]}] however works swimmingly, and \whatever never notices it has been taken for a ride by slipping a ] into its argument. So the braces can be used as a means of hiding occurences of the closing delimiter from TeX without actually affecting the intended argument.

It also means that whenever you call a macro using delimited arguments with a non-literal argument (more exactly, an argument not completely under your own control, as it often happens when you write a macro package to be used by others), you should always add a layer of braces around each delimited argument, like \whatever[{#1}] or similar. There is no other way to ensure that arguments will not get chopped up into something different because they themselves may contain a closing bracket.

8
  • But this isn't always possible: for example, you might use \def\macro#1 #2{...} to look for spaces, and call it inside some other macro as \macro#1...no way to insert braces.
    – Ryan Reich
    Jan 7, 2012 at 11:19
  • Huh? \macro{#1} ... works just fine. Why would you not be able to insert braces here?
    – user9588
    Jan 7, 2012 at 12:40
  • And actually, so will \macro{#1} {...} in case you were concerned about the second, non-delimited argument. Leading spaces are skipped when looking for an undelimited argument, but in this use case, the space has already served its purpose of delimiting the first argument before this skipping could have ensued.
    – user9588
    Jan 7, 2012 at 13:15
  • 2
    That is exactly my point. I want to use delimited macros to do what they are obviously intended to do: text processing. Except for this one strange rule, they do it as expected. Aside from the word "arguments" in the title, which has the technical meaning of "thing that gets put in #1 by TeX", why would you think I meant otherwise?
    – Ryan Reich
    Jan 8, 2012 at 20:10
  • 1
    If your premise is that the real purpose of delimited arguments is not delimiting arguments but text processing, you obviously won't get an answer both in concord with the underlying philosophy as well as your preconceptions. The ability to transparently pass an argument that includes argument delimiters itself by stripping one level of delimiters per call is not unique to TeX. You'll find it, for example, ubiquitous when working with the m4 macro processor.
    – user9588
    Jan 9, 2012 at 15:55
14

I am adding a second answer here because now I am answering a different, implied question: How do I faithfully reproduce an argument possibly brace-enclosed in the form \macro#1\pgfeov? Like putting it as an argument to \message? Well, make sure it is not quite brace-enclosed.

\def\macro{\macroii \empty}
\def\macroii#1\pgfeov{\message{#1}}

In non-expandable contexts, you can work with other stuff that is easy to remove, possibly an empty argument {}. If you were planning to pass this on as delimited argument, you could just use the called command instead of \empty.

6

Although I appreciate David's answers they seem not to grapple with the design question I asked. Therefore I will attempt to answer this myself using the understanding I've gained from our discussions.

Let's start from the premise that delimited arguments are meant to do exactly what they appear to do: grab arguments but ignore certain surrounding and intervening text. It doesn't matter whether the goal is to actually perform text processing or simply use unusual bracing conventions. I claim that:

  1. It is necessary to have a mechanism to prevent the argument scanner from finding certain expressions that would otherwise constitute a delimited argument;

  2. It is logical that this mechanism is to wrap the affected text in braces;

  3. It is consistent with undelimited arguments to remove those braces;

  4. It is furthermore simplest to do it that way, in that the workaround required to defeat the action in point 3 is easier than the workaround required to produce it if the braces aren't removed.

About point 1: of course such a mechanism is required, or else text that might appear as a delimiter could never appear in an argument (and I know that Knuth is irked by that sort of thing because of his objections to certain verbatim-printing techniques in Appendix D of the TeXbook).

About point 2: as David says in his first answer, there's no other established mechanism in TeX to group things (braces are the grouping characters). It is, furthermore, how we group undelimited arguments as well, so this is an issue of consistency.

About point 3: the key point here is that undelimited arguments are not actually treated any differently than delimited arguments! The description makes the delimited ones sound special, but actually, in the question, I summarized the behavior of argument-grabbing in a perfectly neutral way: arguments are grabbed up until the next delimiter, which is then removed. Undelimited arguments have empty delimiters, that's all.

About point 4: okay, just having a logical reason to do something is not a practical justification for doing it, and my question describes a logical reason for not doing it. It would help to have a logical reason not to do the alternative. So suppose braces were not removed from delimited arguments; then you would be forced by point 1 to add them occasionally and have to get rid of them. How would you do that? There are three cases to take care of (#1 = the argument):

(a): #1 = abc...
(b): #1 = {abc...}
(c): #1 = {abc}{...}

Bear in mind that abc... can contain nested groups, so your solution has to respect nesting. Therefore, you either have to implement a brace-parser within TeX, or you have to use some primitive that respects nesting. For the solution to be expandable you have to use \def or \edef. So: you need to \define a macro capable of stripping the outer braces from (b) but no braces from (c), and not altering (a). Here is a macro that handles (b) and (c), but not (a), using some simple utility macros from LaTeX:

\def\stripbraces#1#2\@nil{%
 % #1 is undelimited, so has its braces removed.  If there is no #2, that is good.
 \ifx\relax#2%
  \expandafter\@firstoftwo
 % #2 never has its braces removed, even if it's a single group.
 \else
  \expandafter\@secondoftwo
 \fi
 {#1}{{#1}#2}%
}

You use it as \stripbraces...\@nil and it removes the outer braces from .... This is under the assumption that delimited arguments don't have their braces removed. Alas, if the text had two or more tokens but no braces to begin, then its first token gets braced.

Now, it is possible to use \futurelet (i.e. \@ifnextchar) to figure out if #1 begins with a brace, but that's not expandable. Here is an expandable version inspired by my trickery in this answer, as summarized in this one:

\def\findbrace#1#{%
 % The brace is immediately after the macro
 \ifx\relax#1%
  \expandafter\@firstoftwo
 % The next token is not a brace
 \else
  \expandafter\@secondoftwo
 \fi
 {\stripbraces}{\remove@nil#1}%
}
\def\remove@nil#1\@nil{#1\@gobble}

Used, of course, as \findbrace...\@nil{}, and you have to modify \stripbraces also to \@gobble the dummy braces you put after \@nil. These two macros together will catch (a), (b), and (c) as they should.

The alternative to this workaround is the one described in David's second answer: just prepend, say, \empty to your delimited text and then it will never be surrounded by braces, yet has the same expansion. This is considerably less tricky, to say the least.

I suppose the combination of logical consistency and avoiding extreme trickiness is a good reason to take the route that TeX follows in the real world, though I would like to observe that, since my workaround does exist, the alternative wouldn't have been unacceptable (though possibly unworkable until someone figured it out). And it's not like TeX is exactly averse to trickiness as it is, but by that logic, the present situation with David's workaround is fine too. So I didn't say that.

3
  • 2
    Note that the workaround with \empty is fully robust, regardless of what weird category codes are in place, but your \findbrace (which is missing some \relax I believe) is not. You should instead use \ifcase\expandafter\@gobble\expandafter{\expandafter{\string#1.}1}0 to test if the first character is a brace, then \ifcase\expandafter\@gobble\expandafter{\expandafter{\romannumeral\expandafter\expandafter\expandafter\z@\expandafter\string\@gobble#1}1}1}0 to test for emptyness of \@gobble#1. Robustness requires significantly harder code than what you propose. Jan 10, 2012 at 6:22
  • @Bruno: I take your point, but I call the Pareto principle.
    – Ryan Reich
    Jan 10, 2012 at 6:48
  • Also known as 80-20 law. My attempts to make it apply here were indistinguishable from satire, so I would be interested in your take on it.
    – user9588
    Jan 10, 2012 at 14:58
3

In my opinion the feature of TeX removing - if present - one level of braces surrounding the entire argument, be it delimited or undelimited, is not annoying.

The circumstance that people often don't take this into account when implementing macros with delimited arguments is annoying.

[End of opinion-based remark.]


You can prevent brace-removal by having TeX prepend and laterwards remove a token that does not disturb delimiter-matching:

% compile with pdfetex!
\chardef\stopromannumeral=`\^^00
\def\gobbledot.{}
\long\def\macro{\romannumeral\innermacro.}
\long\def\innermacro#1\pgfeov{%
  \expandafter\expandafter\expandafter\stopromannumeral
  \expandafter\innerinnermacro\expandafter{\gobbledot#1}%
}%
\long\def\innerinnermacro#1{%
  Macro's argument is between {\tt||}: {\tt\detokenize{|#1|}}%
}%

\macro bla\pgfeov

\macro{bla}{bla}\pgfeov

\macro{bla} \pgfeov

\macro{bla}\pgfeov

\bye

enter image description here

If you don't want to have TeX prepend a character-token because it can be changed, e.g., via \uppercase/\lowercase, and don't want to have TeX prepend an ordinary control sequence token as that could be redefined in terms of \outer laterwards, you can have TeX prepend a frozen-\relax:

% compile with pdfetex!
\chardef\stopromannumeral=`\^^00
%
\def\gobblefrozenrelax#1{%
  \def\gobblefrozenrelax#1{}%
  \long\def\macro{\romannumeral\innermacro#1}%
}%
\expandafter\expandafter\expandafter\gobblefrozenrelax
\expandafter\expandafter\expandafter{%
\expandafter\expandafter\ifnum0=0\fi}%
%
\long\def\innermacro#1\pgfeov{%
  \expandafter\expandafter\expandafter\stopromannumeral
  \expandafter\innerinnermacro\expandafter{\gobblefrozenrelax#1}%
}%
\long\def\innerinnermacro#1{%
  Macro's argument is between {\tt||}: {\tt\detokenize{|#1|}}%
}

\macro bla\pgfeov

\macro{bla}{bla}\pgfeov

\macro{bla} \pgfeov

\macro{bla}\pgfeov

\bye

enter image description here

You can implement expandably checking whether the \pgfeov-delimited argument consists of one or two brace-nested components—the following example is very picky about space-tokens surrounding these components.

% compile with pdfetex!
%%-----------------------------------------------------------------------------
%% Paraphernalia:
%%.............................................................................
\long\def\firstoftwo#1#2{#1}%
\long\def\secondoftwo#1#2{#2}%
\chardef\stopromannumeral=`\^^00 %
%%-----------------------------------------------------------------------------
%% Check whether argument is empty:
%%.............................................................................
%% \CheckWhetherNull{<Argument which is to be checked>}%
%%                  {<Tokens to be delivered in case that argument
%%                    which is to be checked is empty>}%
%%                  {<Tokens to be delivered in case that argument
%%                    which is to be checked is not empty>}%
%%
%% The gist of this macro comes from Robert R. Schneck's \ifempty-macro:
%% <https://groups.google.com/forum/#!original/comp.text.tex/kuOEIQIrElc/lUg37FmhA74J>
\long\def\CheckWhetherNull#1{%
  \romannumeral\expandafter\secondoftwo\string{\expandafter
  \secondoftwo\expandafter{\expandafter{\string#1}\expandafter
  \secondoftwo\string}\expandafter\firstoftwo\expandafter{\expandafter
  \secondoftwo\string}\expandafter\stopromannumeral\secondoftwo}{%
  \expandafter\stopromannumeral\firstoftwo}%
}%
%%-----------------------------------------------------------------------------
%% Check whether argument's first token is a catcode-1-character
%%.............................................................................
%% \CheckWhetherBrace{<Argument which is to be checked>}%
%%                   {<Tokens to be delivered in case that argument
%%                     which is to be checked has a leading
%%                     explicit catcode-1-character-token>}%
%%                   {<Tokens to be delivered in case that argument
%%                     which is to be checked does not have a
%%                     leading explicit catcode-1-character-token>}%
\long\def\CheckWhetherBrace#1{%
  \romannumeral\expandafter\secondoftwo\expandafter{\expandafter{%
  \string#1.}\expandafter\firstoftwo\expandafter{\expandafter
  \secondoftwo\string}\expandafter\stopromannumeral\firstoftwo}{%
  \expandafter\stopromannumeral\secondoftwo}%
}%
%%-----------------------------------------------------------------------------
%% \macro..\pgfeov
%%.............................................................................
\def\gobblefrozenrelax#1{%
  \def\gobblefrozenrelax#1{}%
  \long\def\macro{\romannumeral\innermacro#1}%
}%
\expandafter\expandafter\expandafter\gobblefrozenrelax
\expandafter\expandafter\expandafter{%
\expandafter\expandafter\ifnum0=0\fi}%
%
\long\def\innermacro#1\pgfeov{%
  \expandafter\SyntaxChecker\expandafter{\gobblefrozenrelax#1}%
  {\innerinnermacro}{\errormessagemacro}%
}%
\long\def\SyntaxChecker#1#2#3{%
   \CheckWhetherBrace{#1}{%
     \expandafter\CheckWhetherNull\expandafter{\firstoftwo{}#1}{%
       \firstoftwo
     }{%
        \expandafter\CheckWhetherBrace\expandafter{\firstoftwo{}#1}{%
           \expandafter\expandafter\expandafter\CheckWhetherNull
           \expandafter\expandafter\expandafter{%
             \expandafter\firstoftwo\expandafter{\expandafter}\firstoftwo{}#1%
           }{\firstoftwo}{\secondoftwo}%
        }{\secondoftwo}%
     }%
   }{\secondoftwo}%
   {\expandafter\stopromannumeral#2}{\expandafter\stopromannumeral#3}{#1}%
}%


\long\def\innerinnermacro#1{%
  Macro's argument is between {\tt||}: {\tt\detokenize{|#1|}}%
}

\long\def\errormessagemacro#1{%
  Macro's argument is between {\tt||}: {\tt\detokenize{|#1|}}\hfill\break
  {\bf It does not match the requested pattern of consisting of one or two
     brace-nested arguments!%
  }%
}


\noindent
\macro{bla}{bla}\pgfeov

\noindent\hrulefill\null

\noindent
\macro{bla}\pgfeov

\noindent\hrulefill\null

\noindent
\macro{bla}{bla}{bla}\pgfeov

\noindent\hrulefill\null

\noindent
\macro{bla} {bla}\pgfeov

\noindent\hrulefill\null

\noindent
\macro{bla} {bla} \pgfeov

\noindent\hrulefill\null

\noindent
\macro{bla}{bla} \pgfeov

\noindent\hrulefill\null

\noindent
\macro{bla} \pgfeov

\noindent\hrulefill\null

\noindent
\macro bla\pgfeov

\noindent\hrulefill\null

\noindent
\macro \pgfeov

\noindent\hrulefill\null

\noindent
\expandafter\macro\firstoftwo{}{} \pgfeov

\bye

enter image description here

Leading explicit space tokens (which would be discarded when further processing as undelimited arguments) can be taken into account by an expandable loop for removing leading explicit space tokens from arguments:

%  compile with pdfetex!
%%-----------------------------------------------------------------------------
%% Paraphernalia:
%%.............................................................................
\long\def\firstoftwo#1#2{#1}%
\long\def\secondoftwo#1#2{#2}%
\long\def\Exchange#1#2{#2#1}%
\long\def\PassFirstToSecond#1#2{#2{#1}}%
\firstoftwo{\def\removespace}{} {}%
\chardef\stopromannumeral=`\^^00 %
%%-----------------------------------------------------------------------------
%% Check whether argument is empty:
%%.............................................................................
%% \CheckWhetherNull{<Argument which is to be checked>}%
%%                  {<Tokens to be delivered in case that argument
%%                    which is to be checked is empty>}%
%%                  {<Tokens to be delivered in case that argument
%%                    which is to be checked is not empty>}%
%%
%% The gist of this macro comes from Robert R. Schneck's \ifempty-macro:
%% <https://groups.google.com/forum/#!original/comp.text.tex/kuOEIQIrElc/lUg37FmhA74J>
\long\def\CheckWhetherNull#1{%
  \romannumeral\expandafter\secondoftwo\string{\expandafter
  \secondoftwo\expandafter{\expandafter{\string#1}\expandafter
  \secondoftwo\string}\expandafter\firstoftwo\expandafter{\expandafter
  \secondoftwo\string}\expandafter\stopromannumeral\secondoftwo}{%
  \expandafter\stopromannumeral\firstoftwo}%
}%
%%-----------------------------------------------------------------------------
%% Check whether argument's first token is a catcode-1-character
%%.............................................................................
%% \CheckWhetherBrace{<Argument which is to be checked>}%
%%                   {<Tokens to be delivered in case that argument
%%                     which is to be checked has a leading
%%                     explicit catcode-1-character-token>}%
%%                   {<Tokens to be delivered in case that argument
%%                     which is to be checked does not have a
%%                     leading explicit catcode-1-character-token>}%
\long\def\CheckWhetherBrace#1{%
  \romannumeral\expandafter\secondoftwo\expandafter{\expandafter{%
  \string#1.}\expandafter\firstoftwo\expandafter{\expandafter
  \secondoftwo\string}\expandafter\stopromannumeral\firstoftwo}{%
  \expandafter\stopromannumeral\secondoftwo}%
}%
%%-----------------------------------------------------------------------------
%% Check whether brace-balanced argument starts with a space-token
%%.............................................................................
%% \CheckWhetherLeadingExplicitSpace{<Argument which is to be checked>}%
%%                                  {<Tokens to be delivered in case <argument
%%                                    which is to be checked> does have a
%%                                    leading explicit space-token>}%
%%                                  {<Tokens to be delivered in case <argument
%%                                    which is to be checked> does not have a
%%                                    a leading explicit space-token>}%
\long\def\CheckWhetherLeadingExplicitSpace#1{%
  \romannumeral\CheckWhetherNull{#1}%
  {\expandafter\stopromannumeral\secondoftwo}%
  {%
    % Let's nest things into \firstoftwo{...}{} to make sure they are nested in braces
    % and thus do not disturb when the test is carried out within \halign/\valign:
    \expandafter\firstoftwo\expandafter{%
      \expandafter\expandafter\expandafter\stopromannumeral
      \romannumeral\expandafter\secondoftwo
      \string{\CheckWhetherLeadingExplicitSpaceB.#1 }{}%
    }{}%
  }%
}%
\long\def\CheckWhetherLeadingExplicitSpaceB#1 {%
  \expandafter\CheckWhetherNull\expandafter{\firstoftwo{}#1}%
  {\Exchange{\firstoftwo}}{\Exchange{\secondoftwo}}%
  {\expandafter\expandafter\expandafter\stopromannumeral
   \expandafter\expandafter\expandafter}%
   \expandafter\secondoftwo\expandafter{\string}%
}%
%%-----------------------------------------------------------------------------
%% Trim leading space-tokens-loop
%%-----------------------------------------------------------------------------
\long\def\TrimLeadingSpaces#1{%
  \CheckWhetherLeadingExplicitSpace{#1}{%
    \expandafter\TrimLeadingSpaces\expandafter{\removespace#1}%
  }{%
    \PassFirstToSecond{#1}%
  }%
}%

%%-----------------------------------------------------------------------------
%% \macro..\pgfeov
%%.............................................................................
\def\gobblefrozenrelax#1{%
  \def\gobblefrozenrelax#1{}%
  \long\def\macro{\romannumeral\innermacro#1}%
}%
\expandafter\expandafter\expandafter\gobblefrozenrelax
\expandafter\expandafter\expandafter{%
\expandafter\expandafter\ifnum0=0\fi}%
%
\long\def\innermacro#1\pgfeov{%
  \expandafter\SyntaxChecker\expandafter{\gobblefrozenrelax#1}%
  {\innerinnermacro}{\errormessagemacro}%
}%
\long\def\SyntaxChecker#1#2#3{%
   \TrimLeadingSpaces{#1}{\CheckWhetherBrace}{%
     \expandafter\CheckWhetherNull\expandafter{\firstoftwo{}#1}{%
       \firstoftwo
     }{%
        \expandafter\TrimLeadingSpaces\expandafter{\firstoftwo{}#1}{\CheckWhetherBrace}{%
           \expandafter\expandafter\expandafter\CheckWhetherNull
           \expandafter\expandafter\expandafter{%
             \expandafter\firstoftwo\expandafter{\expandafter}\firstoftwo{}#1%
           }{\firstoftwo}{\secondoftwo}%
        }{\secondoftwo}%
     }%
   }{\secondoftwo}%
   {\expandafter\stopromannumeral#2}{\expandafter\stopromannumeral#3}{#1}%
}%


\long\def\innerinnermacro#1{%
  Macro's argument is between {\tt||}: {\tt\detokenize{|#1|}}%
}

\long\def\errormessagemacro#1{%
  Macro's argument is between {\tt||}: {\tt\detokenize{|#1|}}\hfill\break
  {\bf It does not match the requested pattern of consisting of one or two
     brace-nested arguments each of these brace-nested arguments probably
     preceded by some explicit space tokens that would be discarded when
     brace-nested arguments were processed as undelimited arguments!%
  }%
}

\noindent
\macro{bla}\pgfeov

\noindent\hrulefill\null

\noindent
\expandafter\macro\firstoftwo{}{} {bla}\pgfeov

\noindent\hrulefill\null

\noindent
\macro{bla}{bla}\pgfeov

\noindent\hrulefill\null

\noindent
\expandafter\macro\firstoftwo{}{} {bla} {bla}\pgfeov

\noindent\hrulefill\null

\noindent
\macro{bla} {bla}\pgfeov

\noindent\hrulefill\null

\noindent
\macro{bla}{bla}{bla}\pgfeov

\noindent\hrulefill\null

\noindent
\macro{bla} {bla} \pgfeov

\noindent\hrulefill\null

\noindent
\macro{bla}{bla} \pgfeov

\noindent\hrulefill\null

\noindent
\macro{bla} \pgfeov

\noindent\hrulefill\null

\noindent
\macro bl\pgfeov

\noindent\hrulefill\null

\noindent
\macro bla\pgfeov

\noindent\hrulefill\null

\noindent
\macro \pgfeov

\noindent\hrulefill\null

\noindent
\expandafter\macro\firstoftwo{}{} \pgfeov

\bye

enter image description here

You can implement a generic mechanism for defining expandable mechanisms for

  • extracting
  • removing (and thus delivering the remaining elements of that list)

the first element of a ⟨Delimiter⟩-separated list of elements.

Such mechanisms can be used, e.g., for defining an expandable mechanism for removing trailing explicit space-tokens from an undelimited argument which does without so-called "sentinel-tokens", i.e., without tokens whose usage is forbidden in user-supplied arguments.

%  compile with etex-extensions!
%%-----------------------------------------------------------------------------
%% Paraphernalia:
%%.............................................................................
\long\def\firstoftwo#1#2{#1}%
\long\def\secondoftwo#1#2{#2}%
\long\def\Exchange#1#2{#2#1}%
\long\def\PassFirstToSecond#1#2{#2{#1}}%
\firstoftwo{\def\removespace}{} {}%
\chardef\stopromannumeral=`\^^00 %
%%-----------------------------------------------------------------------------
%% Check whether argument is empty:
%%.............................................................................
%% \CheckWhetherNull{<Argument which is to be checked>}%
%%                  {<Tokens to be delivered in case that argument
%%                    which is to be checked is empty>}%
%%                  {<Tokens to be delivered in case that argument
%%                    which is to be checked is not empty>}%
%%
%% The gist of this macro comes from Robert R. Schneck's \ifempty-macro:
%% <https://groups.google.com/forum/#!original/comp.text.tex/kuOEIQIrElc/lUg37FmhA74J>
\long\def\CheckWhetherNull#1{%
  \romannumeral\expandafter\secondoftwo\string{\expandafter
  \secondoftwo\expandafter{\expandafter{\string#1}\expandafter
  \secondoftwo\string}\expandafter\firstoftwo\expandafter{\expandafter
  \secondoftwo\string}\expandafter\stopromannumeral\secondoftwo}{%
  \expandafter\stopromannumeral\firstoftwo}%
}%
%%-----------------------------------------------------------------------------
%% Terminology: An argument is "blank" if it consists either only of
%%              explicit space tokens or doesn't have any token at all.
%% Check whether argument is blank (empty or only spaces):
%%-----------------------------------------------------------------------------
%% -- Take advantage of the fact that TeX discards space tokens when
%%    "fetching" _un_delimited arguments: --
%% \CheckWhetherBlank{<Argument which is to be checked>}%
%%                   {<Tokens to be delivered in case that
%%                     argument which is to be checked is blank>}%
%%                   {<Tokens to be delivered in case that argument
%%                     which is to be checked is not blank>}%
\long\def\CheckWhetherBlank#1{%
  \romannumeral\expandafter\expandafter\expandafter\secondoftwo
  \expandafter\CheckWhetherNull\expandafter{\firstoftwo#1{}{}}%
}%
%%-----------------------------------------------------------------------------
%% Check whether argument's first token is a catcode-1-character
%%.............................................................................
%% \CheckWhetherBrace{<Argument which is to be checked>}%
%%                   {<Tokens to be delivered in case that argument
%%                     which is to be checked has a leading
%%                     explicit catcode-1-character-token>}%
%%                   {<Tokens to be delivered in case that argument
%%                     which is to be checked does not have a
%%                     leading explicit catcode-1-character-token>}%
\long\def\CheckWhetherBrace#1{%
  \romannumeral\expandafter\secondoftwo\expandafter{\expandafter{%
  \string#1.}\expandafter\firstoftwo\expandafter{\expandafter
  \secondoftwo\string}\expandafter\stopromannumeral\firstoftwo}{%
  \expandafter\stopromannumeral\secondoftwo}%
}%
%%-----------------------------------------------------------------------------
%% Check whether brace-balanced argument starts with a space-token
%%.............................................................................
%% \CheckWhetherLeadingExplicitSpace{<Argument which is to be checked>}%
%%                                  {<Tokens to be delivered in case <argument
%%                                    which is to be checked> does have a
%%                                    leading explicit space-token>}%
%%                                  {<Tokens to be delivered in case <argument
%%                                    which is to be checked> does not have a
%%                                    a leading explicit space-token>}%
\long\def\CheckWhetherLeadingExplicitSpace#1{%
  \long\def\CheckWhetherLeadingExplicitSpace##1{%
    \romannumeral\CheckWhetherNull{##1}%
    {\expandafter\stopromannumeral\secondoftwo}%
    {%
      % Let's nest things into \firstoftwo{...}{} to make sure they are nested in braces
      % and thus do not disturb when the test is carried out within \halign/\valign:
      \expandafter\firstoftwo\expandafter{%
        \expandafter\expandafter\expandafter\stopromannumeral
        \romannumeral\expandafter\secondoftwo
        \string{\CheckWhetherLeadingExplicitSpaceB#1##1 }{}%
      }{}%
    }%
  }%
}%
\expandafter\expandafter\expandafter\CheckWhetherLeadingExplicitSpace
\expandafter\expandafter\expandafter{%
\expandafter\expandafter\ifnum0=0\fi}%
%
\long\def\CheckWhetherLeadingExplicitSpaceB#1 {%
  \expandafter\CheckWhetherNull\expandafter{\firstoftwo{}#1}%
  {\Exchange{\firstoftwo}}{\Exchange{\secondoftwo}}%
  {\expandafter\expandafter\expandafter\stopromannumeral
   \expandafter\expandafter\expandafter}%
   \expandafter\secondoftwo\expandafter{\string}%
}%
%%-----------------------------------------------------------------------------
%% Trim leading space-tokens-loop
%%-----------------------------------------------------------------------------
\long\def\TrimLeadingSpaces#1{%
  \CheckWhetherLeadingExplicitSpace{#1}{%
    \expandafter\TrimLeadingSpaces\expandafter{\removespace#1}%
  }{%
    \PassFirstToSecond{#1}%
  }%
}%
%%-----------------------------------------------------------------------------
%% Generic mechanism for defining
%% - an expandable mechanism for extracting the first element of a 
%%   <Delimiter>-separated list of elements.
%% - an expandable mechanism for removing the first element of a 
%%   <Delimiter>-separated list of elements 
%%   and thus delivering the remaining elements of that list.
%%-----------------------------------------------------------------------------
%% \DefineFirstDelimitedArgExtractor{<ExtractorMacro>}%
%%                                  {<RemoverMacro>}%
%%                                  {<Delimiter>}%
%%                                  {<SecondDelimitedArgRemoverMacro>}%
%%                                  {<PrependedBracesRemoverMacro>}%
%%                                  {<FirstDelimitedArgRemoverMacro>}%
%%-----------------------------------------------------------------------------
%% Defines macro-mechanism consisting of <ExtractorMacro>, 
%% <SecondDelimitedArgRemoverMacro> and <BraceRemoverMacro> where
%% <ExtractorMacro>{<Arg>} 
%% - in case <Delimiter> is present in <Arg> delivers the 1st <Delimiter>-
%%   delimited argument of <Arg>, without surrounding braces removed if present
%% - in case <Delimiter> is not present in <Arg> delivers <Arg>
%% within two expansion-steps.
%%
%% Defines macro-mechanism consisting of <RemoverMacro> and
%% <FirstDelimitedRemoverMacro> where
%% <RemoverMacro>{<Arg>}
%% - in case <Delimiter> is present in <Arg> delivers <Arg> with the 1st
%%   <Delimiter>-delimited argument and the 1st <Delimiter> removed
%% - in case <Delimiter> is not present in <Arg> delivers nothing
%% within two expansion-steps.
\long\def\DefineFirstDelimitedArgExtractor#1#2#3#4#5#6{%
  \long\def#6##1#3{}%
  \long\def#4##1#3##2#3{{##1}#3}%
  \long\def#5##1#3{\firstoftwo##1}%
  \long\def#1##1{%
    \romannumeral\ExtractFirstDelimitedLoop{{}{}##1#3}{#4}{#5}{#6}%
  }%
  \long\def#2##1{%
    \romannumeral\expandafter\CheckWhetherNull\expandafter{#6##1#3}%
    {\stopromannumeral}{\expandafter\stopromannumeral#6##1}%
  }%
}%
\long\def\ExtractFirstDelimitedLoop#1#2#3#4{%
  % #1= {}{}##1<Delimiter>
  % #2= <SecondDelimitedArgRemoverMacro>
  % #3= <PrependedBracesRemoverMacro>
  % #4= <FirstDelimitedArgRemoverMacro>
  \expandafter\CheckWhetherNull\expandafter{#4#1}%
  {\expandafter\expandafter\expandafter\stopromannumeral#3#1}%
  {\expandafter\ExtractFirstDelimitedLoop\expandafter{#2#1}{#2}{#3}{#4}}%
}%
%%-----------------------------------------------------------------------------
%% Define the extraction and the removal of a space-delimited argument:
%%-----------------------------------------------------------------------------
\DefineFirstDelimitedArgExtractor{\ExtractFirstSpaceDelimitedArg}%
                                 {\RemoveFirstSpaceDelimitedArg}%
                                 { }%
                                 {\RemoveSecondSpaceDelimitedArg}%
                                 {\RemoveBracesFromSpaceDelimitedArg}%
                                 {\GobbleFirstSpaceDelimitedArg}%
%%-----------------------------------------------------------------------------
%% \trimalltrailingspaces{<Arg>}%
%%
%% After two expansion-steps delivers <Arg> with all trailing spaces removed
%% Algorithm:
%% Initate a loop with a tail-recursive macro with 3 arguments:
%% 1. <Space from previous iteration>. Initially empty.
%% 2. <Argument without trailing spaces gathered so far>. Initially empty.
%% 3. <Remaining argument to process>. Initially <Arg><space>.
%%
%% If <Remaining argument to process> is not blank:
%% Call \trimalltrailingspaces again, with
%%  <Argument without trailing spaces gathered so far>=
%%     <Argument without trailing spaces gathered so far>+<Space from previous iteration>+
%%     <Extraction of 1st space-delimited Arg from <Remaining argument to process> >;
%%  <Remaining argument to process>=<Remaining argument to process> with 1st space-delimited arg removed;
%%  <Space from previous iteration>=<explicit space token>;
%%
%% If <Remaining argument to process> is blank: 
%% Deliver <Argument without trailing spaces gathered so far>
%%  
%%-----------------------------------------------------------------------------
\long\def\trimalltrailspaces#1{\romannumeral\trimalltrailspacesloop{}{}{#1 }}%
\long\def\trimalltrailspacesloop#1#2#3{%
  \CheckWhetherBlank{#3}{\stopromannumeral#2}{%
    \expandafter\expandafter\expandafter\PassFirstToSecond
    \expandafter\expandafter\expandafter{%
      \RemoveFirstSpaceDelimitedArg{#3}%
    }{%
      \expandafter\PassFirstToSecond\expandafter{%
        \romannumeral
        \expandafter\expandafter\expandafter\Exchange
        \expandafter\expandafter\expandafter{\ExtractFirstSpaceDelimitedArg{#3}}%
                                            {\stopromannumeral#2#1}%
      }{%
        \trimalltrailspacesloop{ }%
      }%
    }%
  }%
}%
%%-----------------------------------------------------------------------------
%% \macro..\pgfeov
%%.............................................................................
\def\gobblefrozenrelax#1{%
  \def\gobblefrozenrelax#1{}%
  \long\def\macro{\romannumeral\innermacro#1}%
}%
\expandafter\expandafter\expandafter\gobblefrozenrelax
\expandafter\expandafter\expandafter{%
\expandafter\expandafter\ifnum0=0\fi}%
%
\long\def\innermacro#1\pgfeov{%
  \expandafter\SyntaxChecker\expandafter{%
    \romannumeral
    \trimalltrailspaces{\expandafter\stopromannumeral\gobblefrozenrelax#1}%
  }%
  {\singlecomponentmacro}{\twocomponentsmacro}{\errormessagemacro}{#1}%
}%
\long\def\SyntaxChecker#1#2#3#4#5{%
   \TrimLeadingSpaces{#1}{\CheckWhetherBrace}{%
     \expandafter\CheckWhetherNull\expandafter{\firstoftwo{}#1}{%
       \expandafter\firstoftwo\firstoftwo
     }{%
        \expandafter\TrimLeadingSpaces\expandafter{\firstoftwo{}#1}{\CheckWhetherBrace}{%
           \expandafter\expandafter\expandafter\CheckWhetherNull
           \expandafter\expandafter\expandafter{%
             \expandafter\firstoftwo\expandafter{\expandafter}\firstoftwo{}#1%
           }{\expandafter\secondoftwo\firstoftwo}{\secondoftwo}%
        }{\secondoftwo}%
     }%
   }{\secondoftwo}%
   {{\expandafter\stopromannumeral#2}{\expandafter\stopromannumeral#3}#1}%
   {\expandafter\PassFirstToSecond\expandafter{\gobblefrozenrelax#5}{\expandafter\stopromannumeral#4}}%
}%

\long\def\singlecomponentmacro#1{%
  Macro's argument is a single component, here shown with the outermost curly
  braces removed due to processing as delimited argument, nested between 
  {\tt||}: Component~1: {\tt\detokenize{|#1|}}%
}%

\long\def\twocomponentsmacro#1#2{%
  Macro's argument has two components, here shown with the outermost curly
  braces removed due to processing as delimited arguments, nested between 
  {\tt||}: Component~1: {\tt\detokenize{|#1|}} Component~2: {\tt\detokenize{|#2|}}%
}%

\long\def\errormessagemacro#1{%
  Macro's argument is between {\tt||}: {\tt\detokenize{|#1|}}\hfill\break
  {\bf It does not match the requested pattern of consisting of one or two
     brace-nested arguments each of these brace-nested arguments probably
     preceded/trailed by some explicit space tokens that would be discarded!%
  }%
}%


% Change the papersize %%%%%%%%%%%%%%%%%%%%%%%%%%%%
\advance\vsize by 5cm
\expandafter\ifx\csname XeTeXrevision\endcsname\relax\expandafter\firstoftwo\else\expandafter\secondoftwo\fi
{%
  \expandafter\ifx\csname directlua\endcsname\relax\expandafter\firstoftwo\else\expandafter\secondoftwo\fi
  {%
    \expandafter\ifx\csname pdfoutput\endcsname\relax\expandafter\firstoftwo\else\expandafter\secondoftwo\fi
  }{%
    \secondoftwo
  }%
}{\firstoftwo}%
{\special{papersize=210mm,347mm}}%
{%
  \ifnum\expandafter\ifx\csname pdfoutput\endcsname\relax1\else\csname pdfoutput\endcsname\fi=0 
     \special{papersize=210mm,347mm}%
  \fi
  \expandafter\ifx\csname pdfpageheight\endcsname\relax\else\advance\pdfpageheight by 5cm \fi
  \expandafter\ifx\csname pageheight\endcsname\relax\else\advance\pageheight by 5cm \fi
}%
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%


\noindent
\macro{bla}\pgfeov

\noindent\hrulefill\null

\noindent
\macro{bla} \pgfeov

\noindent\hrulefill\null

\noindent
\expandafter\macro\firstoftwo{}{} {bla}\pgfeov

\noindent\hrulefill\null

\noindent
\macro{bla}{blu}\pgfeov

\noindent\hrulefill\null

\noindent
\expandafter\macro\firstoftwo{}{} {bla} {blu}\pgfeov

\noindent\hrulefill\null

\noindent
\macro{bla} {blu}\pgfeov

\noindent\hrulefill\null

\noindent
\macro{bla}{blu} \pgfeov

\noindent\hrulefill\null

\noindent
\macro{bla} {blu} \pgfeov

\noindent\hrulefill\null

\noindent
\macro{bla}{bla}{bla}\pgfeov

\noindent\hrulefill\null

\noindent
\macro{bla}{bla}{bla} \pgfeov

\noindent\hrulefill\null

\noindent
\macro bl\pgfeov

\noindent\hrulefill\null

\noindent
\macro b{l}\pgfeov

\noindent\hrulefill\null

\noindent
\macro {b}l\pgfeov

\noindent\hrulefill\null

\noindent
\macro bla\pgfeov

\noindent\hrulefill\null

\noindent
\macro \pgfeov

\noindent\hrulefill\null

\noindent
\expandafter\macro\firstoftwo{}{} \pgfeov

\bye

enter image description here

1
  • In my last example with \ExtractFirstDelimitedArgLoop and <PrependedBracesRemoverMacro> TeX prepends (instead of a single token) two empty brace-groups, i.e., {}{} to the argument that is to be processed as a delimited one, and laterwards removes them via \firstoftwo. Two brace-groups instead of just one because in case the argument where to prepend it was empty, TeX would remove a single brace-group in the wrong moment, taking it for a brace-group surrounding an entire otherwise empty delimited argument. Jun 18, 2021 at 12:55
0

The tokcycle package can scan the tokens in advance and save them. Here, \macro is defined with the delimiter \pgfeov. The macro \macroaux is what is provided the original argument as #1, braces intact. Here, I just define \macroaux to detokenize what it was passed.

\documentclass{article}
\usepackage{tokcycle}[2021/03/10]
\usepackage[T1]{fontenc}

\xtokcycleenvironment\macro
  {\whennotprocessingparameter##1{\addcytoks{##1}}}
  {\processtoks{##1}}
  {\addcytoks{##1}}
  {\addcytoks{##1}}
  {\let\pgfeov\endtokcycraw}
  {\expandafter\cytoks\expandafter{\expandafter\cytoks\expandafter{\the\cytoks}}
   \tcafterenv{\expandafter\macroaux\expandafter{\the\cytoks}}}

\newcommand\macroaux[1]{\detokenize{#1}}
\begin{document}
\macro A\pgfeov

\macro{A}\pgfeov

\macro{x{A}yz}x\pgfeov

\macro q{x{{{A}}}yz}x{B}\pgfeov

\newcommand\Q[1]{\macro a{#1}c\pgfeov}
\Q b

\renewcommand\macroaux[1]{#1\Q x}
\macro\def\Q#1{[#1]}\pgfeov
\Q y
\end{document} 

enter image description here

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.