# Purpose of \expandafter in a particular macro

Consider the following set of definitions...

\documentclass[12pt]{article}

\def\foreach{%
\begingroup
\catcode\^^M=12 \xforeach}

{\catcode\^^M=12
\gdef\xforeach #1^^M{%
\first #1,,\endgroup}}

\def\first #1,{%
\if,#1,
\else Do something with #1'.\par\expandafter\first
\fi}

\begin{document}
\foreach a,cde
\end{document}


I would be happy if you could explain to me the meaning (or function) of \expandafter in the macro \first defined above.

My guess is that the \expandafter closes the \if-test and, consequently, \first is inserted. I wouldn't be surprised if I'm wrong.

I will appreciate a detailed description of experienced TeX users.

• Your guess is correct. The \expandafter skips over \first and expands \fi, thus removing it so that the conditional is propeproperly ended before \first is used a second time (I love the contradiction :-) – Phelype Oleinik Dec 31 '19 at 0:17
• @PhelypeOleinik: Could you rewrite your comment as a regular answer to accept it? – Marian G. Dec 31 '19 at 0:22
• The reason is that \first needs to grab the argument AFTER #1, but the \fi is in the way. – John Kormylo Dec 31 '19 at 2:07
• See also overleaf.com/learn/latex/Articles/… for a case study, which involves a more elaborate ...\expandafter{\number#2\expandafter}\fi... construction. – Ruixi Zhang Dec 31 '19 at 2:11
• Is there any particular reason for using the endline as a delimiter for the argument to \foreach and not simply using braces? – egreg Dec 31 '19 at 17:05

Your guess is correct. \expandafter always skips over one token (in this case, \first) and expands the next one (in this case, \fi). The expansion of \fi (or \else, which are somewhat similar) ends the current conditional and removes the \fi token.

Suppose the \expandafter wasn't there: the first iteration of \first (which could use a better name, in my opinion) would be:

\if,a,
\else Do something with a'.\par\first
\fi cde,,\endgroup


The \if test would be false, so The \else branch would be taken. Do something with a'.\par would be typeset and then \first would expand once more. This time, \first would grab (everything to the next ,) \fi cde as argument, and the next iteration would be:

%   V----V frozen \relax
\if,\relax\fi cde,
\else Do something with \fi cde'.\par\first
\fi cde,,\endgroup


Now the \if test would see the \fi token before the conditional was complete, so TeX would insert a frozen \relax and the test \if,\relax would yield false, so TeX would skip to the next \fi, which is the one right after the \relax. Now cde, would be (wrongly) typeset and then TeX, still in the \else branch of the first iteration of \first, would see another \else and would complain:

! Extra \else.
\first #1,->\if ,#1, \else
Do something with #1'.\par \first \fi
l.19 \foreach a,cde


That said, I'd change your code a bit:

1. I'd issue the \endgroup before the loop starts, so that you wouldn't need to worry, for example, with the common issue of definitions inside a PGF \foreach;
2. I'd move all the Do something with #1' outside the conditional, so that possible conditional tokens (\if, \else, \fi, etc.) in the argument won't interfere with the conditional of the loop; and
3. I'd use a safer emptiness test (see here for some examples); \if,#1, is dangerous in case of conditional tokens in the argument (as you saw above) and in case the item you're looping contains a comma as in, for example \foreach a,{,b},c. Better yet, I'd use a unique token to test the end of the loop, so that empty items are allowed (they are not in your current code).

That said, here's the changed code:

\documentclass{article}
\makeatletter
\def\foreach{%
\begingroup
\catcode\^^M=12 \xforeach}
{\catcode\^^M=12
\gdef\xforeach #1^^M{%
\endgroup%
\first #1,,}}
\def\first #1,{%
\if\relax\detokenize{#1}\relax
\expandafter\@firstoftwo
\else
\expandafter\@secondoftwo
\fi
{\@gobble}%
{Do something with #1'.\par}%
\first}
\makeatother
\begin{document}
\foreach a,cde
\end{document}


You could also extend it to put the do something code inline:

\documentclass{article}
\makeatletter
\def\foreach{%
\begingroup
\catcode\^^M=12 \xforeach}
{\catcode\^^M=12
\long\gdef\xforeach #1^^M#2{%
\endgroup%
\def\marian@temp##1{#2}%
\first #1,,}}
\def\first #1,{%
\if\relax\detokenize{#1}\relax
\expandafter\@firstoftwo
\else
\expandafter\@secondoftwo
\fi
{\@gobble}%
{\marian@temp{#1}}%
\first}
\makeatother
\begin{document}
\foreach a,cde
{Do something with #1'.\par}
\end{document}


Or with expl3's \clist_map_inline:nn:

\documentclass{article}
\usepackage{expl3}
\ExplSyntaxOn
\cs_new_protected:Npn \foreach
{
\group_begin:
\char_set_catcode_other:N \^^M
\__marian_foreach:wn
}
\group_begin:
\char_set_catcode_other:N \^^M
\cs_new_protected:Npn \__marian_foreach:wn #1 ^^M #2
{
\group_end:
\clist_map_inline:nn {#1} {#2}
}
\group_end:
\ExplSyntaxOff
\begin{document}
\foreach a,cde
{Do something with #1'.\par}
\end{document}

• Thank you for your detailed instructions. In fact, I'm interested only in plain TeX solutions. Regardless of this, expl3 notation contains a lot of spaces and underscores which is hard to read (at least for me and at this moment). – Marian G. Dec 31 '19 at 9:04
• @MarianG. You're welcome! I wrote the code in LaTeX because your code was too, but all of them (including the expl3 one, given you load it with \input expl3-generic) should work in Plain (you might need to define \def\@gobble#1{}, \def\@firstoftwo#1#2{#1} and \def\@secondoftwo#1#2{#2}). If you want to use Knuth TeX (that is, no e-TeX), then you need a different emptiness test (which doesn't rely on \detokenize); I recommend the Fast \ifempty test in this answer. – Phelype Oleinik Dec 31 '19 at 16:55
• @MarianG. Also, expl3 isn't that hard, once you get used to its verbosity :-) In case you're interested, the expl3 code is more or less a one-to-one adaptation of the one above: \cs_new_protected:Npn replaces \def (the definition is \protected because it's not safely expandable), \group_begin: replaces \begingroup, \char_set_catcode_other:N does as it advertises, and \__marian_foreach:wn replaces \xforeach with a proper namespace. In \__marian_foreach:wn, \group_end: replaces \endgroup and \clist_map_inline:nn` does the looping over the comma-separated list. – Phelype Oleinik Dec 31 '19 at 16:58