# Completely expandable loop macro that also works in tabular

Since I normally use tikz in my documents, I mostly use the \foreach \x in \list loop to repeat things. However, this famously fails in tabular environments and also everywhere where things must be expanded.

Now most of the time I just want to repeat things a handful of times, let's say not more than 50 iterations and no nested loops or stuff like that. Consequently, performance is usually not an issue. Based on these observations I've written a (probably grossly inefficient) loop macro that uses a C-like for syntax. It works in tabular environments to generate multiple columns in a single line or - with some additional care - to generate multiple rows:

Now there are clearly some issues with this loop, but at the moment it "works for me" most of the time. Still I'd like to improve it, so these are the points that bug me the most:

1. Each iteration's loop body is completely expanded using \edef. This ensures that the loop counter is expanded to its value during that iteration. However this also requires the user to protect some things using \noexpand, like \pgfmathparse or \\ and \hline in the tables below. Also it would require double protection (3 \noexpands?) for nested loops.
2. Obviously it's not completely expandable because it uses \def and \edef, so it's not possible to use the loop inside, say, another \edef in order to define a macro that has a regular structure (although of course it's perfectly possible to \def a macro that expands to a loop invocation).

For problem 1 it would be great to have a way to expand only the loop counter anywhere it's used in the loop body, but leave everything else alone. For problem 2 (if at all possible) my guts tell me that I'd probably have to sacrifice some functionality, like the flexible loop counters and conditions (e.g. \repeat{\i}{5}{some code using \i} instead of the C-like syntax).

However I don't really know how to approach these problems. Can anyone give me some pointers in the right direction? Is it at all possible to achieve both, selective expansion and complete expandability?

Finally, here's the code:

\documentclass{article}

\begin{filecontents*}{myforloop.sty}
\usepackage{pgf}
\usepackage{pgffor}
\usepackage{ifthen}

% Example:
% \for (\i = 0; \i < 10; \i = \i + 1) {
%    some code using \i
% }

\def\for(#1=#2;#3;#4=#5)#6{%
\def\forloopresult{}%                        Macro to store the result
\pgfmathtruncatemacro{#1}{#2}%               Set initial value of counter
\def\forloopinner{%                          Recursive loop macro
\pgfmathparse{int(#3)}%                  Calculate loop condition
\ifthenelse{1=\pgfmathresult\relax}{%    if loop condition is true:
\toks@=\expandafter{\forloopresult}%   Get tokens from result macro
\edef\forloopresult{\the\toks@#6}%     Append expanded code to result
\pgfmathtruncatemacro{#1}{#5}%         Set new counter value
\forloopinner%                         Recurse
}{%                                      else:
\forloopresult%                        Expand to result
}%
}%
\forloopinner%                               Start looping
}
\end{filecontents*}
\usepackage{myforloop}

\newcommand\N{5}% Number of loop iterations

\begin{document}
\begin{tabular}{|r|*{\N}{r|}}
\hline
$x$:   \for(\x=0; \x<\N; \x=\x+1) {&\x}\\\hline
$e^x$: \for(\x=0; \x<\N; \x=\x+1) {%
& \noexpand\pgfmathparse{exp(\x)}\noexpand\pgfmathresult%
}\\\hline
\end{tabular}

\vspace{1em}

\begin{tabular}{|l|l|}
\hline
$x$ & $e^x$ \\\hline
\for(\x=0; \x<\N; \x=\x+1) {%
\x & \noexpand\pgfmathparse{exp(\x)}\noexpand\pgfmathresult
\noexpand\\\noexpand\hline
}
\end{tabular}
\end{document}

-

There are a number of possible approaches to doing this without the complexity: I'll cover a couple using expl3. First, if you don't mind keeping things non-expandable then you could do something like

\documentclass{article}
\usepackage{expl3}
\ExplSyntaxOn
\cs_new_protected:Npn \For #1#2#3#4
{
\int_step_inline:nnnn {#1} {#2} {#3 - 1} {#4}
}
\cs_new_eq:NN \fpeval \fp_eval:n
\ExplSyntaxOff
\newcommand*\N{5}
\begin{document}

\begin{tabular}{|r|*{\N}{r|}}
\hline
$x$:   \For{0}{1}{\N}{&#1}\\
\hline
$e^x$: \For{0}{1}{\N}{& \fpeval{round(exp(#1),5)}}\\
\hline
\end{tabular}
\end{document}


where the code to implement the cells can then be given at point-of-use. You could instead have a two-part setup where the functions are pre-coded: this is then expandable at point-of-use

\documentclass{article}
\usepackage{expl3}
\ExplSyntaxOn
\cs_new_protected:Npn \For #1#2#3#4
{
\int_step_function:nnnN {#1} {#2} {#3 - 1} #4
}
\cs_new_eq:NN \fpeval \fp_eval:n
\ExplSyntaxOff
\newcommand*\N{5}
\newcommand\CellNumber[1]{&#1}
\newcommand\CellExp[1]{&\fpeval{round(exp(#1),5)}}
\begin{document}
\begin{tabular}{|r|*{\N}{r|}}
\hline
$x$:   \For{0}{1}{\N}\CellNumber\\
\hline
$e^x$: \For{0}{1}{\N}\CellExp\\
\hline
\end{tabular}
\end{document}


In both cases, the \int_step_... functions work much like David's answer: the loop itself is done by expansion. You can't do an assignment to \x or similar and have expandability.

An alternative approach if you are using LuaTeX is to do everything in Lua: the primitive here is expandable so you can use a perhaps more familiar approach (the question suggests a C background).

\documentclass{article}
\newcommand\For[4]{%
\directlua{%
for x = #1, (#3 - 1), #2 do
tex.print(#4)
end
}%
}
\newcommand*\N{5}
\makeatletter
\let\Percent\@percentchar
\makeatother
\begin{document}

\begin{tabular}{|r|*{\N}{r|}}
\hline
$x$:   \For{0}{1}{\N}{"&" .. x}\\
\hline
$e^x$: \For{0}{1}{\N}{"&" .. string.format("\Percent.3f", math.exp(x))}\\
\hline
\end{tabular}
\end{document}


For the number formatting, we need % which is a bit awkward in TeX: I've given a 'user' name to the LaTeX kernel command \@percentchar (a % which isn't a comment char).

-
As I've used expl3 I've avoided the pgf math system and used our own FPU instead. It's expandable and IEE754-compliant, although it's perhaps not got quite the power of the pgf one for some cases. –  Joseph Wright Aug 18 '14 at 12:36
Note that there are no groups inside \int_step_inline:nnnn so the issues of 'stray grouping' do not apply (\int_step_function:nnnN is expandable so can't use any groups anyway!) –  Joseph Wright Aug 18 '14 at 12:37
Note that if you are using LuaTeX you could of course do the entire loop in Lua: that is expandable even with assignments. –  Joseph Wright Aug 18 '14 at 12:51
This seems really useful. I've tried it out, but it seems that in my second example \begin{tabular}{|l|l|}\hline\For{0}{1}{\N}{#1 & 1\\\hline}\end{tabular} it creates strange artifacts: image. Also, the first one doesn't seem to work in \csvreader, where my loop works, but yours complains about the #1: ! Illegal parameter number in definition of \csv@@body. Maybe I'm really better off using LuaTeX... would you mind giving an example how write and use a LuaTeX loop function/macro as an alternative to your expl3 solution? :-) –  Fritz Aug 18 '14 at 14:59
@Fritz On the artefact, you need to watch exactly where you put the end-of-line stuff: in your example this is not an issue but your snippet in the comment suggests a different structure. On the second question: you will almost certainly need to double # tokens to use #1 inside the argument of \csvreader (this is a standard TeX thing). I'll try to add a Lua answer later today. –  Joseph Wright Aug 18 '14 at 15:04

This loop just uses expansion. In order to use #1 as the loop counter it is easiest to define it anew each time, if you didn't want access to the counter you could more easily use \myrepeat{5}{some code} Note i didn't use \repeat as that name is taken in the existing \loop syntax.

\documentclass{article}

\def\myrepeat#1{\ifnum#1=0 \expandafter\stopmyrepeat\fi
[This is iteration \number#1]
\expandafter\myrepeat\expandafter{\the\numexpr#1-1\relax}}

\def\stopmyrepeat#1\myrepeat#2#3{}

\begin{document}

\myrepeat{18}

\end{document}

-
Interesting. Using this approach you have to define a new loop macro for every code using #1 that you want to repeat, right? On the other hand, Joseph's expl3 answer seems to pass the code that uses #1 as a parameter... somehow. –  Fritz Aug 18 '14 at 12:42
@Fritz no it's the same, just shuffled round, in the expandable expl3 version you still have to define the body as a macro taking #1 as the counter, although there you just do that and pass that function to an existing loop macro. A trivial change to the above would do the same, but it just makes it longer here as you'd have to define the loop macro and the function body instead of a combined macro as above. –  David Carlisle Aug 18 '14 at 12:46
@Fritz As David says, the difference here is that in expl3 we have the set up 'pre-created' so it's work having a nice interface (one set of loop functions, just provide a command to 'do stuff'). –  Joseph Wright Aug 18 '14 at 12:50
This code is buggy as soon as the argument of \myrepeat is 11 or more. (because \myrepeat fetches only the first digit) –  jfbu Mar 11 at 9:57
@jfbu oops, fixed, thanks –  David Carlisle Mar 11 at 11:07

Thanks to the inspiration from David's and Joseph's use of #1 as placeholder for the loop counter, I was able to improve the implementation that I've given in the question. I'll post it here mostly as a learning exercise for anyone who happens to have a similar use case. This version only expands the loop counter and leaves all other tokens alone, removing the need to use \noexpand to protect stuff that doesn't like being expanded like \pgfmath*.

The advantage of using a macro over Lua is in my opinion the easier specification of the loop body. With Lua one has to fight with tex.print and string.format and %-characters and stuff like that. In contrast, a macro version allows you to simply write the loop body using LaTeX code, exactly as you would write it manually without the loop.

An invocation of the loop now looks like this, where the name \i is only used internally (I left it there mostly because I like the syntax) and is not available inside the loop body. Instead #1 must be used to access the counter value:

\For(\i=0; \i<5; \i=\i+1) { Current value of i: #1 }


Of course it's still not expandable, but it seems to work amazingly well, and even nested loops are possible, using ##1 and ####1 as the nested counters. The only thing that didn't work in the quick test (see below) was using \multicolumn in a loop: It complains about Misplaced \omit., which seems to occur when there is 'stuff' before the \multicolumn command, though I couldn't figure out what exactly was the issue in this case.

Here's the compiled example, followed by the code that produced it.

\documentclass{article}

% Some test data for \csvread
\begin{filecontents*}{test.csv}
1,2,3,4,5,6,7
4,5,6,7,8,9,0
\end{filecontents*}

% The actual loop package
\begin{filecontents*}{myforloop.sty}
\usepackage{pgf}
\usepackage{ifthen}

% Example (spaces optional):
% \For (\i = 0; \i < 10; \i = \i + 1) {
%   some code using #1
% }

\newtoks\For@R% Accumulator for tokens from all iterations
\newtoks\For@B% Loop body with #1 replaced by current counter value

\def\For(#1=#2;#3;#4=#5)#6{%
\let\EA=\expandafter%                          For shorter writing ;-)
\def\For@result{}%                             Macro to store the result
\pgfmathtruncatemacro{#1}{#2}%                 Set initial value of counter
\def\For@body##1{#6}%                          Loop body for replacing counter
\def\For@inner{%                               Recursive loop macro
\pgfmathparse{int(#3)}%                    Calculate loop condition
\ifthenelse{1=\pgfmathresult\relax}{%      if loop condition is true:
\For@R=\EA{\For@result}%                 Get current result tokens
\For@B=\EA\EA\EA{\EA\For@body\EA{#1}}%   Replace *only* counter in body
\edef\For@result{\the\For@R\the\For@B}%  Append current body to result
\pgfmathtruncatemacro{#1}{#5}%           Set new counter value
\For@inner%                              Recurse
}{%                                        else:
\For@result%                             Expand to result
}%
}%
\For@inner%                                  Start looping
}
\end{filecontents*}

\usepackage{myforloop}
\usepackage{csvsimple}

\newcommand\N{4}% Number of loop iterations

\begin{document}
% Generate multiple columns in a single row
\begin{tabular}{|r|*{\N}{r|}}
\hline
$x$:   \For(\i=0; \i<\N; \i=\i+1) {&#1}\\\hline
$e^x$: \For(\i=0; \i<\N; \i=\i+1) {& \pgfmathparse{exp(#1)}\pgfmathresult}\\\hline
\end{tabular}
\\[1em]

% Generate multiple rows with a single loop
\begin{tabular}{|l|l|}
\hline
$x$ & $e^x$ \\\hline
\For(\i=0; \i<\N; \i=\i+1) {%
#1 & \pgfmathparse{exp(#1)}\pgfmathresult \\\hline
}
\end{tabular}
\\[1em]

% Generate a multi-dimensional table using nested loops
\begin{tabular}{l*{\N}{r}}
\For(\k=0; \k<3; \k=\k+1) {%
Group #1:\\
\For(\i=0; \i<\N; \i=\i+1) {%
Row ##1: \For(\j=0; \j<\N; \j=\j+1) {& #1,##1,####1} \\
}
}
\end{tabular}
\\[1em]

% Generate columns 2 to 7: \csvcoli to \csvcolvii:
\csvcoli \For(\i=2; \i<=7; \i=\i+1) {& \csname csvcol\romannumeral ##1\endcsname}
}
\\[1em]

% Some things do not work :-(
\begin{tabular}{lll}
% Complains about Misplaced \omit.
%\For(\k=0; \k<\N; \k=\k+1) {\multicolumn{3}{c}{center}}
\end{tabular}
\\[1em]

\end{document}


If you've read this far, you probably agree that there's still room for improvement in the current version. For example the two token registers and the \edef could probably be replaced by a single token register assignment. However I couldn't find the right sequence of \expandafters to expand the counter macro #1, followed by the loop body \For@body{counter-value}, followed by the current accumulator \the\For@R. The best I could think of was:

\For@R=\EA\EA\EA{\EA\the\EA\For@R\EA\For@body\EA{#1}}


but this hits some kind of internal TeX limit. And from my limited experience with the matter, it's probably not enough \EAs. ;-)

-