4

Long time reader, first time poster.

I have a question about creating a macro that iterates through a list. I've read through some questions posted by other users, but all of those have something to do with using LaTeX in a complicated way. My question's about using plain TeX to iterate through a list and construct a new list by modifying each element in the old list. I'm currently self-learning TeX and I think programming some basic, but versatile and robust macros will help me get a better sense of how the guts of TeX work. Anyways, enough background, onto my question.

This is my code so far:

I defined the iterator recursively. It reads one argument at a time, determines if the argument is a comma or a stop marker (of my own definition, called \myStop), moves on if it's a comma, stops if it's the stop marker, and otherwise replaces the element by itself and the string (or token list) "+ 1".

\edef\myStop{)}
\def\Iterator#1{
        \ifx\myStop#1 %do nothing {end of list}
            \else %
                \ifx,#1 \expandafter\IteratorIntermediateOne % currently 2 if-statements deep
                    \else #1 + 1, \expandafter\IteratorIntermediateOne %currently 2 if-statements deep
            \fi%
        \fi%
}

I've had to define an intermediate Iterator called IteratorIntermediateOne to house the command \expandafter\Iterator because I'm not currently aware of a way to group terms after an \expandafter command in a way that would be equivalent to something like \expandafter{\expandafter\Iterator}\fi\fi. So I guess that's my first question: Is there a way to define nested \expandafter commands?

Now that everything's in context, here's all my code:

\edef\MyList{1,2,3}
\edef\myStop{)}
\def\IteratorIntermediateOne{\expandafter\Iterator}
\def\Iterator#1{%
        \ifx\myStop#1 %do nothing {end of list}
            \else %
                \ifx,#1 \expandafter\IteratorIntermediateOne % currently 2 if-statements deep
                    \else #1 + 1, \expandafter\IteratorIntermediateOne %currently 2 if-statements deep
            \fi%
        \fi%
}

\edef\newList{\expandafter\Iterator\MyList\myStop}

Disclaimer: I'm aware of the extra comma this places after the last element. I am also aware there aren't enough cases to detect poorly constructed lists. I don't know enough about TeX to even begin to imagine how to safeguard the macro against improperly constructed token lists, so apologies if the more experienced of you out can't help but laugh at this code.

Okay, my other question is this: Is there a more efficient way to define a macro that calls itself? Is it better practice to define an iterator macro separately and call it using TeX's built-in \loop command? If so, can someone walk me through how I would do that, because I'm having trouble making sense of the loop call in the context of TeX's eye's, mouth, gullet, and stomach processes. Does the loop expand the macro, reach \repeat pass it on to the stomach and return back to the macro? I can't find a good explanation anywhere.

Thanks for all your help!

17
  • \loop isn't built in it's just a recursive macro definition not so unlike this Aug 27, 2020 at 16:49
  • it's not clear what your input syntax is, you are iterating token by token not over comma separated items so are you assuming eacg item is a singl etoken or could you have 123,456 currently you would loop 7 iterations over that not two. or are you assuming that the input would be {123},{456} in that case (but then you can not use ifx tests) Aug 27, 2020 at 16:52
  • to expand both fi the usual idiom is \expandafter\expandafter\expandafter\Iterator\fi\fi. Aug 27, 2020 at 16:54
  • @DavidCarlisle Hi David, yes, I am assuming my input would be a single-digit list like {1,2,3,4,6}.
    – Harry
    Aug 27, 2020 at 17:02
  • @DavidCarlisle So \expandafter\expandafter\expandafter\Iterator\fi\fi means \expandafter{\expandafter{\expandafter....?
    – Harry
    Aug 27, 2020 at 17:04

4 Answers 4

5

The intention seems to be to add 1 to each item I would code it more like this (assuming etex)

\edef\MyList{1,2,3,25,456,2}

\def\Iterator#1{\expandafter\xiterator#1\stopiteration,}

\def\xiterator#1,{\the\numexpr#1+1\relax,\xiterator}
\def\stopiteration#1\relax#2\xiterator{#1\relax}


\message{\Iterator\MyList}

\bye

which makes the message

2,3,4,26,457,3 
1
  • Thanks David! This helped a lot. Your code is efficient and makes me realize I have a long way to go!
    – Harry
    Aug 29, 2020 at 1:49
3
\input listofitems
\def\MyList{1,2,3,25,456,2}
\def\processlist#1{%
  \readlist\myterms\MyList
  \foreachitem\z\in\myterms{%
    \ifnum\zcnt=1\else,\fi
    \the\numexpr\z+1\relax
  }%
}

\processlist\Mylist
\bye

enter image description here

If you actually need to save the updated list, we can do so in the \mytoks token list:

\input listofitems
\newtoks\mytoks
\def\MyList{1,2,3,25,456,2}
\def\processlist#1{%
  \mytoks{}%
  \readlist\myterms\MyList
  \foreachitem\z\in\myterms{%
    \ifnum\zcnt=1\else\mytoks\expandafter{\the\mytoks,}\fi
    \mytoks\expandafter\expandafter\expandafter{%
      \expandafter\the\expandafter\mytoks\the\numexpr\z+1\relax}
  }%
}

\processlist\Mylist

List is \the\mytoks
\bye

enter image description here

2
  • Thanks for your input Steven. Your code is a little advance for me atm. It has a number of commands like \readlist that I have not seen before, and while the \foreachitem command is clearly a loop command, I'm not aware of how TeX would process it. I'm still just getting my feet wet. Better to stick to the simple solutions for now!
    – Harry
    Aug 29, 2020 at 1:55
  • @Harry This solution is based on the listofitems package, which is available to both LaTeX as well as plain TeX. I can understand your desire to stick to basics. Aug 29, 2020 at 2:53
2

You said:

...so apologies if the more experienced of you out can't help but laugh at this code.

When I started learning TeX, I had the feeling that it was a very steep learning curve.

From time to time I was frustrated. ;-)

I do not believe that people like you, who are taking on this learning curve, are in a situation where it is appropriate to laugh at their attempts at macro programming/TeX-programming.

I think any attempt to achieve or learn something good in a way which both is also based on reason and in itself is no misdeed, basically deserves respect.

If you have questions about how the code from my examples below works, don't hesitate to ask them. It is then useful to describe how you think the code works and where you get stuck with the understanding. In my experience, this makes it easier for the respondents to find out exactly what information (e.g. about how TeX primitives work and which of the "side effects" briefly hinted at in the back chapters of the TeXbook are used for programming tricks) is still missing for understanding.


Assuming that the entries in your comma-list are not surrounded by spaces, and that the \relax-primitive does not occur within the comma-list, and that \numexpr from the ε-TeX-extensions is available, you can probably do something like this:

\long\def\gobble#1{}%
\long\def\firstofone#1{#1}%

\def\Iterator#1#2,{%
  % #1 - element-separator to prepend; empty in the 1st iteration;
  %      comma in consecutive iterations
  % #2 - either current element of old list or the \relax that was
  %      appended for denoting the end of the list
  \ifx\relax#2\expandafter\gobble\else\expandafter\firstofone\fi
  {%
    #1\number\numexpr#2+1\relax\Iterator{,}%
  }%
}%

\def\MyList{1,2,3}

\edef\newList{\expandafter\Iterator\expandafter{\expandafter}\MyList,\relax,}

\begingroup

\tt

\string\MyList: \meaning\MyList

\string\newList: \meaning\newList

\endgroup

\bye

The gist with this example is:

Within the \edef-definition-text of \newList the TeX-primitive \expandafter is used for expanding \MyList. Also the sequence ,\relax, is appended.

This way at the time of defining \newList \edef-driven expansion of the definition-text of \newList at some stage yields a sequence \Iterator{}Comma,sparated,items,from,\MyList,\relax,.

So \relax, marks the end of the list.

Now—still driven by \edef-expansion—\Iterator (recursively) picks a non-delimited-argument #1 (which in the first iteration is empty and in consecutive iterations holds a comma, i.e., which holds the separator to prepend to the item of the new list) and a comma-delimited-argument #2 which either holds the next item coming from \myList's comma-list or holds the end-marker \relax , and in any case places—nested in curly braces— a token-sequence denoting the next iteration, formed by

  • the undelimited-argument #1, i.e., the separator that must precede the next item of the new list,
  • the expression \number\numexpr#2+1\relax for adding 1 to the value represented by the comma-delimited-argument and this way forming the next item of the new list,
  • a call to itself for processing the next item remaining from the expansion of \myList, this time providing a comma within the undelimited argument, denoting that next time the next item of the new list is to be preceded by a comma.

Via \ifx\relax#2 it is checked whether the end of the comma-list/the \relax appended to the list in the beginning of \edef-expansion is reached. If so, the token-sequence nested in curly braces denoting the next iteration is "gobbled/removed" via \gobble and thus not carried out, which terminates iteration/tail-recursion. If not so, the surrounding curly braces are removed from that sequence by applying \firstofone whereafter that sequence gets processed.

The undelimited argument #1 of \Iterator, which holds the separator to prepend to the item of the new list, only in the first iteration is empty. In each consecutive iteration it holds a comma which in that consecutive iteration's previous iteration was provided by the \Iterator-macro itself as a part of the token-sequence which then formed the next iteration. This way (only) the first item of the new list is not preceded by a comma.


If you don't have ε-TeX-extensions' \numexpr available I can offer a routine for incrementing non-negative integer numbers. (In "real life" you might be interested in the packages intcalc and bigintcalc.)

%------------------------------------------------------------------------------
% Expandable incrementing of non-negative integer number formed by a sequence
% of explicit catcode-12-character-tokens from the set {0,1,2,3,4,5,6,7,8,9}
%..............................................................................
% \Increment{<non-negative integer number k as sequence of explicit
%              catcode-12-character-tokens from the set 0123456789>}
% ->
% <natural number (k+1) as sequence of explicit catcode-12-character-tokens
%  from the set 0123456789>
% In expansion-contexts the result is delivered after two expansion-steps/is 
% obtained by "hitting" \Increment with \expandafter twice.
%------------------------------------------------------------------------------
\def\Increment#1{%
  \romannumeral0%
  \IncrementReverse{\IncrementFork{}}{\relax}{}#1\relax
}%
\def\IncrementReverse#1#2#3#4{%
  % #1 - tokens to prepend to reversed list
  % #2 - tokens to append to reversed list
  % #3 - reversed list constructed so far
  % #4 - current element of not-reversed list
  \ifx\relax#4%
    \expandafter\firstoftwo
  \else
    \expandafter\secondoftwo
  \fi
  {#1#3#2}{\IncrementReverse{#1}{#2}{#4#3}}%
}%
\def\IncrementSelect#10123456789\relax#2#3!!{#2}%
\def\IncrementFork#1#2{%
  % #1 - digits incremented so far
  % #2 - current digit to increment or end-marker \relax
  \IncrementSelect
  #2123456789\relax{\IncrementReverse{ }{}{}#11}%
  0#223456789\relax{\IncrementReverse{ }{}{}#12}%
  01#23456789\relax{\IncrementReverse{ }{}{}#13}%
  012#2456789\relax{\IncrementReverse{ }{}{}#14}%
  0123#256789\relax{\IncrementReverse{ }{}{}#15}%
  01234#26789\relax{\IncrementReverse{ }{}{}#16}%
  012345#2789\relax{\IncrementReverse{ }{}{}#17}%
  0123456#289\relax{\IncrementReverse{ }{}{}#18}%
  01234567#29\relax{\IncrementReverse{ }{}{}#19}%
  012345678#2\relax{\IncrementFork{#10}}%
  0123456789#2{\IncrementReverse{ }{}{}#11\relax}%
  0123456789\relax{\IncrementReverse{ }{}{}#11#2}%
  !!%
}%
%%-----------------------------------------------------------------------------
\long\def\firstoftwo#1#2{#1}%
\long\def\secondoftwo#1#2{#2}%

\def\Iterator#1#2,{%
  % #1 - element-separator to prepend
  % #2 - current element of old list
  \ifx\relax#2\expandafter\firstoftwo\else\expandafter\secondoftwo\fi
  {}{%
    #1\Increment{#2}\Iterator{,}%
  }%
}%

\def\MyList{1,2,3}

\edef\newList{\expandafter\Iterator\expandafter{\expandafter}\MyList,\relax,}

\begingroup

\tt

\string\MyList: \meaning\MyList

\string\newList: \meaning\newList

\endgroup

\bye

If you wish a routine which does without \edef, you can, e.g., use the \romannumeral0-expansion- and argument-exchanging-technique—the gist of \romannumeral0-expansion is:

  • TeX expands expandable tokens while gathering tokens that belong to the ⟨number⟩-quantity that is to be represented in roman numerals.
  • If the first token which TeX finds while gathering the ⟨number⟩-quantity is a digit, e.g., 0, then the process of gathering tokens that belong to the ⟨number⟩-quantity turns into a process of gathering more digits or something that is not a digit and therefore terminates the process of gathering. Expandable tokens get expanded while gathering digits. A space-token terminating a digit-sequence terminates the process of gathering more digits and gets silently discarded.
  • If the number gathered is not positive, TeX will silently swallow the tokens forming the ⟨number⟩-quantity without delivering any token in return.

This implies that \romannumeral can be used for tricking TeX into doing a lot of expansion- and argument-exchanging-work as long as it is ensured that in the end a non-positive number is found.

%------------------------------------------------------------------------------
% Expandable incrementing of non-negative integer number formed by a sequence
% of explicit catcode-12-character-tokens from the set {0,1,2,3,4,5,6,7,8,9}
%..............................................................................
% \Increment{<non-negative integer number k as sequence of explicit
%              catcode-12-character-tokens from the set 0123456789>}
% ->
% <natural number (k+1) as sequence of explicit catcode-12-character-tokens
%  from the set 0123456789>
% In expansion-contexts the result is delivered after two expansion-steps/is 
% obtained by "hitting" \Increment with \expandafter twice.
%------------------------------------------------------------------------------
\def\Increment#1{%
  \romannumeral0%
  \IncrementReverse{\IncrementFork{}}{\relax}{}#1\relax
}%
\def\IncrementReverse#1#2#3#4{%
  % #1 - tokens to prepend to reversed list
  % #2 - tokens to append to reversed list
  % #3 - reversed list constructed so far
  % #4 - current element of not-reversed list
  \ifx\relax#4%
    \expandafter\firstoftwo
  \else
    \expandafter\secondoftwo
  \fi
  {#1#3#2}{\IncrementReverse{#1}{#2}{#4#3}}%
}%
\def\IncrementSelect#10123456789\relax#2#3!!{#2}%
\def\IncrementFork#1#2{%
  % #1 - digits incremented so far
  % #2 - current digit to increment or end-marker \relax
  \IncrementSelect
  #2123456789\relax{\IncrementReverse{ }{}{}#11}%
  0#223456789\relax{\IncrementReverse{ }{}{}#12}%
  01#23456789\relax{\IncrementReverse{ }{}{}#13}%
  012#2456789\relax{\IncrementReverse{ }{}{}#14}%
  0123#256789\relax{\IncrementReverse{ }{}{}#15}%
  01234#26789\relax{\IncrementReverse{ }{}{}#16}%
  012345#2789\relax{\IncrementReverse{ }{}{}#17}%
  0123456#289\relax{\IncrementReverse{ }{}{}#18}%
  01234567#29\relax{\IncrementReverse{ }{}{}#19}%
  012345678#2\relax{\IncrementFork{#10}}%
  0123456789#2{\IncrementReverse{ }{}{}#11\relax}%
  0123456789\relax{\IncrementReverse{ }{}{}#11#2}%
  !!%
}%
%%-----------------------------------------------------------------------------

\long\def\firstoftwo#1#2{#1}%
\long\def\secondoftwo#1#2{#2}%
\long\def\exchange#1#2{#2#1}%

\def\Iterator#1,#2\relax#3#4{%
  % #1 - current element of old list
  % #2 - remaining elements of old list
  % #3 - element-separator to prepend
  % #4 - new list constructed so far
  \ifx\relax#1\expandafter\firstoftwo\else\expandafter\secondoftwo\fi
  { #4}{%
    \expandafter\exchange
    \expandafter{%
    \expandafter{%
    \romannumeral0%
    \expandafter\expandafter\expandafter\exchange
    \expandafter\expandafter\expandafter{%
    \Increment{#1}}{ #4#3}}}{\Iterator#2\relax{,}}%
  }%
}%

\def\MyList{0,1,2,3}

\expandafter\def
\expandafter\newList
\expandafter{%
\romannumeral0\expandafter\Iterator\MyList,{\relax},\relax{}{}}%

\begingroup

\tt

\string\MyList: \meaning\MyList

\string\newList: \meaning\newList

\endgroup

\bye
1
  • Thanks Ulrich! Your answer is much appreciated! Unfortunely, I don't know enough TeX to actually use it in any productive way. I'm afraid most of it's beyond me. I'm still just learning the basics of TeX. However, I will return to it in a few weeks to see if I've made any progress! In the meantime, all I can do is thank you for your effort!
    – Harry
    Aug 29, 2020 at 2:01
0

Since you're new, you can start with expl3.

\documentclass{article}
\usepackage{xparse,xfp}

\ExplSyntaxOn

\NewDocumentCommand{\generatelist}{mmm}
 {% #1=output, #2=input, #3=iterator
  \harry_list_generate:nnn { #1 } { #2 } { #3 }
 }

% variables
\clist_new:N \l__harry_list_input_clist
\clist_new:N \l__harry_list_output_clist

% the main function
\cs_new_protected:Nn \harry_list_generate:nnn
 {
  % if the input is a single token, assume it is a control sequence
  \tl_if_single:nTF { #2 }
   { \clist_set_eq:NN \l__harry_list_input_clist #2 }
   { \clist_set:Nn \l__harry_list_input_clist { #2 } }
  % now \l__harry_list_input_clist contains the input

  % clear the output list
  \clist_clear:N \l__harry_list_output_clist

  % map the input list applying the iterator to each item
  \clist_map_inline:Nn \l__harry_list_input_clist
   {
    \clist_put_right:Nx \l__harry_list_output_clist { #3 { ##1 } }
   }

  % make the output list
  \clist_set_eq:NN #1 \l__harry_list_output_clist
 }

\ExplSyntaxOff

% two example iterators
\newcommand{\addone}[1]{\inteval{#1+1}}
\newcommand{\addhyphens}[1]{-#1-}

% a control sequence expanding to a list
\newcommand{\List}{1,2,3,41}

\generatelist{\ListA}{\List}{\addone}

\generatelist{\ListB}{1,2,3}{\addhyphens}

\show\ListA
\show\ListB

This outputs

> \ListA=macro:
->2,3,4,42.
l.50 \show\ListA

?
> \ListB=macro:
->-1-,-2-,-3-.
l.51 \show\ListB
1
  • The expl3 package is not for beginners. The documentation is very confusing. On page 2 it says "Variables are named in a similar manner to functions, but begin with a single letter to define the type of variable." It goes on to say that one of those letters is l. Based on that, I'm assuming \l__harry_list_input_clist is a variable, and the name of the new list. On page 4 the manual says a <name> following a function means that the function "expects the name of a [function]", but it doesn't say that that name should be a variable.
    – Harry
    Sep 1, 2020 at 6:22

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .