Building and using a list

I made a macro for subtracting two lists which does what was intended. Then, I found a set-up, here on TeX.SE, with the macro \g@addto@macro and \@gobble which is much simpler and gives, in principle, the same output.

When trying to use this second form, some problems did arise. After some searching, it became clear that the \@gobble had not been expanded yet, adding an additional element to the list and causing trouble.

What is the best solution for this problem?

I feel that it is preferable to process the resulting \processedlist so that it can be used in any macro which expects a list. Then, we get again something similar as in the first macro, a lot of \expandafters, although this time they will be used only once and, thus, it should be faster.

\documentclass{article}

\setlength{\parindent}{0pt}
\setlength{\parskip}{1.5ex plus 0.5ex minus 0.2ex}
\begin{document}
\makeatletter

\section*{First macro for subtracting two lists}

\newif\iffirstterm
\newif\ifisinlist

\def\subtractlists#1#2{% original list, remove list
\def\processedlist{}%
\firsttermtrue%
\@for\tempa:=#1\do{%
\isinlistfalse%
\@for\tempb:=#2\do{\ifx\tempa\tempb\isinlisttrue\fi}%
\ifisinlist%
\else%
\iffirstterm%
\expandafter\def\expandafter\processedlist\expandafter{\tempa}%
\firsttermfalse%
\else%
\expandafter\expandafter\expandafter\def%
\expandafter\expandafter\expandafter\processedlist%
\expandafter\expandafter\expandafter{%
\expandafter\processedlist\expandafter,\tempa}%
\fi%
\fi}%
}

%%% some lists for testing:
\def\originallist{Lima,Alpha,Delta,Oscar,Tango,Whisky,Echo,Romeo,Xray}
\def\removelist{Whisky,Oscar,Romeo,Delta}

%The list can be entered as comma separated items or as a macro:

\verb|\originallist|: \originallist\par
\verb|\removelist|: \removelist\par
\verb|\subtractlists{\originallist}{\removelist}|\par
\subtractlists{\originallist}{\removelist}
\verb|\processedlist|: \processedlist\par

%\@for\tempa:=\processedlist\do{(\tempa)}

\section*{Second macro for subtracting two lists}

\def\subtractlist#1#2{% #1:original list,#2:remove list
\def\processedlist{\@gobble}%
\@for\tempa:=#1\do{%
\isinlistfalse%
\@for\tempb:=#2\do{\ifx\tempa\tempb\isinlisttrue\fi}%
\ifisinlist%
\else%
\expandafter\processedlist%
\expandafter{\expandafter,\tempa}%
\fi}%
}

\verb|\originallist|: \originallist\par
\verb|\removelist|: \removelist\par
\verb|\subtractlist{\originallist}{\removelist}|\par
\subtractlist{\originallist}{\removelist}
\verb|\processedlist|: \processedlist\par

\section*{Problems with second approach}

There is some extra matter:

\@for\tempa:=\processedlist\do{(\tempa)}

the first comma has not been gobbled yet:

\verb|\meaning\processedlist|: \meaning\processedlist

\section*{What is the solution?}

This works:

\expandafter\@for\expandafter\tempa%
\expandafter:\expandafter=\processedlist\do{(\tempa)}

but maybe this is safer?:

\expandafter\@for\expandafter\tempa%
\expandafter:\expandafter=%
\expandafter{\processedlist}\do{(\tempa)}

However, the objective is to have \verb|\processedlist| clean of the extra
matter to use it as list in some other macro. Maybe:

\expandafter\expandafter\expandafter\def%
\expandafter\expandafter\expandafter\processedlist%
\expandafter\expandafter\expandafter{\processedlist}

\meaning\processedlist

which is more or less what we had in the first macro.

\end{document}


• Have you thought about using LaTeX3? It has powerful inbuilt list functions and, once you get used to the syntax, it results in cleaner code.
– user30471
Feb 27 '20 at 11:19
• @Andrew Frankly speaking, every time I see some code of LaTeX3, it overwhelms me. I am not very sure if learning a complex new language, if I can call it like that, for using LaTeX is worth the effort. Changing from \bfto \textbfwas not so complicated. I do not know the new command needed now, but in general they are very far from the natural language. I am very sure that my question can be solved in LaTeX3, but probably, I will not understand how it works. Feb 27 '20 at 13:08
• LaTeX3 is a little unnerving at first. I avoided using it for a while because of this but I've now started using it in my work because, once you get used to it (there's a learning curve), it's easier.
– user30471
Feb 28 '20 at 4:29

As Andrew said, expl3 has great high-level tools for many things, including list processing. But expl3 or not, the most important thing is to understand what you are doing, and post a minimal working example when you are stuck (your code doesn't have \end{document}).

So, I'll focus on your “second approach.” You initialize the result with a first item which is \@gobble, and expect it to magically go away? Why would it? It doesn't, and that's what \meaning showed you. The idea was probably to finalize the result with something like \xdef\processedlist{\processedlist} so that the \@gobble eats the comma that follows it, but this is very ugly, as \xdef (like \edef) is likely to cause nasty problems when expanding tokens from the real items (stuff like \textbf, \ref, etc.). \protected@xdef would be less problematic, but still, it would expand things in all items for no good reason.

Here, since the items are only made of non-active character tokens, this approach would work, but this will break with more delicate input, has undesirable side effects, so is bad practice IMO. My recommendation would be not to add the \@gobble item at all and simply check whether \processedlist is empty in order to decide whether to prepend a comma to the newly-added item.

That said, I'll give a way to do with the \@gobble at the end with no unwanted side effects. It is a bit longer than the following code but should be slightly faster, as the code for adding items to \processedlist has one less check to do.

Regardless of the technique used to get rid of the first comma, the algorithm is relatively inefficient, because even when the inner loop has found the current item of #1 in #2, it will continue to check all remaining items of #2. expl3 has \prg_break_point: and \prg_break: to avoid this problem and, more importantly, mapping functions that support this kind of loop breaking (e.g., \clist_map_inline:nn and \clist_map_break:).

\documentclass{article}
\usepackage[T1]{fontenc}        % for the '>' character in horizontal mode

\makeatletter
\newif\ifisinlist

\def\subtractlist#1#2{% #1:original list, #2:remove list
\def\processedlist{}%
\@for\tempa:=#1\do{%
\isinlistfalse
\@for\tempb:=#2\do{\ifx\tempa\tempb\isinlisttrue\fi}%
\ifisinlist
\else
\ifx\processedlist\empty
\expandafter\gdef\expandafter\processedlist\expandafter{\tempa}%
\else
\expandafter\processedlist
\expandafter{\expandafter,\tempa}%
\fi
\fi
}%
}
\makeatother

\setlength{\parindent}{0pt}
\setlength{\parskip}{1.5ex plus 0.5ex minus 0.2ex}

\begin{document}

\section*{Second macro for subtracting two lists}

\def\originallist{Lima,Alpha,Delta,Oscar,Tango,Whisky,Echo,Romeo,Xray}
\def\removelist{Whisky,Oscar,Romeo,Delta}

\verb|\originallist|: \originallist\par
\verb|\removelist|: \removelist\par
\verb|\subtractlist{\originallist}{\removelist}|\par
\subtractlist{\originallist}{\removelist}
\verb|\processedlist|: \processedlist\par

\meaning\processedlist

\end{document}


With the \@gobble

As promised, here is a way to do things correctly with the initial \@gobble:

\documentclass{article}
\usepackage[T1]{fontenc}        % for the '>' character in horizontal mode

\makeatletter
\newif\ifisinlist

\def\startingPoint{\@gobble}

\def\subtractlist#1#2{% #1:original list, #2:remove list
\global\let\processedlist\startingPoint
\@for\tempa:=#1\do{%
\isinlistfalse
\@for\tempb:=#2\do{\ifx\tempa\tempb\isinlisttrue\fi}%
\ifisinlist
\else
\expandafter\processedlist
\expandafter{\expandafter,\tempa}%
\fi
}%
\ifx\processedlist\startingPoint
\gdef\processedlist{}%
\else
% Expand twice so that the \@gobble eats the comma that follows it.
\xdef\processedlist{%
\unexpanded\expandafter\expandafter\expandafter{\processedlist}}%
\fi
}
\makeatother

\setlength{\parindent}{0pt}
\setlength{\parskip}{1.5ex plus 0.5ex minus 0.2ex}

\begin{document}

\section*{Second macro for subtracting two lists}

\def\originallist{Lima,Alpha,Delta,Oscar,Tango,Whisky,Echo,Romeo,Xray}
\def\removelist{Whisky,Oscar,Romeo,Delta}

\verb|\originallist|: \originallist\par
\verb|\removelist|: \removelist\par
\verb|\subtractlist{\originallist}{\removelist}|\par
\subtractlist{\originallist}{\removelist}
\verb|\processedlist|: \processedlist\par

\meaning\processedlist

\end{document}


Same output as above.

Note: if you don't understand the following piece of code:

\xdef\processedlist{%
\unexpanded\expandafter\expandafter\expandafter{\processedlist}}%


it does the same as:

\expandafter\expandafter
\expandafter\gdef
\expandafter\expandafter
\expandafter\processedlist
\expandafter\expandafter
\expandafter{\processedlist}%


P.S.: I find simple expl3 code more readable than piles of \expandafters used to do very simple things.

• I added a second method that uses the \@gobble without side effects. Feb 27 '20 at 18:20
• \xdef\processedlist{\unexpanded\expandafter{\processedlist}} is easier than that chain of \expandafter's Feb 27 '20 at 22:15
• @egreg But this expands \processedlist only once, thus the \@gobble stays there. \xdef\processedlist{\unexpanded\expandafter\expandafter\expandafter{\processedlist} does the job. I'll add it, thanks for the idea. Feb 27 '20 at 22:22
• Oh, yes, there's the nasty \@gobble to take care of. Happily, expl3 does this out of the box. Feb 27 '20 at 22:25
• @egreg That was the whole point of this last “thing”. :) I've added both ways, hopefully the OP will be happy... Feb 27 '20 at 22:27

This is essentially in expl3 out of the box. The idea is to map the original list; if an item is not in the remove list, add it to a temporary list and, at the end, reset the original list to the temporary one.

I added just some syntactic sugar to name lists, which allows to have many of them without defining new macros.

\documentclass{article}
\usepackage{xparse}

\ExplSyntaxOn
\NewDocumentCommand{\definelist}{mm}
{% #1 = list name, #2 = items
% allocate a new list or clear an existing one
\clist_clear_new:c { l_kessels_#1_clist }
% set the list
\clist_set:cn { l_kessels_#1_clist } { #2 }
}

\NewDocumentCommand{\subtractlist}{mm}
{% #1 = original, #2 = items to subtract
\kessels_list_subtract:nn { #1 } { #2 }
}

\NewDocumentCommand{\uselist}{mm}
{% #1 = list name, #2 = output separator
\clist_use:cn { l_kessels_#1_clist } { #2 }
}

\clist_new:N \l__kessels_list_temp_clist

\cs_new_protected:Nn \kessels_list_subtract:nn
{
% clear the temporary list
\clist_clear:N \l__kessels_list_temp_clist
% map the original list
\clist_map_inline:cn { l_kessels_#1_clist }
{
% if the item doesn't appear in the remove list, add it to the temp one
\clist_if_in:cnF { l_kessels_#2_clist } { ##1 }
{
\clist_put_right:Nn \l__kessels_list_temp_clist { ##1 }
}
}
% reconstitute the original list
\clist_set_eq:cN { l_kessels_#1_clist } \l__kessels_list_temp_clist
}

\ExplSyntaxOff

\begin{document}

\definelist{original}{Lima,Alpha,Delta,Oscar,Tango,Whisky,Echo,Romeo,Xray}
\definelist{remove}{Whisky,Oscar,Romeo,Delta}

\uselist{original}{, }

\subtractlist{original}{remove}

\uselist{original}{, }

\end{document}


One could think to add \copylist to save a copy of a list and \maplist to define actions on the items.

A more complete implementation. The second argument to \maplist is a template taking each item as #1.

\documentclass{article}
\usepackage{xparse}

\ExplSyntaxOn
\NewDocumentCommand{\definelist}{mm}
{% #1 = list name, #2 = items
% allocate a new list or clear an existing one
\clist_clear_new:c { l_kessels_#1_clist }
% set the list
\clist_set:cn { l_kessels_#1_clist } { #2 }
}
\NewDocumentCommand{\prependtolist}{mm}
{
\clist_put_left:cn { l_kessels_#1_clist } { #2 }
}
\NewDocumentCommand{\appendtolist}{mm}
{
\clist_put_right:cn { l_kessels_#1_clist } { #2 }
}

\NewDocumentCommand{\subtractlist}{mm}
{% #1 = original, #2 = items to subtract
\kessels_list_subtract:nn { #1 } { #2 }
}

\NewDocumentCommand{\uselist}{mm}
{% #1 = list name, #2 = output separator
\clist_use:cn { l_kessels_#1_clist } { #2 }
}

\NewDocumentCommand{\copylist}{mm}
{
\clist_clear_new:c { l_kessels_#2_clist }
\clist_set_eq:cc { l_kessels_#2_clist } { l_kessels_#1_clist }
}

\NewDocumentCommand{\maplist}{mm}
{
\cs_set:Nn \__kessels_list_map:n { #2 }
\clist_map_function:cN { l_kessels_#1_clist } \__kessels_list_map:n
}

\clist_new:N \l__kessels_list_temp_clist

\cs_new_protected:Nn \kessels_list_subtract:nn
{
% clear the temporary list
\clist_clear:N \l__kessels_list_temp_clist
% map the original list
\clist_map_inline:cn { l_kessels_#1_clist }
{
% if the item doesn't appear in the remove list, add it to the temp one
\clist_if_in:cnF { l_kessels_#2_clist } { ##1 }
{
\clist_put_right:Nn \l__kessels_list_temp_clist { ##1 }
}
}
% reconstitute the original list
\clist_set_eq:cN { l_kessels_#1_clist } \l__kessels_list_temp_clist
}

\ExplSyntaxOff

\begin{document}

\definelist{original}{Lima,Alpha,Delta,Oscar,Tango,Whisky,Echo,Romeo,Xray}
\definelist{remove}{Whisky,Oscar,Romeo,Delta}

\uselist{original}{, }

\copylist{original}{changed}

\subtractlist{changed}{remove}

\uselist{changed}{, }

\maplist{changed}{\fbox{#1\vphantom{Ay}} }

\maplist{original}{\fbox{#1\vphantom{Ay}} }

\end{document}


• Thank you, I will do my best to understand those \NewDocumentCommandconstructions. That \maplist looks very interesting. +1 Feb 28 '20 at 7:32
• I am doing my best to try to understand this, but it goes slowly. Shouldn't \appendtolist be \clist_put_right? Mar 2 '20 at 13:25
• @RaoulKessels Sure! Forgot to edit after pasting, sorry. Mar 2 '20 at 13:37

A different approach with listofitems. In essence, I use the remove-list as the list separators. Thus, they don't show up in a digested list regurgitation, because they were removed as list separators (along with the excess commas).

\documentclass{article}
\usepackage{listofitems}
\newcommand\subtractlists[2]{%
\expandafter\setsepchar\expandafter{\expandafter,\expandafter/#2}%
\foreachitem\z\in\mylist[]{%
\ifnum\zcnt>1\relax\ifnum\listlen\mylist[\zcnt]=1\relax,\fi\fi
\foreachitem\zz\in\mylist[\zcnt]{%
\ifnum\zzcnt>0\relax\zz\fi}}%
}
\begin{document}
\def\originallist{Lima,Alpha,Delta,Oscar,Tango,Whisky,Echo,Romeo,Xray}
List: \originallist

\def\removelist{Whisky||Oscar||Romeo||Delta}
Remove: \removelist

Result:
\subtractlists{\originallist}{\removelist}
\end{document}


If one insists that the remove list be comma separated (rather than ||), a little extra is needed:

\documentclass{article}
\usepackage{listofitems}
\makeatletter
\newcommand\subtractlists[2]{%
\setsepchar{,}%
\def\tmp{,/}%
\expandafter\setsepchar\expandafter{\tmp}%
\foreachitem\z\in\mylist[]{%
\ifnum\zcnt>1\relax\ifnum\listlen\mylist[\zcnt]=1\relax,\fi\fi
\foreachitem\zz\in\mylist[\zcnt]{%
\ifnum\zzcnt>0\relax\zz\fi}}%
}
\makeatother
\begin{document}
\def\originallist{Lima,Alpha,Delta,Oscar,Tango,Whisky,Echo,Romeo,Xray}
List: \originallist

\def\removelist{Whisky,Oscar,Romeo,Delta}
Remove: \removelist

Result:
\subtractlists{\originallist}{\removelist}
\end{document}


• Thank you, I was not aware of package listofitems and shall have a look at it. +1 Feb 28 '20 at 7:27

etoolbox also provides list processing capabilities. Below I've implemented something similar to your needs:

\documentclass{article}

\usepackage{etoolbox}

\newcommand{\subtractlists}[3][]{% Subtract two lists
% Check for empty optional argument (https://tex.stackexchange.com/a/53091/5764)
\if\relax\detokenize{#1}\relax
% How each item is processed
\renewcommand*{\do}[1]{\listremove{#2}{##1}}% Remove elements from original list
\else
\let#1#2% Copy original list
% How each item is processed
\renewcommand*{\do}[1]{\listremove{#1}{##1}}% Remove elements from optional list argument
\fi
\dolistloop{#3}% Process list
}
% https://tex.stackexchange.com/a/89187/5764
\newcommand{\printlist}[2][,\,]{% \printlist[<delim>]{<list cs>}
\def\itemdelim{\def\itemdelim{#1}}% Item delimiter delayed by one cycle
\renewcommand*{\do}[1]{\itemdelim##1}% How each item is processed
\dolistloop{#2}}% Process list

\setlength{\parindent}{0pt}% Just for this example

\begin{document}

% Create some lists for testing
\createlist\originallist{Lima,Alpha,Delta,Oscar,Tango,Whisky,Echo,Romeo,Xray}
\createlist\removelist{Whisky,Oscar,Romeo,Delta}

\verb|\originallist|: \printlist\originallist \par
\verb|\removelist|: \printlist\removelist \par
\verb|\subtractlists{\originallist}{\removelist}|:
\subtractlists[\processedlist]{\originallist}{\removelist}\printlist\processedlist \par
\verb|\processedlist|: \printlist\processedlist

\end{document}

• etoolbox is another of those huge packages which I have not learned yet. I understand that the \processedlist can be fed to another list processing macro if the delimiter is set to ,. Very nice how you delay the delimiter by one cycle. +1 Feb 28 '20 at 7:51