Building and using a list

Question

I made a macro for subtracting two lists which does what was intended. Then, I found a set-up, here on TeX.SE, with the macro \g@addto@macro and \@gobble which is much simpler and gives, in principle, the same output.

When trying to use this second form, some problems did arise. After some searching, it became clear that the \@gobble had not been expanded yet, adding an additional element to the list and causing trouble.

What is the best solution for this problem?

I feel that it is preferable to process the resulting \processedlist so that it can be used in any macro which expects a list. Then, we get again something similar as in the first macro, a lot of \expandafters, although this time they will be used only once and, thus, it should be faster.

\documentclass{article}

\setlength{\parindent}{0pt}
\setlength{\parskip}{1.5ex plus 0.5ex minus 0.2ex}
\begin{document}
\makeatletter

\section*{First macro for subtracting two lists}

\newif\iffirstterm
\newif\ifisinlist

\def\subtractlists#1#2{% original list, remove list
\def\processedlist{}%
\firsttermtrue%
\@for\tempa:=#1\do{%
  \isinlistfalse%
  \@for\tempb:=#2\do{\ifx\tempa\tempb\isinlisttrue\fi}%
  \ifisinlist%
    \else%
      \iffirstterm%
          \expandafter\def\expandafter\processedlist\expandafter{\tempa}%  
          \firsttermfalse%
        \else%
          \expandafter\expandafter\expandafter\def%
          \expandafter\expandafter\expandafter\processedlist%
          \expandafter\expandafter\expandafter{%
          \expandafter\processedlist\expandafter,\tempa}%  
      \fi%
  \fi}%
}

%%% some lists for testing:
\def\originallist{Lima,Alpha,Delta,Oscar,Tango,Whisky,Echo,Romeo,Xray}
\def\removelist{Whisky,Oscar,Romeo,Delta}

%The list can be entered as comma separated items or as a macro:

\verb|\originallist|: \originallist\par
\verb|\removelist|: \removelist\par
\verb|\subtractlists{\originallist}{\removelist}|\par
\subtractlists{\originallist}{\removelist}
\verb|\processedlist|: \processedlist\par

%\@for\tempa:=\processedlist\do{(\tempa)}

\section*{Second macro for subtracting two lists}

\def\subtractlist#1#2{% #1:original list,#2:remove list
  \def\processedlist{\@gobble}%
  \@for\tempa:=#1\do{%
    \isinlistfalse%
    \@for\tempb:=#2\do{\ifx\tempa\tempb\isinlisttrue\fi}%
      \ifisinlist%
         \else%
           \expandafter\g@addto@macro%
           \expandafter\processedlist%
           \expandafter{\expandafter,\tempa}%      
       \fi}%
}


\verb|\originallist|: \originallist\par
\verb|\removelist|: \removelist\par
\verb|\subtractlist{\originallist}{\removelist}|\par
\subtractlist{\originallist}{\removelist}
\verb|\processedlist|: \processedlist\par

\section*{Problems with second approach}

There is some extra matter:

\@for\tempa:=\processedlist\do{(\tempa)}

the first comma has not been gobbled yet:

\verb|\meaning\processedlist|: \meaning\processedlist


\section*{What is the solution?}

This works:

\expandafter\@for\expandafter\tempa%
\expandafter:\expandafter=\processedlist\do{(\tempa)}

but maybe this is safer?:

\expandafter\@for\expandafter\tempa%
\expandafter:\expandafter=%
\expandafter{\processedlist}\do{(\tempa)}

However, the objective is to have \verb|\processedlist| clean of the extra
matter to use it as list in some other macro. Maybe:

\expandafter\expandafter\expandafter\def%
\expandafter\expandafter\expandafter\processedlist%
\expandafter\expandafter\expandafter{\processedlist}

\meaning\processedlist

which is more or less what we had in the first macro.

\end{document}

Answer 1

As Andrew said, expl3 has great high-level tools for many things, including list processing. But expl3 or not, the most important thing is to understand what you are doing, and post a minimal working example when you are stuck (your code doesn't have \end{document}).

So, I'll focus on your “second approach.” You initialize the result with a first item which is \@gobble, and expect it to magically go away? Why would it? It doesn't, and that's what \meaning showed you. The idea was probably to finalize the result with something like \xdef\processedlist{\processedlist} so that the \@gobble eats the comma that follows it, but this is very ugly, as \xdef (like \edef) is likely to cause nasty problems when expanding tokens from the real items (stuff like \textbf, \ref, etc.). \protected@xdef would be less problematic, but still, it would expand things in all items for no good reason.

Here, since the items are only made of non-active character tokens, this approach would work, but this will break with more delicate input, has undesirable side effects, so is bad practice IMO. My recommendation would be not to add the \@gobble item at all and simply check whether \processedlist is empty in order to decide whether to prepend a comma to the newly-added item.

That said, I'll give a way to do with the \@gobble at the end with no unwanted side effects. It is a bit longer than the following code but should be slightly faster, as the code for adding items to \processedlist has one less check to do.

Regardless of the technique used to get rid of the first comma, the algorithm is relatively inefficient, because even when the inner loop has found the current item of #1 in #2, it will continue to check all remaining items of #2. expl3 has \prg_break_point: and \prg_break: to avoid this problem and, more importantly, mapping functions that support this kind of loop breaking (e.g., \clist_map_inline:nn and \clist_map_break:).

\documentclass{article}
\usepackage[T1]{fontenc}        % for the '>' character in horizontal mode

\makeatletter
\newif\ifisinlist

\def\subtractlist#1#2{% #1:original list, #2:remove list
  \def\processedlist{}%
  \@for\tempa:=#1\do{%
    \isinlistfalse
    \@for\tempb:=#2\do{\ifx\tempa\tempb\isinlisttrue\fi}%
    \ifisinlist
    \else
      \ifx\processedlist\empty
        \expandafter\gdef\expandafter\processedlist\expandafter{\tempa}%
      \else
         \expandafter\g@addto@macro
         \expandafter\processedlist
         \expandafter{\expandafter,\tempa}%
      \fi
    \fi
  }%
}
\makeatother

\setlength{\parindent}{0pt}
\setlength{\parskip}{1.5ex plus 0.5ex minus 0.2ex}

\begin{document}

\section*{Second macro for subtracting two lists}

\def\originallist{Lima,Alpha,Delta,Oscar,Tango,Whisky,Echo,Romeo,Xray}
\def\removelist{Whisky,Oscar,Romeo,Delta}

\verb|\originallist|: \originallist\par
\verb|\removelist|: \removelist\par
\verb|\subtractlist{\originallist}{\removelist}|\par
\subtractlist{\originallist}{\removelist}
\verb|\processedlist|: \processedlist\par

\meaning\processedlist

\end{document}

With the \@gobble

As promised, here is a way to do things correctly with the initial \@gobble:

\documentclass{article}
\usepackage[T1]{fontenc}        % for the '>' character in horizontal mode

\makeatletter
\newif\ifisinlist

\def\startingPoint{\@gobble}

\def\subtractlist#1#2{% #1:original list, #2:remove list
  \global\let\processedlist\startingPoint
  \@for\tempa:=#1\do{%
    \isinlistfalse
    \@for\tempb:=#2\do{\ifx\tempa\tempb\isinlisttrue\fi}%
    \ifisinlist
    \else
      \expandafter\g@addto@macro
      \expandafter\processedlist
      \expandafter{\expandafter,\tempa}%
    \fi
  }%
  \ifx\processedlist\startingPoint
    \gdef\processedlist{}%
  \else
    % Expand twice so that the \@gobble eats the comma that follows it.
    \xdef\processedlist{%
      \unexpanded\expandafter\expandafter\expandafter{\processedlist}}%
  \fi
}
\makeatother

\setlength{\parindent}{0pt}
\setlength{\parskip}{1.5ex plus 0.5ex minus 0.2ex}

\begin{document}

\section*{Second macro for subtracting two lists}

\def\originallist{Lima,Alpha,Delta,Oscar,Tango,Whisky,Echo,Romeo,Xray}
\def\removelist{Whisky,Oscar,Romeo,Delta}

\verb|\originallist|: \originallist\par
\verb|\removelist|: \removelist\par
\verb|\subtractlist{\originallist}{\removelist}|\par
\subtractlist{\originallist}{\removelist}
\verb|\processedlist|: \processedlist\par

\meaning\processedlist

\end{document}

Same output as above.

Note: if you don't understand the following piece of code:

\xdef\processedlist{%
  \unexpanded\expandafter\expandafter\expandafter{\processedlist}}%

it does the same as:

\expandafter\expandafter
\expandafter\gdef
\expandafter\expandafter
\expandafter\processedlist
\expandafter\expandafter
\expandafter{\processedlist}%

P.S.: I find simple expl3 code more readable than piles of \expandafters used to do very simple things.

Answer 2

This is essentially in expl3 out of the box. The idea is to map the original list; if an item is not in the remove list, add it to a temporary list and, at the end, reset the original list to the temporary one.

I added just some syntactic sugar to name lists, which allows to have many of them without defining new macros.

\documentclass{article}
\usepackage{xparse}

\ExplSyntaxOn
\NewDocumentCommand{\definelist}{mm}
 {% #1 = list name, #2 = items
  % allocate a new list or clear an existing one
  \clist_clear_new:c { l_kessels_#1_clist }
  % set the list
  \clist_set:cn { l_kessels_#1_clist } { #2 }
 }

\NewDocumentCommand{\subtractlist}{mm}
 {% #1 = original, #2 = items to subtract
  \kessels_list_subtract:nn { #1 } { #2 }
 }

\NewDocumentCommand{\uselist}{mm}
 {% #1 = list name, #2 = output separator
  \clist_use:cn { l_kessels_#1_clist } { #2 }
 }

\clist_new:N \l__kessels_list_temp_clist

\cs_new_protected:Nn \kessels_list_subtract:nn
 {
  % clear the temporary list
  \clist_clear:N \l__kessels_list_temp_clist
  % map the original list
  \clist_map_inline:cn { l_kessels_#1_clist }
   {
    % if the item doesn't appear in the remove list, add it to the temp one
    \clist_if_in:cnF { l_kessels_#2_clist } { ##1 }
     {
      \clist_put_right:Nn \l__kessels_list_temp_clist { ##1 }
     }
   }
  % reconstitute the original list
  \clist_set_eq:cN { l_kessels_#1_clist } \l__kessels_list_temp_clist
 }

\ExplSyntaxOff

\begin{document}

\definelist{original}{Lima,Alpha,Delta,Oscar,Tango,Whisky,Echo,Romeo,Xray}
\definelist{remove}{Whisky,Oscar,Romeo,Delta}

\uselist{original}{, }

\subtractlist{original}{remove}

\uselist{original}{, }

\end{document}

One could think to add \copylist to save a copy of a list and \maplist to define actions on the items.

A more complete implementation. The second argument to \maplist is a template taking each item as #1.

\documentclass{article}
\usepackage{xparse}

\ExplSyntaxOn
\NewDocumentCommand{\definelist}{mm}
 {% #1 = list name, #2 = items
  % allocate a new list or clear an existing one
  \clist_clear_new:c { l_kessels_#1_clist }
  % set the list
  \clist_set:cn { l_kessels_#1_clist } { #2 }
 }
\NewDocumentCommand{\prependtolist}{mm}
 {
  \clist_put_left:cn { l_kessels_#1_clist } { #2 }
 }
\NewDocumentCommand{\appendtolist}{mm}
 {
  \clist_put_right:cn { l_kessels_#1_clist } { #2 }
 }

\NewDocumentCommand{\subtractlist}{mm}
 {% #1 = original, #2 = items to subtract
  \kessels_list_subtract:nn { #1 } { #2 }
 }

\NewDocumentCommand{\uselist}{mm}
 {% #1 = list name, #2 = output separator
  \clist_use:cn { l_kessels_#1_clist } { #2 }
 }

\NewDocumentCommand{\copylist}{mm}
 {
  \clist_clear_new:c { l_kessels_#2_clist }
  \clist_set_eq:cc { l_kessels_#2_clist } { l_kessels_#1_clist }
 }

\NewDocumentCommand{\maplist}{mm}
 {
  \cs_set:Nn \__kessels_list_map:n { #2 }
  \clist_map_function:cN { l_kessels_#1_clist } \__kessels_list_map:n
 }

\clist_new:N \l__kessels_list_temp_clist

\cs_new_protected:Nn \kessels_list_subtract:nn
 {
  % clear the temporary list
  \clist_clear:N \l__kessels_list_temp_clist
  % map the original list
  \clist_map_inline:cn { l_kessels_#1_clist }
   {
    % if the item doesn't appear in the remove list, add it to the temp one
    \clist_if_in:cnF { l_kessels_#2_clist } { ##1 }
     {
      \clist_put_right:Nn \l__kessels_list_temp_clist { ##1 }
     }
   }
  % reconstitute the original list
  \clist_set_eq:cN { l_kessels_#1_clist } \l__kessels_list_temp_clist
 }

\ExplSyntaxOff

\begin{document}

\definelist{original}{Lima,Alpha,Delta,Oscar,Tango,Whisky,Echo,Romeo,Xray}
\definelist{remove}{Whisky,Oscar,Romeo,Delta}

\uselist{original}{, }

\copylist{original}{changed}

\subtractlist{changed}{remove}

\uselist{changed}{, }

\maplist{changed}{\fbox{#1\vphantom{Ay}} }

\maplist{original}{\fbox{#1\vphantom{Ay}} }

\end{document}

Answer 3

A different approach with listofitems. In essence, I use the remove-list as the list separators. Thus, they don't show up in a digested list regurgitation, because they were removed as list separators (along with the excess commas).

\documentclass{article}
\usepackage{listofitems}
\newcommand\subtractlists[2]{%
  \expandafter\setsepchar\expandafter{\expandafter,\expandafter/#2}%
  \readlist*\mylist{#1}%
  \foreachitem\z\in\mylist[]{%
    \ifnum\zcnt>1\relax\ifnum\listlen\mylist[\zcnt]=1\relax,\fi\fi
    \foreachitem\zz\in\mylist[\zcnt]{%
      \ifnum\zzcnt>0\relax\zz\fi}}%
}
\begin{document}
\def\originallist{Lima,Alpha,Delta,Oscar,Tango,Whisky,Echo,Romeo,Xray}
List: \originallist

\def\removelist{Whisky||Oscar||Romeo||Delta}
Remove: \removelist

Result:
\subtractlists{\originallist}{\removelist}
\end{document}

If one insists that the remove list be comma separated (rather than ||), a little extra is needed:

\documentclass{article}
\usepackage{listofitems}
\makeatletter
\newcommand\subtractlists[2]{%
  \setsepchar{,}%
  \readlist\remlist{#2}%
  \def\tmp{,/}%
  \foreachitem\z\in\remlist[]{\ifnum\zcnt=1\else\g@addto@macro\tmp{||}\fi
    \expandafter\g@addto@macro\expandafter\tmp\expandafter{\z}}%
  \expandafter\setsepchar\expandafter{\tmp}%
  \readlist*\mylist{#1}%
  \foreachitem\z\in\mylist[]{%
    \ifnum\zcnt>1\relax\ifnum\listlen\mylist[\zcnt]=1\relax,\fi\fi
    \foreachitem\zz\in\mylist[\zcnt]{%
      \ifnum\zzcnt>0\relax\zz\fi}}%
}
\makeatother
\begin{document}
\def\originallist{Lima,Alpha,Delta,Oscar,Tango,Whisky,Echo,Romeo,Xray}
List: \originallist

\def\removelist{Whisky,Oscar,Romeo,Delta}
Remove: \removelist

Result:
\subtractlists{\originallist}{\removelist}
\end{document}

Answer 4

etoolbox also provides list processing capabilities. Below I've implemented something similar to your needs:

\documentclass{article}

\usepackage{etoolbox}

\newcommand{\createlist}[1]{\forcsvlist{\listadd{#1}}}% Create a list
\newcommand{\subtractlists}[3][]{% Subtract two lists
  % Check for empty optional argument (https://tex.stackexchange.com/a/53091/5764)
  \if\relax\detokenize{#1}\relax
    % How each item is processed
    \renewcommand*{\do}[1]{\listremove{#2}{##1}}% Remove elements from original list
  \else
    \let#1#2% Copy original list
    % How each item is processed
    \renewcommand*{\do}[1]{\listremove{#1}{##1}}% Remove elements from optional list argument
  \fi
  \dolistloop{#3}% Process list
}
% https://tex.stackexchange.com/a/89187/5764
\newcommand{\printlist}[2][,\,]{% \printlist[<delim>]{<list cs>}
  \def\itemdelim{\def\itemdelim{#1}}% Item delimiter delayed by one cycle
  \renewcommand*{\do}[1]{\itemdelim##1}% How each item is processed
  \dolistloop{#2}}% Process list

\setlength{\parindent}{0pt}% Just for this example

\begin{document}

% Create some lists for testing
\createlist\originallist{Lima,Alpha,Delta,Oscar,Tango,Whisky,Echo,Romeo,Xray}
\createlist\removelist{Whisky,Oscar,Romeo,Delta}

\verb|\originallist|: \printlist\originallist \par
\verb|\removelist|: \printlist\removelist \par
\verb|\subtractlists{\originallist}{\removelist}|:
  \subtractlists[\processedlist]{\originallist}{\removelist}\printlist\processedlist \par
\verb|\processedlist|: \printlist\processedlist

\end{document}