Sorting Comma Separated Lists defined with, or without macro

Question

I would like to be able to sort a comma separated list.

Before I resort to writing something that would require \ensureunix :-) and having that do the sort for me, I thought I'd make an attempt to doing it within TeX. So, with the help of the two questions listed in the references I have been able to get this to work, but only for the case where the list is not defined in a macro. Unfortunately for me, I only need the case where it works with macro defined lists, but would be nice to have one version which handles both.

1. With %\def\SupportMacroDefinedList{} commented out (as in the given MWE), you can see that the sorting works fine for the case when the list is not defined in a macro (right hand side of image). There are no spurious spaces and empty lists are handled just fine:

   

2. For the macro defined list (i.e., with \def\SupportMacroDefinedList{} uncommented), this is the best I can do:

   

   This is labelled as "almost" working for several reasons, and note that there are different definitions of the list macros for each case. Some of the problems are:

   • A trailing % was required after the first entry. Without that the sort order changes!!
   • It does not properly handle empty lists (which the non-macro version above had no problems with).
   • Multiple consecutive commas are properly treated as non-list members as in Zebra,,, but if there is a space in between the commas as in Zebra, , then this yields the error Undefined control sequence. <argument> \@xfor@endmarker.
   • Similarly, can't have a line with just a comma (probably same issues as the spaces in multiple consecutive commas).
   • This requires a trailing % after the last list member. Try taking that out, or adding a space before the % and making sense of the sort order.

Notes:

• The commas in the output are printed at the end of each list member only to be able to see if there is any spurious spaces.
• According to the comments in How to process a comma separated list?, \clist_map_inline:nn is supposed to be able to strip spurious spaces, but does not seem to for me. Instead it treats the macro defined list as a single member list. Also, this same linked question references \clist_map_inline:on but this yields a compile time error: Undefined control sequence for me.

References:

• How to process a comma separated list?
• How to sort an alphanumeric list

Minimum Requirements:

For this to useful for me, I really only need it to handle

1. Empty lists (only commas, spaces, and/or newlines),

2. No trailing , on the last list member:

   \newcommand*{\ListMembers}{%
     Odd,
     Zebra,
     Even,
     Alpha
   }%
   

3. with a trailing , on the last member:

   \newcommand*{\ListMembers}{%
     Odd,
     Zebra,
     Even,
     Alpha,
   }%
   

Something that did not use the expl3 syntax would be great too as then I would at least have have some hope of modifying it if it was ever needed. :-)

Code:

%\def\SupportMacroDefinedList{}%

\documentclass{article}
\usepackage{datatool}
\usepackage{xparse}
\usepackage{xstring}

\newcommand*{\SortDBName}{sortDB}%
\newcommand*{\DBKey}{label}%
\newcommand{\TitleA}{\textit{My Enumerated List:}}%
\newcommand{\TitleB}{\textit{My Empty List:}}%


%------------------------------------- Sort CSV list
% Adapted from:
%    https://tex.stackexchange.com/questions/40031/how-to-process-a-comma-separated-list
%    https://tex.stackexchange.com/questions/6988/how-to-sort-an-alphanumeric-list

\newcommand{\SortItem}[1]{%
    \IfStrEq{#1}{\empty}{%
        % Skip empty list members
    }{%
        \DTLnewrow{\SortDBName}%
        \IfEndWith{#1}{\space}{% attempt to remove any trailing space
            \DTLnewdbentry{\SortDBName}{\DBKey}{\StrGobbleRight{#1}{1}}%
        }{%
            \DTLnewdbentry{\SortDBName}{\DBKey}{#1}%
        }%
    }%
}

% How do I make \SortCommaSeparatedList work for BOTH cases 
\ifdefined\SupportMacroDefinedList%
    \ExplSyntaxOn
    \NewDocumentCommand{\SortCommaSeparatedList}{>{\SplitList {,}}m}
      { \clist_map_inline:Nn {#1} { \SortItem {##1} } }% List in macro
    % Note: nn is supposed to handle spurious spaces as per ... but doesn't
    \ExplSyntaxOff
\else
    \ExplSyntaxOn
    \NewDocumentCommand{\SortCommaSeparatedList}{>{\SplitList {,}}m}
     { \tl_map_inline:nn    {#1} { \SortItem {##1} } }% Non macro list
    \ExplSyntaxOff
\fi%



\newcommand{\SortedList}[3]{%
    % #1 = type of list 
    % #2 = title to print
    % #3 = list content (Comma separated list)
    %
    \noindent#2\par%
    %
    \DTLifdbexists{\SortDBName}%
        {\DTLcleardb{\SortDBName}}% DB exists, so just clear it
        {\DTLnewdb{\SortDBName}}%   DB does not exist, so create it
    %
    \SortCommaSeparatedList{#3}%
    \DTLsort{\DBKey}{\SortDBName}%
    \IfEq{\DTLrowcount{\SortDBName}}{0}{%
        \par%
        No items to print for ``#1" list.%
    }{%
        \begin{#1}%
            \DTLforeach*{\SortDBName}{\CurrentItem=\DBKey}{%
                \item \CurrentItem,%
        }%
        \end{#1}%
    }%
}%

\ifdefined\SupportMacroDefinedList%
    \newcommand*{\ListMembers}{% Why is % required after the first line?
        Odd,%
        Zebra,
        %,%  Can't have this
        Even,
        Alpha%
    }%
    \newcommand*{\EmptyListMembers}{%
     , ,% %%% Note: Spaces after last comma NOT-ok for macro version
    }%
\else%
    \newcommand*{\ListMembers}{%
        Odd,, 
        Zebra, ,
        ,
        Even,    
        Alpha  ,
    }%
    \newcommand*{\EmptyListMembers}{%
     , ,  % %%% Note: Spaces after last comma ok for non-macro version
    }%
\fi%


\begin{document}
\ifdefined\SupportMacroDefinedList%
    \section*{Almost works for Macro Defined List}%
\else%
    \section*{Works for non-macro Defined List}%
\fi%

\begin{minipage}[t]{0.45\linewidth}\noindent%
    \textbf{Macro defined list}\medskip\par%
    \SortedList{enumerate}{\TitleA}{\ListMembers}
    \SortedList{itemize}{\TitleB}{\EmptyListMembers}
\end{minipage}%
%
\hfill
%
\begin{minipage}[t]{0.45\linewidth}%
    \textbf{Non-macro defined list}\medskip\par\noindent%
    \SortedList{enumerate}{\TitleA}{  Odd, Zebra, , Even , Alpha  ,}
    \SortedList{itemize}{\TitleB}{ , , }
\end{minipage}%
\end{document}

Answer 1

There are many ways of tackling this problem: which you choose depends on your particular requirements.

Taking the example in the question, the reason for the apparent failure with \clist_map_inline:nn is that expl3 is very careful not to expand anything 'by accident'. Thus when the argument grabbed is a macro containing a comma-separated list, the code never sees the commas: for all you know this could be a list of one item only which just happens to be a macro itself containing a further list! There is also a difference in expl3 between functions which deal with 'stored' and 'inline' comma lists. Essentially, the idea is that a 'stored' list will already have been sanitized to remove spaces and empty items. So what you need to do is use the inline list function and expand your input once:

\NewDocumentCommand \SortCommaSeparatedList { m }
  { \exp_args:No \clist_map_function:nN {#1} \SortItem }

This will work with both forms of your input as your 'inline' list only contains unexpandable tokens. In general you cannot assume that, so I would say that \SortCommaSeparatedList should be described as accepting either a macro containing a list, or as accepting a list, but not both.

To avoid expl3, perhaps the easiest way would be to use LaTeX2e's \@for along with some space stripping code, again altering as little as possible:

\makeatletter
\newcommand*{\SortCommaSeparatedList}[1]{%
  \expandafter\@for\expandafter\@tempa\expandafter:\expandafter=#1\do{%
    \edef\@tempa{\expandafter\trim@spaces\expandafter{\@tempa}}%
    \expandafter\SortItem\expandafter{\@tempa}%
  }
}

% This is expl3's \tl_trim_spaces:n
\def\@tempa#1{%
  \newcommand{\trim@spaces}[1]{%
    \unexpanded\trim@spaces@aux@i\@mark##1\@nil\@mark#1{}\@mark
      \trim@spaces@aux@ii\trim@spaces@aux@iii#1\@nil\trim@spaces@aux@iv\@stop
  }
  \newcommand{\trim@spaces@aux@i}{}
  \long\def\trim@spaces@aux@i##1\@mark#1##2\@mark##3{%
    ##3%
    \trim@spaces@aux@i\@mark##2\@mark#1{##1}%
  }
  \newcommand{\trim@spaces@aux@ii}{}
  \long\def\trim@spaces@aux@ii##1\@mark\@mark##2{%
    \trim@spaces@aux@iii##2%
  }
  \newcommand{\trim@spaces@aux@iii}{}
  \long\def\trim@spaces@aux@iii##1#1\@nil##2{%
    ##2%
    ##1\@nil
    \trim@spaces@aux@iii
  }
  \newcommand{\trim@spaces@aux@iv}{}
  \long\def\trim@spaces@aux@iv##1\@nil##2\@stop{%
    \expandafter{\@gobble##1}%
  }
}
\@tempa{ }
\makeatother

This again expands the argument once, and this time we do a more awkward loop over every item. The space-trimming code is exactly that in expl3, but written in a more 'traditional' form. (You could write the loop more efficiently here by using that from expl3, but that seems like more effort for very little real gain.)

You could go further in a few ways. First, if you are willing to stick with expl3 then you could avoid loading xstring and do the comparisons using \tl_if_empty:nTF and so forth. There is also an experimental sorting module which would do the entire job for you! On the other hand, as expl3 requires the \pdfstrcmp primitive you could use that for the sort, although that will be slightly complicated as it works purely on character codes. Finally of course you could use LuaTeX, and do the sort in Lua (I guess that you want some reasonably general, so that is probably out).