# Can LaTeX3 clists contain quarks?

The introduction to l3clist in interface3.pdf states

In addition, comma lists cannot store quarks \q_mark or \q_stop.

However, this is not mentioned at any other point in the part (in contrast to a warning about "{, }, or # (assuming the usual TEX category codes apply)" in the preceding sentence, which is repeated in the descriptions of the relevant functions).

I tried to put the mentioned quarks into a clist do some basic tests and everything seems to work fine:

\documentclass{article}

\usepackage[T1]{fontenc}
\usepackage{expl3}

\begin{document}

\ExplSyntaxOn

\clist_new:N \l_test_clist
\clist_set:Nn \l_test_clist { \q_mark, \q_stop }
\clist_map_inline:Nn \l_test_clist { \tl_to_str:n { #1 } }

\ExplSyntaxOff

\end{document}

• Is there actually any problem with putting quarks into clists? Can you given an example?
• If so, does this only apply to \q_mark and \q_stop, as the quote above seems to imply?
• If so, assuming that this is because they are used in the implementation, why weren't some module specific quarks chosen instead in the implementation to prevent this problem?

I would add \q_nil and \q_recursion_tail to that statement as well :-)

• I made a few tests by browsing the source of l3clist and managed to get a few examples of \q_nil, \q_mark, \q_stop, and \q_recursion_tail breaking clist functions. Not all of them are affected because, obviously, their implementation is different, and they are affected in different ways. The list is definitely not exhaustive, as I looked for obvious cases only, for example \clist_get:NN, which breaks with a \q_stop:

\cs_new_protected:Npn \clist_get:NN #1#2
{
\if_meaning:w #1 \c_empty_clist
\tl_set:Nn #2 { \q_no_value }
\else:
\exp_after:wN \@@_get:wN #1 , \q_stop #2
\fi:
}
\cs_new_protected:Npn \@@_get:wN #1 , #2 \q_stop #3
{ \tl_set:Nn #3 {#1} }

• \q_recursion_tail also breaks some functions. \q_recursion_tail is used to indicate that the end of a recursion was reached, so after seeing that the functions thinks the list is over and tries to terminate the execution. \q_recursion_stop doesn't seem to cause harm if it is found alone (the quark the functions look for is \q_recursion_tail), but as I said I only checked the obvious problems.

• [Space for someone in the LaTeX3 team to fill in :-)]1 I'd say it's because quarks aren't supposed to be used by end users (duh) and by package writers which will only use the higher level functions. Quarks are supposed to be used if you are writing a lower level function, and if you are doing that you probably know the risks and how to avoid the problem. However I think that for safety reasons the code could be changed to use private quarks to avoid this kind of issue.

1 I guess I can't pull off this excuse anymore ;-)

Things I managed to break using quarks (if you find any other, feel free to add to the list):

• By using \q_mark:
• \clist_reverse:N (and \clist_reverse:n by extension): Using a \q_mark in the list leads to the infinite expansion of \q_mark;
• By using \q_stop:
• \clist_get:NN: Leads to a ! Missing control sequence inserted error.
• \clist_pop:NN: Leads to a ! Missing control sequence inserted error.
• \clist_item:Nn: Leads to the premature termination of the list, then the code leaves another \q_stop in the input stream to expand infinitely. However, this only happens if the second argument (the item index) is less than the position of the quark in the list.
• By using \q_recursion_tail:
• \clist_map_function:NN: Leads to the premature termination of the mapping. No error, but all items after \q_recursion_tail are ignored. Feature?
• By using \q_nil:
• \clist_(g)set:NN: Leads to the truncation of the given item. I found this out when using unravel with \unravel[prompt-input=u\q_nil] to debug some code and nothing happened, because unravel uses a clist internally, and the \q_nil, which is supposed to be part of the input, disappears, and the u alone does nothing.

Here's the test file:

\documentclass{article}
\usepackage[T1]{fontenc}
\usepackage{expl3}
\begin{document}
\ExplSyntaxOn
\clist_new:N \l_test_clist
\tl_new:N \l_test_tl
\cs_new:Npn \schtandard_function:n #1 { ( \tl_to_str:n {#1} ) }

% \q_mark breaks \clist_reverse:N
\clist_set:Nn \l_test_clist { a, \q_mark, b }
% \clist_reverse:N \l_test_clist

% \q_stop breaks \clist_get:NN, \clist_(g)pop:NN, \clist_item:Nn (if #2 is less than the position of \q_stop)
\clist_set:Nn \l_test_clist { a, \q_stop, b }
% \clist_get:NN \l_test_clist \l_test_tl
% \clist_pop:NN \l_test_clist \l_test_tl
% \clist_item:Nn \l_test_clist {1}

% \q_recursion_tail breaks \clist_map_function:NN (prematurely ends the mapping)
\clist_set:Nn \l_test_clist { a, \q_recursion_tail, b }
% \clist_map_function:NN \l_test_clist \schtandard_function:n

% \q_nil breaks \clist_set:Nn (trims the current item)
\clist_set:Nn \l_test_clist { u \q_nil v }
\clist_show:N \l_test_clist % prints:
% The comma list \l_tmpa_clist contains the items (without outer braces):
% >  {u}.

\ExplSyntaxOff
\end{document}