# LaTeX3 iteration and token list comparisons-code improvement

I am trying to make a fully expandable function that uses two arguments. The first one is the tokenlist to test against the second argument another tokenlist.

The test is a character by character comparison (I know regex is a better solution, and as a matter of fact maybe plain TeX as well), but I am trying to familiarize myself with LaTeX3.

How would you improve the code? The code with some more tests can be useful to check for numbers, vowels, valid character input etc.

\documentclass{article}
\usepackage{expl3,xcolor}
\begin{document}
\ExplSyntaxOn
\def\PASS{\par{\bfseries\textcolor{green!80!blue}{PASS\ }}}
\def\FAIL{\par{\bfseries\textcolor{red!70!black}{FAIL\ }}}
\cs_new:Npn \test_two:nn #1#2 {

% syntactic cyanide for expandafters
\cs_generate_variant:Nn \tl_if_in:NnTF {ff }
\cs_generate_variant:Nn \cs_gset:Npn {Npf}
\cs_generate_variant:Nn \tl_set_eq:NN {Nn}
\cs_generate_variant:Nn \int_step_inline:nnnn {nnfn, nnVn}
\cs_generate_variant:Nn \int_set_eq:NN {No,Nf}
\cs_generate_variant:Nn \tl_gset:Nn {No,Nf}

% set the lists
\tl_gset:Nx \temp {#2}
\tl_gset:Nx \temp_needle_tl {#1}
\tl_gset:Nn \start_tl {\temp_needle_tl }

% print some values
tail~at~ start~ \start_tl\ par

% set iteration limit
\int_set_eq:Nf  \g_tmpa_int {\tl_count:V\temp_needle_tl}
Number~of~items~to~test \int_use:N \g_tmpa_int\par

% iteration
\int_step_inline:nnnn {1} {1} {\g_tmpa_int}{

% test value
{
}
{
}

%           Swap and go
\tl_gset:Nf \oldtail_tl {\start_tl }
\tl_gset:Nn \start_tl {\tl_tail:f \oldtail_tl}
}
}
\test_two:nn {1234567890AAA}{-1234567890)(}
\test_two:nn {apple}{aeiouAEIOU)(}

\end{document}


Edit: Originally I asked for some error correction, which I fixed. I am now asking for improvement to the code.

• At first sight: Why are you using \cs_generate_variant:Nn inside a macro? Why all those _set and _gset? If you want an expandable function, you definitely have to leave those out. I'm not sure what you want to do, but you have \tl_item:nn at your disposition. – Manuel May 18 '15 at 19:39
• @Manuel The key words are "I am trying...", but you right I will move the generate_variant functions out. I also changed all _set to Nx these are \edefs, so it should be closer to fully expandable. – Yiannis Lazarides May 18 '15 at 19:41
• Mmm… I'm not sure I understand (or you don't understand). If you want a function to be fully expandable, you need that everything inside is expandable and definetly \edef is an assignment, so it's not expandable. – Manuel May 18 '15 at 19:49
• I'm afraid there are too many things that should be improved. Your naming of variables is wrong, but it's the least thing. The last lines: \tl_gset:Nn \head_tl{\tl_head:f \start_tl} is completely wrong, because you probably want \tl_gset:Nf \head_tl {\tl_head:V \start_tl}; instead of \tl_gset:Nf \oldtail_tl {\start_tl } you should say \tl_gset_eq:NN \oldtail_tl \start_tl. But it's unclear why global setting is used. – egreg May 18 '15 at 21:27
• If you really want a critique of your approach, I'll drop you a line-by-line one by e-mail. – Joseph Wright May 18 '15 at 21:54

As noted in comments, if you want an expandable function you cannot use any non-expandable functions inside it. Moreover, there are all sorts of odd/wrong argument type assignments in the question. However, the entire thing can be done a lot more compactly, at least if we can assume we don't have to worry about spaces, brace groups or the like.

\documentclass{article}
\usepackage{expl3,xcolor}
\begin{document}
\def\PASS{\par{\bfseries\textcolor{green!80!blue}{PASS\ }}}
\def\FAIL{\par{\bfseries\textcolor{red!70!black}{FAIL\ }}}
\ExplSyntaxOn
\cs_new:Npn \ylcompare #1#2
{
\__yl_compare_auxi:nN {#2} #1 \q_recursion_tail \q_recursion_stop
}
\cs_new:Npn \__yl_compare_auxi:nN #1#2
{
\quark_if_recursion_tail_stop:N #2
\__yl_compare_auxii:nN {#1} #2
\__yl_compare_auxi:nN {#1}
}
\cs_new:Npn \__yl_compare_auxii:nN #1#2
{
\__yl_compare_auxiii:NN #2 #1 \q_recursion_tail \q_recursion_stop
}
\cs_new:Npn \__yl_compare_auxiii:NN #1#2
{
\quark_if_recursion_tail_stop_do:Nn #2 { \FAIL #1 }
\str_if_eq:nnT {#1} {#2}
{
\use_i_delimit_by_q_recursion_stop:nw { \PASS #1 }
}
\__yl_compare_auxiii:NN #1
}
\ExplSyntaxOff

\ylcompare{1234567890AAA}{-1234567890)(}
\ylcompare{apple}{aeiouAEIOU)(}

\end{document}


The idea here is to set up an expandable recursion over the list to be checked, then fire off a second one over the 'allowed' list. This uses the generic recursion ideas provided by \q_recursion_tail and so on and discussed in interface3.

If there is a need to cover more complex cases, that can be done by using an implementation using the 'token list action' approach, used for example in expandable case changing.

For the case of checking for an integer value (or indeed a real number) I would hard-code the various subloops. (Indeed, I'd probably do this non-expandably unless absolutely required. For example, siunitx tests for valid input but does not do so by expansion as it carries out typesetting, while l3fp does do expansion-based checking of values as it needs to be expandable.) Assuming we are sticking to an expl3 expandable approach, one might do something like

\documentclass{article}
\usepackage{expl3}
\begin{document}
\ExplSyntaxOn
\cs_new:Npn \ylinttest #1
{ \__yl_compare_auxi:N #1 \q_recursion_tail \q_recursion_stop }
\cs_new:Npn \__yl_compare_auxi:N #1
{
\quark_if_recursion_tail_stop_do:Nn #1 { FAIL }
\bool_if:nTF
{
\str_if_eq_p:nn {#1} { + }
|| \str_if_eq_p:nn {#1} { - }
}
{ \__yl_compare_auxi:N }
{ \__yl_compare_auxii:N #1 }
}
\cs_new:Npn \__yl_compare_auxii:N #1
{
\quark_if_recursion_tail_stop_do:Nn #1 { PASS }
\__yl_compare_auxiii:N #1
\__yl_compare_auxii:N
}
\cs_new:Npn \__yl_compare_auxiii:N #1
{
\__yl_compare_auxiv:NN #1 0123456789 \q_recursion_tail \q_recursion_stop
}
\cs_new:Npn \__yl_compare_auxiv:NN #1#2
{
\quark_if_recursion_tail_stop_do:Nn #2
{ \use_i_delimit_by_q_recursion_stop:nw { FAIL } }
\str_if_eq:nnT {#1} {#2}
{ \use_none_delimit_by_q_recursion_stop:w }
\__yl_compare_auxiv:NN #1
}
\ExplSyntaxOff

\ylinttest{1234567890}
\ylinttest{---12345}
\ylinttest{---1-2345}
\ylinttest{12345A}
\ylinttest{}

\end{document}


The idea here is that a valid integer can optionally have one or more + or - at the start followed by at least one numerical value. By splitting the looping into parts there is no need to try to track the state. In a real case, rather than PASS/FAIL you'd probably have two code paths following the test and \use_i:n/\use_ii:nn to pick the appropriate one.

A test for a floating point number can be done using the same approach but needs more loops. I'd probably take a look at the version Bruno coded in l3fp for that: it works well and is fully-tested, so is a good place to start.

• Thanks for the re-write:) What is the token list action approach? For example the above can test well for vowels but for numbers one would test for the case of two minus signs (allowing two at the head, but not in the body), how would you incorporate such tests? – Yiannis Lazarides May 19 '15 at 8:20
• @YiannisLazarides I don't understand what you mean about numbers. From what I understood in the question, the test here is 'is token list A made up only of tokens present in token list B'. Can you clarify what the test actually is? – Joseph Wright May 19 '15 at 9:04
• @YiannisLazarides On the more complex looping approach to cope with spaces and brace groups, I think I need a clearer spec. What should the treatment of spaces be: do we need to allow for the possibility they can be on both lists, can we say that they are always allowed (so hard-code that part), ...? Similarly, in brace groups should the test be recursive, should they be skipped, ...? – Joseph Wright May 19 '15 at 9:15
• Thanks. Consider the use of -+1234567890 as the allowed list. If instead of printing the assertion 'FAIL' we set a false predicate (initially set at true). It can determine validy of integers (sorry about not being specific enough and said numbers). This can only fail if the integer to be tested has mutliple pluses or minuses or spaces. If we have a separate problem to find for example that a character is part of an alphabet, the functions as shown can do the job perfectly. – Yiannis Lazarides May 19 '15 at 9:29
• Add the . in the look-up list and you can check for validity of reals and so one. No need to worry about a spec, this is a problem I tried to solve while I am learning expl3. The code has matured very well, the Team did an excellent job and I intend to start using it for development. – Yiannis Lazarides May 19 '15 at 9:35