# How to count the frequency of a token in a token list

Is it possible to count the number of a token in a token list?

\documentclass{article}

\usepackage{expl3}

\ExplSyntaxOn

\tl_new:N \tobi_list_tl
\tl_set:Nn \tobi_list_tl {foo~bar~baz}

\begin{document}
'\tl_use:N \tobi_list_tl'~contains~?~spaces.
\end{document}


In a macro to typeset syllables and prosoidic symbols I need to count the number of spaces in a token list.

\cs_new_protected:Npn \tobi_metrics:nn #1 #2
{
\tl_set:Nn \l_tobi_words_tl { #2 }
% change spaces into -\q_tobi_space_marker-
\tl_replace_all:Nnn \l_tobi_words_tl { ~ } { - \q_tobi_space_marker - }
% split the (modified) second argument at -
\seq_set_split:NnV \l_tobi_syllables_seq { - } \l_tobi_words_tl
% split the first argument "at nothing"
\seq_set_split:Nnn \l_tobi_measures_seq { } { #1 }
% AT THIS POINT I WANT TO KNOW THE NUMBER OF SPACES                          <---------
% IN \l_tobi_words OR THE NUMBER OF \q_tobi_space_marker                  <------------
% IN \l_tobi_syllables_seq                                                   <---------
\int_zero:N \l_tobi_process_int
\seq_map_inline:Nn \l_tobi_syllables_seq
{
\int_incr:N \l_tobi_process_int
\tl_if_eq:nnTF { ##1 } { \q_tobi_space_marker }
{% we had a space, step back the counter and issue a space
\c_space_token
}
{% print the syllable with the metric marker
\tobi_print_syllable:n { ##1 }
}
}
}


I need the number to check whether there are enaugh symbols in the list to by combined with syllables, this is my test so far:

\int_compare:nT
{
\seq_count:N \l_@@_syllables_seq - 1 != \seq_count:N \l_@@_symbols_seq
}
{
ERROR
}


The -1 must be replaced by the number of spaces.

Maybe I should add some information on the usage. This will be part of a macro

   \metrics{_    u  u  _   _  _  | _    _  u  u   _  _   _   }
{flos ve-te-ris vi-ni | meis na-ri-bus ob-iec-tust}


The first argument is a space separated list of metric symbols (accesd by abbreviations like u or _) and the second list contains hyphenated words. I need to check if there is a symbol for every syllable but \seq_count: doesn’t work here since spaces from the second list are treated specially. The test must be

# of symbols         = # of syllables
lengt of symbol list = lengthh of syllablelist – # of spaces


\cs_new_protected:Npn \__metrix_metrics:nn #1 #2
{
\tl_set:Nn \l__metrix_words_tl { #2 }
\tl_replace_all:Nnn \l__metrix_words_tl { ~ } { - \q__metrix_space_marker - }
\seq_set_split:NnV \l__metrix_syllables_seq { - } \l__metrix_words_tl
\seq_set_split:Nnn \l__metrix_symbols_seq { ~ } { #1 }
\int_zero:N \l__metrix_process_int
\seq_map_inline:Nn \l__metrix_syllables_seq
{
\tl_if_eq:nnT { ##1 } { \q__metrix_space_marker }
{ \int_incr:N \l__metrix_process_int }
}
\int_compare:nTF
{
\seq_count:N \l__metrix_syllables_seq -
\seq_count:N \l__metrix_symbols_seq = \l__metrix_process_int
}
{
\int_zero:N \l__metrix_process_int
\seq_map_inline:Nn \l__metrix_syllables_seq
{
\int_incr:N \l__metrix_process_int
\tl_if_eq:nnTF { ##1 } { \q__metrix_space_marker }
{
\c_space_token
}
{
\str_case:nnn { ##1 }
{
{ |  } { \__metrix_l_break:  }
{ || } { \__metrix_ll_break: }
}
{
\__metrix_print_syllable:n { ##1 }
}
}
}
}
{
\__metrix_error_msg:n { Numbers~of~symbols~and~syllables~mismatch. }
}
}


Every works except when one or both of the lists end with a space, so it would be great if all spaces could be cut from the right side before start processing the lists.

• Are we allowed to assume 'well-behaved' token lists? If so, a quick \seq_set_split:NnV looks like the best plan. Jul 19, 2013 at 14:40
• @JosephWright: What do you mean with “well-behaved”? I think we can assume that the list look like flos ve-te-ris vi-ni | meis, i.e. containing letters (latin1), spaces, hyphens and pipes.
– Tobi
Jul 19, 2013 at 14:43
– Tobi
Jul 19, 2013 at 15:21

You can do it in several ways.

## First way (map the syllable sequence)

%%% Code to add for checking the syllables and the measures are the same number
\int_zero:N \l_tobi_process_int
\seq_map_inline:Nn \l_tobi_syllables_seq
{
\tl_if_eq:nnT { ##1 } { \q_tobi_space_marker }
{ \int_incr:N \l_tobi_process_int }
}
\int_compare:nF
{
\seq_count:N \l_tobi_syllables_seq -
\seq_count:N \l_tobi_measures_seq = \l_tobi_process_int
}
{ ERROR! }
%%% End of code to add


## Second way (direct count)

This requires l3regex:

\RequirePackage{l3regex}
\cs_generate_variant:Nn \regex_count:nnN { nV }

%%% Code to add for checking the syllables and the measures are the same number
\regex_count:nVN { \c{q_tobi_space_marker} } \l_tobi_words_tl \l_tobi_process_int
\int_compare:nF
{
\seq_count:N \l_tobi_syllables_seq -
\seq_count:N \l_tobi_measures_seq = \l_tobi_process_int
}
{ ERROR! }
%%% End of code to add

• @egrg: Thanks, your first way works nearly fine: It breaks when the symbol list ends with one or more spaces, e.g. \metrics{_ }{it} is it possible to ignore (gobble) the spaces from the end of a list (e.g. before the whole processing starts)? (I added the latest version of the whole macro to my question)
– Tobi
Jul 19, 2013 at 19:52
• @Tobi When you do \seq_set_split:Nnn { ~ } { #1 } leading and trailing spaces are ignored, as far as I know. Jul 19, 2013 at 20:09
• @Tobi If I add the suggested code to my previous answer, \metrics{_ }{it} doesn't raise errors. Jul 19, 2013 at 20:23
• @egreg: \seq_set_split:Nnn more or less starts by splitting into items at each marker, then removes leading and trailing spaces, so when splitting at spaces, leading and trailing spaces just give empty items, and are not discarded. One option would be \seq_set_split:Nnx \l_foo_seq { ~ } { \tl_trim_spaces:n {#1} }. Jul 19, 2013 at 23:21
• @BrunoLeFloch: Hm. I get an error that \seq_set_split:Nnx with Nnx is undefined. And I can’t find it in interface3.pdf. I made an update half an hour ago, with TL2013. I can use the Nnn version, but it doesn’t really work.
– Tobi
Jul 19, 2013 at 23:37

Perhaps the simplest solution is to use a standard TeX delimited argument approach. As we are delimiting by spaces, that is slightly trickier than normal:

\documentclass{article}

\usepackage{expl3}

\ExplSyntaxOn

\tl_new:N \tobi_list_tl
\tl_set:Nn \tobi_list_tl {foo~bar~baz}
\use:x
{
\cs_new:Npn \exp_not:N \tl_count_spaces:n ##1
{
\exp_not:N \int_eval:n
{
-1 \exp_not:N \__tl_count_spaces:w ##1 ~
\exp_not:N \q_recursion_tail \c_space_tl
\exp_not:N \q_recursion_stop
}
}
}
\cs_new:Npn \__tl_count_spaces:w #1 ~
{
\quark_if_recursion_tail_stop:n {#1}
+1
\__tl_count_spaces:w
}
\cs_generate_variant:Nn \tl_count_spaces:n { V }
\begin{document}
'\tl_use:N \tobi_list_tl'~contains~\tl_count_spaces:V \tobi_list_tl \c_space_tlspaces.
\end{document}


You could use a two-part approach, using \seq_set_split:Nnn to store the token list split on spaces, then \seq_count:N to get the value.