# Using Expl3 token-list variables where token lists are called for

The code below illustrates the trouble I’m having. It seems I cannot directly use a token-list variable where a token list is called for, but why, and how can I get the result I want?

\documentclass{article}
\usepackage{xparse}
\ExplSyntaxOn

\bool_new:N \found_bool

\cs_new_protected:Npn \find_token_in_tl:Nn #1#2
{
\bool_set_false:N \found_bool
\tl_map_inline:nn { #2 }
{
\token_if_eq_charcode:NNT ##1 #1
{
\bool_set_true:N \found_bool
\tl_map_break:
}
}
}

\NewDocumentCommand { \FindTokInList } { m m }
{
\find_token_in_tl:Nn #1 {#2}
\bool_if:NTF \found_bool { ! } { ? }
}

\tl_const:Nn \punct_tl {,.;:}

\NewDocumentCommand { \FindTokInPunctList } { m }
{
\find_token_in_tl:Nn #1 \punct_tl
\bool_if:NTF \found_bool { ! } { ? }
}
\ExplSyntaxOff

\begin{document}
dot: \FindTokInList{.}{,.;:} % finds token & prints “!”

x: \FindTokInList{x}{,.;:} % doesn’t find token & prints “?”

dot: \FindTokInPunctList{.} % doesn’t find token & prints “?”, but why?

x: \FindTokInPunctList{x} % doesn’t find token & prints “?”
\end{document}

-
Generating a variant \cs_generate_variant:Nn \find_token_in_tl:Nn {NV} and using that in \FindTokInPunctList fixes the problem but I'm not completely sure why. –  Scott H. Jul 20 '12 at 17:54
Why not using \tl_if_in:NnTF to begin with? –  egreg Jul 20 '12 at 17:55
@egreg One reason could be because the charcode test is looser than \tl_if_in:NnTF. For instance, one wants to put a given space both before an active : or an other :. –  Bruno Le Floch Jul 20 '12 at 23:41
@BrunoLeFloch, that’s something I hadn’t even considered, though it does explain the other results I mentioned in comments below. I may be out of my depth here… ☺ –  J. C. Salomon Jul 22 '12 at 1:49

There are token lists and token list variables, which are n and N type, respectively. The LaTeX3 programming layer is careful not to expand anything it should not, so when you do

\tl_map_inline:nn {#2} { <code> }


with #2 = \l_JS_punct_tl you passing the single token \l_JS_punct_tl to the <code>, not the content of the variable. For that, you want

\tl_map_inline:Nn #2 { <code> }


You could also pass the value using

\cs_generate_variant:Nn \tl_map_inline:nn { V }
\tl_map_inline:Vn #2 { <code> }


but it's not necessary.

As an aside, LaTeX3 code variables should always be named

\<scope>_<module>_<description>_<type>


so I've used \l_JS_punct_tl in place of \punct_tl in the question.

-
Shouldn't it be \c_JS_punct_tl? Also \found_bool should be \l_JS_found_bool –  egreg Jul 20 '12 at 18:09
@egreg I'd not spotted that it's a constant, but this seems wrong to me anyway as I'd expect a real case to make this user-definable. I'd noticed the bool, but made my comment general to cover this case too. –  Joseph Wright Jul 20 '12 at 18:11

There are already \tl_if_in:nnTF and \tl_if_in:NnTF:

\documentclass{article}
\usepackage{xparse}
\ExplSyntaxOn

\NewDocumentCommand { \FindTokInList } { m m }
{
\tl_if_in:nnTF {#2} {#1} { ! } { ? }
}

\tl_const:Nn \c_JS_punct_tl {,.;:}

\NewDocumentCommand { \FindTokInPunctList } { m }
{
\tl_if_in:NnTF \c_JS_punct_tl { #1 } { ! } { ? }
}
\ExplSyntaxOff

\begin{document}
dot: \FindTokInList{.}{,.;:} % finds token & prints !

x: \FindTokInList{x}{,.;:} % doesn't find token & prints ?

dot: \FindTokInPunctList{.} % finds token & prints !

x: \FindTokInPunctList{x} % doesn't find token & prints ?
\end{document}

-
That was helpful! It’s too easy for an expl3 newbie to miss things like that, and I’m glad of the help. –  J. C. Salomon Jul 20 '12 at 18:52
@J.C.Salomon It's difficult to remember all the functions; but a couple of guidelines are: basic tasks are covered and the naming conventions are good. So looking for tl_if_in conduced to the right functions. –  egreg Jul 20 '12 at 18:59
My actual use-case looks more like \token_new:Nn\period{.} \tl_if_in:nnTF{,.;}{\period}{!}{?}, using a token variable (actually \l_peek_token). –  J. C. Salomon Jul 20 '12 at 19:13
@J.C.Salomon The use case you describe in that last comment is not covered by \tl_if_in since that \period token is an implicit token, which does not "look" like a period character (but has the same meaning, which means that it behaves the same when typeset). –  Bruno Le Floch Jul 20 '12 at 23:43