# Splitting delimited token list argument

I wrote a little piece of code (function \getfirst:w) that splits its delimited argument at the first embedded :. The code works well if used standalone, but I am unable to place it into the .code:n of a key definition. The document command (\separatethis) that executes the key evaluation stops with error:

! Argument of \getfirst:w has an extra }.


What is going wrong?

\documentclass{article}
\usepackage{expl3}
\usepackage{xparse}

\ExplSyntaxOn
\cs_new:Npn\getfirst:w#1:#2\q_stop{#1}
\keys_define:nn{mypkg}{
myopt .code:n = {
\tl_set:Nx\myarg_tl{#1}
\tl_set:Nx\first_tl{\exp_after:wN\getfirst:w\myarg_tl\q_stop}
}
}
\NewDocumentCommand\separatethis{O{}}{
\keys_set:nn{mypkg}{#1}
\tl_use:N\first_tl
}
\ExplSyntaxOff

\begin{document}

%works
\ExplSyntaxOn
\tl_set:Nn\myarg_tl{foo:bar}
\tl_set:Nx\first_tl{\exp_after:wN\getfirst:w\myarg_tl\q_stop}
\tl_use:N\first_tl
\ExplSyntaxOff

%doesn't work
\separatethis[myopt=foo:bar]

\end{document}

-
The colon has category code 11 under \ExplSyntaxOn –  egreg Jan 23 '13 at 22:21

The problem is catcode, as : is a 'letter' inside a code block. You need to use the approach which you'd also use in classical TeX programming: use a substitute letter and use a case change to get the right thing in place.

\documentclass{article}
\usepackage{expl3}
\usepackage{xparse}

\ExplSyntaxOn
\group_begin:
\char_set_lccode:nn { \+ } { \: }
\tl_to_lowercase:n
{
\group_end:
\cs_new:Npn \getfirst:w #1 + #2 \q_stop {#1}
}
\keys_define:nn { mypkg }
{
myopt .code:n =
{ \tl_set:No \l_first_tl { \getfirst:w #1 \q_stop } }
}
\NewDocumentCommand { \separatethis } { O { } }
{
\keys_set:nn { mypkg } {#1}
\tl_use:N\l_first_tl
}
\ExplSyntaxOff

\begin{document}

\separatethis[myopt=foo:bar]

\end{document}


(We would like a cleaner approach to this, but have yet to find a suitable interface.)

-

A different approach could be with regex substitutions:

\documentclass{article}
\usepackage{expl3,l3regex}
\usepackage{xparse}

\ExplSyntaxOn
\tl_new:N \l_alexg_first_tl
\keys_define:nn { mypkg }
{
myopt .code:n =
{
\tl_set:Nn \l_alexg_first_tl { #1 }
\regex_replace_once:nnN { (.*?) \: .* } { \1 } \l_alexg_first_tl
}
}
\NewDocumentCommand { \separatethis } { O { } }
{
\keys_set:nn { mypkg } {#1}
\tl_show:N \l_alexg_first_tl
}
\ExplSyntaxOff

\begin{document}

\separatethis[myopt=foo:bar]

\end{document}


> \l_alexg_first_tl=macro:
->foo.


You basically are keeping all tokens until the first colon (category code 12) is found and the rest is thrown away. LaTeX3 regular expressions are somewhat different from the POSIX ones, but these are exactly what you'd write in Perl for doing the same thing. The "search string" does

1. (.*?)\: matches all characters (tokens, actually) until the first colon is found (the first because of the "nongreedy" quantifier *?);

2. because of the parentheses, the matched tokens up to the colon are remembered in \1

3. .* matches all tokens in the token list the search is applied to.

In particular the entire token list is a match for the "search string".

The "replacement" string just replaces all that's been matched with what has been remembered as \1.

-
Thanks for this alternative approach. I will use it as it is more easily accessible to a non-TeX programmer like me. And I will likely be able to understand it in a year from now. –  AlexG Jan 24 '13 at 7:49
What is the purpose of the ? in the regex part before \:? I would use one of .* or .+. –  AlexG Jan 24 '13 at 7:56
I see. Without the non-greedy quantifier, the colon might be swallowed by the first part. –  AlexG Jan 24 '13 at 8:06
Why the backslash before the colon? :` doesn't have a special meaning in regexes, afaik. –  AlexG Jan 24 '13 at 8:09
@egreg I think performance should get a mention here: this will be slow compared with my plan :-) –  Joseph Wright Jan 24 '13 at 8:15