# xparse – regex in «u» argument

I am wondering whether it is possible to pass several tokens to the «u» argument in an xparse command so they would be chosen by "OR" rule? Preferably with regex that will allow much more to choose from.

Say, I have such a command that should make bold till it runs into either ,, ., (, ) or any capital letter.

Expecting to have something like so:

\documentclass{article}
\DeclareDocumentCommand{\example}{ u{/[,.A-Z]/g} }{\textbf{#1}}

\begin{document}

Some \example text goes here, also bold \example this part as well But should stop before 'B'.

\end{document}


• It should be possible to implement a parser. Such parser would have to look ahead, token by token, and then compare each token with the list you provided. However, this parsing would be quite slow. Also, u is deprecated, so it's unlikely that it's developed further. Currently the u argument works by simply defining an auxiliary macro delimited by the token you passed to u (something like \def\tmp#1<tokens>{<pass-#1-to-main-macro>}), so it uses TeX's argument scanner, which doesn't allow an argument to be delimited conditionally. – Phelype Oleinik Jun 9 at 19:53

The u argument type of xparse uses a TeX delimited macro underneath to grab the argument to the command. The definition:

\NewDocumentCommand\example{u{X}}{\textbf{#1}}


is equivalent to

\def\example#1X{\textbf{#1}}


though the former has a bunch more sanity checks, but eventually xparse uses the latter to grab the argument. And TeX doesn't allow a conditionally-delimited macro: if the delimiter is inserted in the macro definition, it has to be used.

That said, it is possible to write a parser that looks up your list of tokens and stops when you want it to.

I offer two largely different implementations. Version 2 is a regex-only, much more flexible, but also much riskier approach. Version 1 is a bit more conservative and much faster, but also less flexible.

### Version 2

This implementation defines:

\GrabUntil[*]\command{<regex>}   <text>


and

\GrabUntil[*]{<inline code>}{<regex>}   <text>


The command \GrabUntil will scan ahead and add the tokens of <text> into an internal <token list> (<tl>, for short) individually. After each token added, the command will check if <tl> matches <regex>. If it does, then <regex> is extracted from <tl>, and the command leaves \command{<tl>}<match> in the input stream. If the optional * is given, the <match> is not reinserted. If <inline code> is given instead, the argument grabbed is passed to <inline code> as #1 (this essentially defines the <inline code> as a temporary macro and then uses that as the \command).

{ and } can be used in the <regex> but they must be balanced because the code will always grab balanced token lists. Be careful when using this parser because since its stop conditions are potentially much harder to fulfill than the one from Version 1, it may eat the rest of your document without notice. The code assumes that the <match> is located at the end of the <tl> when extracting it back, and the behaviour should not be relied upon otherwise. It is recommended to end your <regex> with $ to make sure of that. This parser can be (easily, dare I say) added to xparse as an argument type, but it is not recommended to mess with xparse's internals, so it's left as an exercise for the reader ;-) Running the example with your sample text gives: \documentclass{article} \usepackage{xparse} \ExplSyntaxOn \NewDocumentCommand \GrabUntil { s m +m } { \tl_if_single_token:nTF {#2} { \cs_set_eq:NN \__antshar_cmd:n #2 } { \cs_set_protected:Npn \__antshar_cmd:n ##1 {#2} } \IfBooleanTF {#1} { \antshar_grab_delimited_regex:NNn \c_true_bool } { \antshar_grab_delimited_regex:NNn \c_false_bool } \__antshar_cmd:n {#3} } \tl_new:N \l__antshar_arg_tl \tl_new:N \l__antshar_tmp_tl \seq_new:N \l__antshar_return_seq \bool_new:N \l__antshar_remove_bool \cs_new_eq:NN \__antshar_cmd:n ? \cs_new_eq:NN \__antshar_run_cmd:n ? \regex_new:N \l__antshar_delim_regex \cs_new_protected:Npn \antshar_grab_delimited_regex:NNn #1 #2 #3 { \tl_clear:N \l__antshar_arg_tl \bool_set_eq:NN \l__antshar_remove_bool #1 \cs_set_protected:Npn \__antshar_run_cmd:n ##1 { #2 {##1} } \regex_set:Nn \l__antshar_delim_regex {#3} \__antshar_scan:w } \cs_generate_variant:Nn \__antshar_run_cmd:n { V } \cs_new_protected:Npn \__antshar_scan:w { \peek_meaning:NTF \c_group_begin_token { \__antshar_add_group:n } { \peek_meaning:NTF \c_space_token { \__antshar_add_space:w } { \__antshar_add_token:N } } } \cs_new_protected:Npn \__antshar_add_arg:n #1 { \tl_put_right:Nn \l__antshar_arg_tl {#1} \regex_match:NVTF \l__antshar_delim_regex \l__antshar_arg_tl { \regex_extract_once:NVN \l__antshar_delim_regex \l__antshar_arg_tl \l__antshar_return_seq \regex_replace_once:NnN \l__antshar_delim_regex { } \l__antshar_arg_tl \__antshar_finish: } { \__antshar_scan:w } } \cs_generate_variant:Nn \regex_extract_once:NnN { NV } \prg_generate_conditional_variant:Nnn \regex_match:Nn { NV } { TF } \exp_last_unbraced:NNo \cs_new_protected:Npn \__antshar_add_space:w \c_space_tl { \__antshar_add_arg:n { ~ } } \cs_new_protected:Npn \__antshar_add_group:n #1 { \__antshar_add_arg:n { {#1} } } \cs_new_protected:Npn \__antshar_add_token:N #1 { \__antshar_add_arg:n {#1} } \cs_new_protected:Npn \__antshar_finish: { \use:x { \__antshar_run_cmd:V \exp_not:N \l__antshar_arg_tl \bool_if:NF \l__antshar_remove_bool { \seq_use:Nn \l__antshar_return_seq { } } } } \msg_new:nnn { antshar } { braced-tokens } { Unsupported~braced~tokens~#1'~found~ in~argument~to~\iow_char:N\\GrabUntil. } \ExplSyntaxOff \begin{document} \NewDocumentCommand{\example}{}% {\GrabUntil\textbf{\ [A-Z]|[.,()]$}}

\NewDocumentCommand{\inline}{}%
{\GrabUntil{\textbf{(##1)}}{\ [A-Z]|[.,()]$}} Some \example text goes here, also bold \example this part as well But should stop before 'B'. Some \inline text goes here, also bold \inline this part as well But should stop before 'B'. % ----- \NewDocumentCommand{\exampleA}{}% {\GrabUntil*\textbf{\ [A-Z]|[.,()]$}}

\NewDocumentCommand{\inlineA}{}%
{\GrabUntil*{\textbf{(##1)}}{\ [A-Z]|[.,()]\$}}

Some \exampleA text goes here, also bold \exampleA this part as well But should stop before 'B'.

Some \inlineA text goes here, also bold \inlineA this part as well But should stop before 'B'.

\end{document}


### Version 1

The implementation below defines:

\GrabUntil[*]\command{<tokens|regex>}   <text>


and

\GrabUntil[*]{<inline code>}{<tokens|regex>}   <text>


The command \GrabUntil will scan ahead, looking at each token in <text> individually, and if any occurs in <tokens>, it is put back on the input stream and \command is called with the tokens collected so far. If the optional * is given, the command uses \regex_match:NnTF instead to match the token from <text> with the given <regex>. If <inline code> is given instead, the argument grabbed is passed to <inline code> as #1 (this essentially defines the <inline code> as a temporary macro and then uses that as the \command).

One limitation is that { and } cannot be used as delimiters (though \{ and \} can), and groups of tokens are passed in full, so with \GrabUntil\test{abc} x{abc}za, you get x{abc}z as argument to \test. The other limitation is that this only allows single tokens as delimiters, since the scanner looks at tokens individually, so the regex is inserted in a character class ([...]) so that only single tokens match.

Running the example with your sample text gives:

\documentclass{article}
\usepackage{xparse}
\ExplSyntaxOn
\NewDocumentCommand \GrabUntil { s m +m }
{
\tl_if_single_token:nTF {#2}
{ \cs_set_eq:NN \__antshar_cmd:n #2 }
{ \cs_set_protected:Npn \__antshar_cmd:n ##1 {#2} }
\IfBooleanTF {#1}
{ \antshar_grab_delimited_regex:Nn \__antshar_cmd:n {#3} }
{ \antshar_grab_delimited:Nn \__antshar_cmd:n {#3} }
}
\tl_new:N \l__antshar_arg_tl
\tl_new:N \l__antshar_delim_tl
\regex_new:N \l__antshar_delim_regex
\cs_new_eq:NN \__antshar_cmd:n ?
\cs_new_eq:NN \__antshar_run_cmd:n ?
\cs_new_eq:NN \__antshar_examine_next:n ?
\cs_new_protected:Npn \antshar_grab_delimited:Nn #1 #2
{
\cs_set_protected:Npn \__antshar_run_cmd:n ##1 { #1 {##1} }
\tl_clear:N \l__antshar_arg_tl
\tl_clear:N \l__antshar_delim_tl
\tl_set:Nn \l__antshar_delim_tl {#2}
\cs_set_eq:NN \__antshar_examine_next:n \__antshar_examine_tl:n
\__antshar_scan:w
}
\cs_new_protected:Npn \antshar_grab_delimited_regex:Nn #1 #2
{
\cs_set_protected:Npn \__antshar_run_cmd:n ##1 { #1 {##1} }
\tl_clear:N \l__antshar_arg_tl
\regex_set:Nn \l__antshar_delim_regex { [#2] }
\cs_set_eq:NN \__antshar_examine_next:n \__antshar_examine_regex:n
\__antshar_scan:w
}
\cs_generate_variant:Nn \__antshar_run_cmd:n { V }
\cs_new_protected:Npn \__antshar_scan:w
{
\peek_meaning:NTF \c_group_begin_token
{
\peek_meaning:NTF \c_space_token
}
}
\cs_new_protected:Npn \__antshar_examine_tl:n #1
{
\tl_if_in:NnTF \l__antshar_delim_tl {#1}
{ \__antshar_finish: #1 }
{ \__antshar_continue:n {#1} }
}
\cs_new_protected:Npn \__antshar_examine_regex:n #1
{
\regex_match:NnTF \l__antshar_delim_regex {#1}
{ \__antshar_finish: #1 }
{ \__antshar_continue:n {#1} }
}
\cs_new_protected:Npn \__antshar_continue:n #1
{
\tl_put_right:Nn \l__antshar_arg_tl {#1}
\__antshar_scan:w
}
\exp_last_unbraced:NNo
{ \__antshar_examine_next:n { ~ } }
{ \__antshar_examine_next:n { {#1} } }
{ \__antshar_examine_next:n {#1} }
\cs_new_protected:Npn \__antshar_finish:
{ \__antshar_run_cmd:V \l__antshar_arg_tl }
\msg_new:nnn { antshar } { braced-tokens }
{
Unsupported~braced~tokens~#1'~found~
in~argument~to~\iow_char:N\\GrabUntil.
}
\ExplSyntaxOff

\begin{document}

\NewDocumentCommand{\example}{}%
{\GrabUntil\textbf{,.()ABCDEFGHIJKLMNOPQRZTUVWXYZ}}

\NewDocumentCommand{\exampleregex}{}%
{\GrabUntil*\textbf{,\.A-Z}}

\NewDocumentCommand{\exampleinline}{}%
{\GrabUntil*{\textbf{(##1)}}{,\.A-Z}}

Some \example text goes here, also bold \example this part as well But should stop before 'B'.

Some \exampleregex text goes here, also bold \exampleregex this part as well But should stop before 'B'.

Some \exampleinline text goes here, also bold \exampleinline this part as well But should stop before 'B'.

\end{document}

• Amazing! Thank you for the answer. I would only like to ask whether it is possible to make a little bit more flexible, so that the matched text will be like #1? In order to add not only boldness, but also additional character in front and at the end of command input, for example, or any other embelishments to the matches text. – antshar Jun 9 at 23:27
• @antshar Done: I edited the answer. I extended the syntax so that both versions work. If the \command argument consists of multiple tokens, then it is treated as <inline code> instead, but it defines a temporary command \def\tmp#1{<code with #1>} and then calls the normal syntax with \tmp as the command. Though note that if you use it in a definition (like inside \NewDocumentCommand) you have to double the hashes (##1 rather than #1). – Phelype Oleinik Jun 10 at 2:16
• Thanks! It is amlost perfect. Sorry for another request, but it woul be so helpful: now it looks like my regex automatically is put around [ ], so it can look only for a single character. It would be nice to be able to control by myself. That is, I can match groups, use quantifiers etc., for example find a space before a capital letter or don't before special characters: ( [A-Z])|[.,] – antshar Jun 10 at 8:23
• @antshar The choice of adding [...] by default was deliberate. That version could only cope with single tokens as delimiters, because it looks at each token individually and doesn't care about what's already been added to the argument (this makes that method a lot faster). I added a version 2, which allows multiple tokens to be matched (but it's quite a bit slower and much easier to mess up, so use carefully). – Phelype Oleinik Jun 10 at 17:13
• Thank you so much for another adit for my demand.I just can't leave it without "answer" check mark. I would only additionaly ask why doesn't it work it my regex contain \ ? For example, if I want to bold everything till \\  it doesn't seem to compile. And \par matches a little bit further than need. – antshar Jun 10 at 18:46