15

I apologize that this question may seem very much like several other recent ones I have asked (Parsing leading hardspaces, Parsing a \$ as part of an improved \getargs command). They are all related to using a recursive TeXnique to parse strings. David Carlisle has already helped a great deal in the prior queries in pointing me to \protected@edef to handle special (e.g., \$) characters and in using \let to redefine hardspaces to something innocuous (either nulls or soft spaces).

But now I find the algorithm chokes on diacritical marks like umlauts (\"u). I guessing that these are not "characters" but expandable "macros" which might explain why an algorithm that searches character by character may choke. So I'm looking for a clever workaround, something like a temporary \let or \catcode redefinition that would allow the algorithm to digest these types of "characters".

So the code is below. I commented out the one line (2nd from end) which will cause the algorithm to choke. What the algorithm does is looks at the argument and parses it into single words. It places the first word in \argi, the second in \argii, etc. and the total number of words in \narg. And its much faster than its predecessor in the stringstrings package. It works great with alphanumeric inputs, special characters, hard and soft spaces, but not (yet) with diacritical marks.

\documentclass{article}
\makeatletter
\usepackage{ifnextok}
\usepackage{ifthen}
\def\string@end{$}
\def\converttilde{T}
\newcounter{arg@index}
\let\SaveHardspace~
\def\getargsF#1{%
  \if T\converttilde\def~{ }\fi%
  \protected@edef\the@string{#1}%
  \def\argi{}\setcounter{arg@index}{1}%
  \expandafter\parse@Block\the@string\string@end%
  \let~\SaveHardspace%
}
%ifthenelse needed when leading spaces end up as \argi
\def\parse@Block{\IfNextToken\string@end%
  {\edef\narg{\arabic{arg@index}}\@gobble}%
  {\IfNextToken\@sptoken{\addtocounter{arg@index}{1}%
   \ifthenelse{\equal{\argi}{}}{\addtocounter{arg@index}{-1}}{}%
   \expandafter\def\csname arg\roman{arg@index}\endcsname{}%
   \add@to{\parse@Block}}%
  {\add@to{\parse@Block}}}}
\def\add@to#1#2{\if\@sptoken#2\else\expandafter\protected@edef%
                 \csname arg\roman{arg@index}\endcsname%
                {\csname arg\roman{arg@index}\endcsname#2}\fi#1}
\makeatother
\begin{document}
\getargsF{mein hut} \argi~\argii $<$NO PROBLEM\\
BUT PUT UMLAUT ON THE u AND IT BREAKS
%\getargsF{mein h\"ut} \argi~\argii
\end{document}
1
  • 1
    The answers by Scott H, unbonpetit, and David Carlisle are all excellent responses. I don't know xparse, but it looks amazingly powerful. The ability of unbonpetit's approach to dissect all the macros is extremely noteworthy, and I have to still unravel the magic performed by Mr. Carlisle (though for my larger application I think I will have to step through token by token). It will take me a while to digest them to see which works best with my intended direction. Thanks all. Mar 9, 2013 at 14:02

3 Answers 3

6

Once you have expanded everything in the original input with \protected@edef you really don't need to step through token by token which makes it hard to handle any kind of brace group or command taking arguments. This just scans with a delimited argument looking for spaces which is much simpler.

enter image description here

 \documentclass{article}
\makeatletter

\def\string@end{$\SaveHardspace}
\def\converttilde{T}
\newcounter{arg@index}
\let\SaveHardspace~%%%
\def\getargsF#1{%
  \if T\converttilde\def~{ }\fi
  \protected@edef\the@string{#1}%
  \setcounter{arg@index}{0}%
  \lowercase{\expandafter\parse@Block\the@string} \string@end
  \let~\SaveHardspace
}

\def\parse@Block#1 {%
  \stepcounter{arg@index}%
  \@namedef{arg\roman{arg@index}}{#1}%
  \futurelet\tmp\parse@Block@}

\def\parse@Block@{%
\ifx\tmp\string@end\edef\narg{\thearg@index}\expandafter\@gobble
\else\expandafter\parse@Block\fi}

\makeatother
\begin{document}
\getargsF{mein hut} \argi~\argii~[\narg] $<$ NO PROBLEM


BUT PUT UMLAUT ON THE u AND IT BREAKS

\getargsF{mein h\"ut see}  \argi~\argii~[\narg]
\end{document}
3
  • The use of \futurelet looks like a really useful command for these sorts of problems. Also, I see \@namedef is a good typing saver. Mar 10, 2013 at 0:23
  • 1) Is there significance to the triple %%% following the \let? 2) What's the significance of the \lowercase? 3) Why did you add a \SaveHardspace to the \string@end definition? 4) What does putting \narg in brackets [] do? Mar 10, 2013 at 2:23
  • @unbonpetit It is difficult to select the Accepted Answer on this question, since all answers were great! unbonpetit's coding is prettiest, and perhaps should be adopted by others following this track (and I recommend others to "like" it as useful). But my request was for a "workaround" indicating minimum changes from what I started with. With that as the metric, the xparse solution gets eliminated while David Carlisle's code accomplished the goal with the minimum disruption to my original MWE. Thus, he gets his answer accepted on technical grounds. My thanks to all. Mar 10, 2013 at 2:35
13

The following works with the input described above.

enter image description here

\documentclass{article}
\usepackage{xparse}
\ExplSyntaxOn
\NewDocumentCommand{\getargs}{m}{
    \get_args:n {#1}
}
\int_new:N \narg
\cs_new_protected:Npn \get_args:n #1 {
    \seq_set_split:Nnn \l_tmpa_seq {~} {#1}
        \seq_map_inline:Nn \l_tmpa_seq {
            \int_incr:N \l_tmpa_int
            \tl_set:cx {arg\int_to_roman:n {\l_tmpa_int}}{##1}
        }
    \int_set:Nn \narg {\l_tmpa_int}
    \int_zero:N \l_tmpa_int
    }
\ExplSyntaxOff
\begin{document}
\getargs{mein hut} \argi~\argii $<$NO PROBLEM\\
BUT PUT UMLAUT ON THE u AND IT BREAKS
\getargs{mein h\"ut} \argi~\argii
\end{document}
10

Here is a solution with no expansion, able to parse macros and groups {...}:

\documentclass{article}
\makeatletter
\newcount\arg@index
\newtoks\code@toks

\def\getargsF#1{%
    \arg@index=1
    \code@toks{}%
    \parse@i#1\parse@stop
}

\def\add@tok#1{\code@toks\expandafter{\the\code@toks#1}}

\def\parse@stop{\parse@stop}

\def\parse@i{\futurelet\nxttok\parse@ii}

\def\parse@ii{%
    \ifx\nxttok\parse@stop \let\next@action\parse@stop@i
    \else
        \ifx\nxttok\@sptoken \let\next@action\read@space
        \else
            \ifx\nxttok\bgroup \let\next@action\read@bracearg
            \else \let\next@action\testtoken
            \fi
        \fi
    \fi
    \next@action
}

\def\parse@stop@i\parse@stop{\assign@arg}

\expandafter\def\expandafter\read@space\space{%
    \assign@arg
    \advance\arg@index1
    \code@toks{}%
    \parse@i
}

\def\read@bracearg#1{%
    \add@tok{{#1}}%
    \parse@i
}

\def\testtoken#1{%
    \if\noexpand~\noexpand#1%
        \ifnum\catcode`#1=\active
            \assign@arg
            \advance\arg@index1
            \code@toks{}%
        \else
            \add@tok{#1}%
        \fi
    \else
        \add@tok{#1}%
    \fi
    \parse@i
}

\def\assign@arg{%
    \expandafter\edef\csname arg\romannumeral\arg@index\endcsname{\the\code@toks}%
}
\makeatother
\begin{document}
\getargsF{abcd efgh~h\"ut}\argi\#\argii\#\argiii\#

\getargsF{maths $1+1=2$ \textbf{b\textit{ol}d} \^a\'i\'o~hardspc~end}\argi\#\argii\#\argiii\#\argiv\#\argv\#\argvi\#

\catcode`\~12
\getargsF{maths $1+1=2$ \textbf{b\textit{ol}d} \^a\'i\'o~hardspc~end}\argi\#\argii\#\argiii\#\argiv\#
\end{document}

which gives

enter image description here

3
  • Just as \protected@edef was a major revelation to me the other week, so your answer reveals the equally amazing \code@toks command. If there are any documents that discuss this command, I'd love a pointer. Mar 9, 2013 at 23:36
  • Also, \futurelet is new to me, but a very useful command, I see in Will Robertson's answer at tex.stackexchange.com/questions/850/… Mar 10, 2013 at 0:14
  • 2
    @StevenB.Segletes \code@toks isn't a pre-defined command it is just a register defined on line 4, same as a count register but for token lists rather than numbers. \futurelet is the tex primitive behind latex's \@ifnextchar or the \IfNextToken that you used. Mar 10, 2013 at 0:42

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .