# Parse Argument by Character while Executing Embedded Macros

I am interesting in parsing a string of input character by character, and doing something to each character. In this MWE, I merely apply a \textbf to each successive character as an example, to verify that the parser is working.

The problem arises if the argument contains embedded macros. What I would like to do, if I come a across a macro in the argument, is to halt the parsing and let the macro execute, eating up as many arguments as it would desire from the original argument stream, and then resume parsing the remainder of the argument.

While \charparse is the parsing macro, I am trying to design a helper macro \execmacro to do what needs to occur when a macro is detected in the argument stream. In the MWE below, I can accomplish that, but only if I presuppose the nature of the embedded macros. In particular, I show 3 versions of \execmacro, depending on whether I presuppose the embedded macros to require 0, 1, or 2 arguments, respectively.

What I would like instead is to have a version of \execmacro that will work regardless of how many arguments the embedded macros demand.

\documentclass{article}
\newcommand\charparse[1]{\charparsehelp#1\relax\relax\relax}
\def\charparsehelp#1#2\relax{%
\ifcat\noexpand\relax\noexpand#1%
%   MACRO DETECTED IN INPUT STREAM
\else\textbf{#1}% BOLDING IS JUST AN EXAMPLE TO SHOW THAT PARSING MACRO IS WORKING
\ifx\relax#2\else
\charparsehelp#2\relax
\fi\fi
}
\begin{document}
% THIS ONLY WORKS IF THE EMBEDDED MACRO TAKES EXACTLY ZERO ARGUMENTS
\def\execmacro#1#2\relax{#1\ifx\relax#2\else\charparsehelp#2\relax\fi}Case 1\par
\charparse{0123\itshape456\upshape789}\par
I would like the output from the above character-parsing macro to be\par
\charparse{0123}\itshape\charparse{456}\upshape\charparse{789}\medskip

% THIS ONLY WORKS IF THE EMBEDDED MACRO TAKES EXACTLY ONE ARGUMENT
\def\execmacro#1#2#3\relax{#1{#2}\ifx\relax#3\else\charparsehelp#3\relax\fi}Case 2\par
\charparse{0123\textit{456}789}\par
I would like the output from the above character-parsing macro to be\par
\charparse{0123}\textit{456}\charparse{789}\medskip

% THIS ONLY WORKS IF THE EMBEDDED MACRO TAKES EXACTLY TWO ARGUMENTS
\def\execmacro#1#2#3#4\relax{#1{#2}{#3}\ifx\relax#4\else\charparsehelp#4\relax\fi}Case 3\par
\charparse{0123\rule{3ex}{1ex}456789}\par
I would like the output from the above character-parsing macro to be\par
\charparse{0123}\rule{3ex}{1ex}\charparse{456789}
\end{document}


• You surely know that a macro such as \makebox has no argument: you'd need to build a TeX parser. – egreg Oct 7 '15 at 14:12
• @egreg I do not understand what your comment is trying to tell me. – Steven B. Segletes Oct 7 '15 at 14:20
• You can't decide the action of a macro just based on the number of its arguments. – egreg Oct 7 '15 at 14:22
• @egreg So that I understand you, you mean that a macros actions can chew up more arguments than are nominally called for. – Steven B. Segletes Oct 7 '15 at 14:24
• @egreg I understand that, in advance, one in general does not know how much of the argument stream will be eaten by the macro. What I was hoping is that a way exists to temporarily redefine the macro to reinvoke parsing as its final dying act. – Steven B. Segletes Oct 7 '15 at 14:29

Well, since no one wants to touch this one, and all the gurus advise strongly against it, I should probably let it lie. But I won't. Just to show what might be done, I followed the hint mentioned by egreg, to the effect of "registering" allowable macros that can be processed.

At this time, the presence of optional arguments are not allowed, which is something of a drawback. And for demonstration purposes, I have set it up to handle macros with up to 2 arguments only.

So here is how it works. I wish to develop \charparse{} that parses the argument 1 character at a time. But the key is to be able to execute macros in the argument and then resume the character by character parsing following their execution.

If a macro has no arguments, it should not be registered. Otherwise, the macros have to be registered with \registerparsemacro}{\<macroname>}{<arguments>}. To repeat, optional arguments are not allowed. So, for instance, I invoke

\registerparsemacro{\textit}{1}
\registerparsemacro{\rule}{2}


At that point I can include invocations of \textit and \rule (without optional argument) in the argument to \charparse. The macros \itshape and \upshape should not be registered, since they take no arguments.

The following MWE creates the same output as the question shows.

\documentclass{article}
\usepackage{ifthen}
\newcommand\charparse[1]{\charparsehelp#1\relax\relax\relax}
\def\charparsehelp#1#2\relax{%
\ifcat\noexpand\relax\noexpand#1%
%   MACRO DETECTED IN INPUT STREAM
\else\textbf{#1}% BOLDING IS JUST AN EXAMPLE TO SHOW THAT PARSING MACRO IS WORKING
\ifx\relax#2\else
\charparsehelp#2\relax
\fi\fi
}
\newcounter{parsemacro}
\def\execmacro#1#2#3#4\relax{%
\setcounter{parsemacro}{0}%
\whiledo{\value{parsemacro} < \value{parsemacrocount}}{%
\stepcounter{parsemacro}%
\expandafter\expandafter\expandafter%
\ifx\csname parsemacro\romannumeral\value{parsemacro}\endcsname#1%
\if1\csname parsemacroarguments\romannumeral\value{parsemacro}\endcsname
\execmacroONE#1{#2}#3#4\relax\else
\if2\csname parsemacroarguments\romannumeral\value{parsemacro}\endcsname
\execmacroTWO#1{#2}{#3}#4\relax\else
\fi\fi
\setcounter{parsemacro}{\numexpr\value{parsemacrocount}+1}%
\fi
}%
\ifnum\value{parsemacro}=\value{parsemacrocount}\execmacroZERO#1#2#3#4\relax\fi
}
\def\execmacroZERO#1#2\relax{#1\ifx\relax#2\else
\charparsehelp#2\relax\fi}
\def\execmacroONE#1#2#3\relax{#1{#2}\ifx\relax#3\else
\charparsehelp#3\relax\fi}
\def\execmacroTWO#1#2#3#4\relax{#1{#2}{#3}\ifx\relax#4\else
\charparsehelp#4\relax\fi}
\newcounter{parsemacrocount}
\setcounter{parsemacrocount}{0}
\def\registerparsemacro#1#2{%
\stepcounter{parsemacrocount}%
\expandafter\def\csname parsemacro\romannumeral\value{parsemacrocount}\endcsname{#1}%
\expandafter\def\csname parsemacroarguments\romannumeral\value{parsemacrocount}%
\endcsname{#2}%
}
\begin{document}
\registerparsemacro{\textit}{1}
\registerparsemacro{\rule}{2}
Case 1\par\charparse{0123\itshape456\upshape789}\par
I would like the output from the above character-parsing macro to be\par
\charparse{0123}\itshape\charparse{456}\upshape\charparse{789}\medskip

Case 2\par\charparse{0123\textit{456}789}\par
I would like the output from the above character-parsing macro to be\par
\charparse{0123}\textit{456}\charparse{789}\medskip

Case 3\par\charparse{0123\rule{3ex}{1ex}456789}\par
I would like the output from the above character-parsing macro to be\par
\charparse{0123}\rule{3ex}{1ex}\charparse{456789}
\end{document}

• I would just step over all commands and brace groups then you don't need to list most commands \foo{zz} could be skipped (if you don't mind skipping {abc} but * forms and optional arguments are pretty hard to deal with something like \section*[abc]{yyz} the *[abc] is just character data unless you know the operational behaviour of \section – David Carlisle Oct 7 '15 at 20:06
• @DavidCarlisle Thanks for the idea of skipping over brace groups. To go that route would allow the avoidance of registering each macro anticipated. But I would still need to execute them as I am skipping over them. – Steven B. Segletes Oct 7 '15 at 20:10
• No I'd just gather up all of the original argument into a token register, characters you split up and add to the token register as \foo{x} csnames and brace groups just add to the register as-is, then at the end do \the\toks@` to play it all through, – David Carlisle Oct 7 '15 at 20:14