4

The following (classical, I assume) "iterator" definition would be examined here:

\def\zEnd{\zEnd}
\def\zzIterator#1{%
    \ifx#1\zEnd
    \else
      #1%
      \expandafter\zzIterator
    \fi
  }
\def\zIterator#1{\zzIterator#1\zEnd}
\zIterator{Something...}

Foremost, I've seen \def\zEnd{\zEnd} and \def\zEnd{\zIterator} used, what's the difference (which one is better to use)?

I've been referred to "Iterate over tokens" to see how to retain (otherwise gobbled/lost) spaces. Since I prefer to work in as much plain TeX as I can, I picked the answer which uses \let. But since I also have to capitalize some of the characters (tokens), \let method either needs modification (which I'm not aware of) or should be abandoned in favor of another (method). This is my effort to determine, with your help (of course), how to solve this problem.

I ran into a dead end (once again) trying to resolve it my way: by implementing a switch (count register) I were to indicate if the current token in iteration has to be uppercase (so that the conversion to uppercase could take place immediately, switch were to be set inside one of the conditional cases that would determine which tokens to convert--hence the need for a switch, multiple cases), but the value of the switch for some reason wasn't set until next iteration (which was too late, next token's iteration was processing previous token's switch instead of previous token's iteration processing it). Here is my erroneous effort (not to mention it doesn't retain spaces):

\documentclass[margin=5mm,varwidth]{standalone}
\begin{document}
  \newcount\zCapSwitch % UPPERCASE SWITCH
  \zCapSwitch0 % SET TO FALSE (NO UPPERCASE CONVERSION NEEDED)
  \def\zEnd{\zEnd}
  \def\zzIterator#1{%
      \ifx#1\zEnd
      \else
        % ------------ %
        % OUTPUT CHUNK %
        % ------------ %
        % CAPITALIZE "s" AND "i"
        % SOMETHING IS WRONG HERE
        %  (COUNTER DOESN'T GET SET UNTIL
        %   NEXT ITERATION, WHEN IT'S TOO LATE)
        \ifx#1s\zCapSwitch1\fi
        \ifx#1i\zCapSwitch1\fi
        \ifnum\zCapSwitch=1
          \uppercase{#1}% IT'S TOO LATE, WE'RE UPPERCASING THE WRONG TOKEN
          \zCapSwitch0 % RESET SWITCH (TO FALSE)
        \else
          #1%
        \fi
        % ------------ %
        \expandafter\zzIterator
      \fi
    }
  \def\zIterator#1{%
      \zzIterator#1\zEnd
    }
  \zIterator{Keep spaces intact!}
\end{document}

Your solutions are welcome (plain TeX is of precedence).

  • 2
    Why do you ask for a plain tex solution but then provide a test document that requires latex and can not be run with plain? LaTeX does not include the plain tex sources at all. – David Carlisle Nov 18 '19 at 9:28
  • This could be of academic interest, but I'm not really going into it unless you show a good reason for using \let\zTest=a in the first place and what all this would be used for. – egreg Nov 18 '19 at 11:07
4

Here's a completely different approach. It is the same approach as in Joseph's answer here, which allows one to iterate over a token list looking for a \%, and it's the same approach as used in the LaTeX3 kernel for \tl_(upper|lower|mixed)_case:n, for the emulated e-type expansion prior to TL 19, when \expanded wasn't available in most engines (darker ages. . .), for \tl_count_tokens:n, \tl_reverse, and a couple of others. It goes without saying that a lot of the code here is a copy of expl3 functions, since you ruled that out.

Also, I suppose from your code that you want to iterate over a token list and make both i and s uppercase. I defined the function below to do so, but it isn't very clear what you want to achieve.

The problem with iterating over a token list is that sometimes you can't simply grab something as argument. When TeX grabs an undelimited argument, it a) ignores spaces until the first non-space token, and b) removes one layer of braces if the grabbed argument starts with { and ends with }. So, suppose \def\foo#1{(#1)}, all three of \foo A, \foo ␣A, and \foo{A} expand to the same (A). Here's where you lose your spaces (and groups) with your \zIterator function.

To preserve spaces and groups you need to define speficic macros to handle spaces and groups, and another to handle other (so-called N-type) tokens. The code below defines (read: copies from expl3 and replaces _ and : by @) \tlhead@if@Ntype, \tlhead@if@group, and \tlhead@if@space. These three conditional functions take a token list as argument and check what of the three cases above the first token of the token list falls in. With these three conditionals you can do something like \ifNtype{<normal>}{\ifgroup{<grouped tokens>}{<space token>}} and process the entire token list.

The code below then defines \zIterator as a shorthand for \zIterator@loop#1\zIterator@stop plus some bookkeeping. \zIterator@loop will loop through the token list, checking the type of the next token, and using \zIterator@Ntype, \zIterator@group, or \zIterator@space, as necessary. \zIterator@group will simply call \zIterator recursively inside a group. \zIterator@space will move one space from the input to the output token list.

\zIterator@Ntype will call a \zIterator@do@token which will take one token as argument, and process it as required. I defined \zIterator@do@token to check for an i or for an s, and make them uppercase. The upper casing is based on a lookup in the hash table. For each character <char> to be upper cased, there is a macro \zIterator@upper@<char>@case which expands to the upper case version of that character. You can define these with \zIteratorUppercase{<char>}{<upper case form>}. I didn't put too much thought in the upper casing code, so you could probably improve it a lot. This one is just for a proof of concept.

With the code below, the output of \tt\zIterator{Keep spaces {\bf (and groups)} intact!} is:

enter image description here

And here's the code:

\catcode`\@=11
% Copies of \tl_if_head_is(N_type|group|space):nTF from expl3:
\def\usenoneN#1{}
\def\useINN#1#2{#1}
\def\useIINN#1#2{#2}
\newcount\exp@end \exp@end=0
\long\def\tlhead@if@Ntype#1{%
  \ifcat
      \iffalse{\fi\tlhead@if@Ntype@?#1 }%
        \expandafter\usenoneN
        \expandafter{\expandafter{\string#1?}}%
      **%
    \expandafter\useINN
  \else
    \expandafter\useIINN
  \fi}
\long\def\tlhead@if@Ntype@#1 {%
  \if\relax\detokenize\expandafter{\usenoneN#1}\relax^\fi
  \expandafter\usenoneN\expandafter{\iffalse}\fi}
\long\def\tlhead@if@group#1{%
  \ifcat
      \expandafter\usenoneN
      \expandafter{\expandafter{\string#1?}}%
      **%
    \expandafter\useIINN
  \else
    \expandafter\useINN
  \fi}
\long\def\tlhead@if@space#1{%
  \romannumeral\iffalse{\fi
  \tlhead@if@space@?#1? }}
\long\def\tlhead@if@space@#1 {%
  \if\relax\detokenize\expandafter{\usenoneN#1}\relax
    \tlhead@if@space@@\useINN
  \else
    \tlhead@if@space@@\useIINN
  \fi
  \expandafter\usenoneN\expandafter{\iffalse}\fi}
\def\tlhead@if@space@@#1#2\fi{\fi\expandafter\expandafter\expandafter#1}
% Iterate over the token list:
\def\zIterator@end{\zIterator@end}
\long\def\zIterator#1{\romannumeral%
  \zIterator@loop#1\zIterator@end\zIterator@stop{}}
\long\def\zIterator@loop#1\zIterator@stop{%
  \tlhead@if@Ntype{#1}
    {\zIterator@Ntype}
    {\tlhead@if@group{#1}
      {\zIterator@group}
      {\zIterator@space}}%
  #1\zIterator@stop}
% Handling N-type tokens
\long\def\zIterator@Ntype#1{%
  \ifx\zIterator@end#1%
    \expandafter\zIterator@finish
  \fi
  \zIterator@do@token{#1}}
% Handling space tokens
\useINN{\long\def\zIterator@space}{} {\zIterator@return{ }}
% Handling grouped tokens
\long\def\zIterator@group#1{%
  \expandafter\expandafter\expandafter\zIterator@group@return
  \expandafter\expandafter\expandafter{\zIterator{#1}}}
\long\def\zIterator@group@return#1{\zIterator@return{{#1}}}
% Output:
\long\def\zIterator@return#1#2\zIterator@stop#3{%
  \zIterator@loop#2\zIterator@stop{#3#1}}
\long\def\zIterator@finish#1\zIterator@stop#2{\exp@end#2}
%
% Hash table-based upper casing:
\long\def\zIterator@do@token#1{%
  \ifnum0%
      \if s\noexpand#11\fi
      \if i\noexpand#11\fi
      >0
    \expandafter\zIterator@upper@case
  \else
    \expandafter\zIterator@return
  \fi{#1}}
\long\def\zIterator@upper@case#1{%
  \expandafter\expandafter\expandafter\zIterator@return
  \expandafter\expandafter\expandafter{\csname zIterator@upper@#1@case\endcsname}}
\long\def\zIteratorUppercase#1#2{%
  \expandafter\def\csname zIterator@upper@#1@case\endcsname{#2}}
\zIteratorUppercase{s}{S}
\zIteratorUppercase{i}{I}

\tt\zIterator{Keep spaces {\bf (and groups)} intact!}

\bye

And with the proper (10 or so lines of) boilerplate code it works with (ε-)INITEX, so it doesn't even depend on plain TeX, should Knuth decide to make some radical changes in it next year ;-)


Since Ulrich mentioned expl3 in his answer, I figured I'd add an implementation here, just for comparison (about 40 lines saved, plus much easier case changing):

\input expl3-generic
\ExplSyntaxOn
\quark_new:N \q__zbp_end
% Iterate over the token list:
\cs_new:Npn \zbp_iterator:n #1
  { \exp:w \__zbp_loop:w #1 \q__zbp_end \q__zbp_stop { } }
\cs_new:Npn \__zbp_loop:w #1 \q__zbp_stop
  {
    \tl_if_head_is_N_type:nTF {#1}
      { \__zbp_N_type:N }
      {
        \tl_if_head_is_group:nTF {#1}
          { \__zbp_group:n } { \__zbp_space:w }
      }
    #1 \q__zbp_stop
  }
% Handling N-type tokens
\cs_new:Npn \__zbp_N_type:N #1
  {
    \token_if_eq_meaning:NNT #1 \q__zbp_end { \__zbp_finish:w }
    \__zbp_do_token:N #1
  }
% Handling space tokens
\exp_last_unbraced:NNo
\cs_new:Npn \__zbp_space:w \c_space_tl { \__zbp_return:n { ~ } }
% Handling grouped tokens
\cs_new:Npn \__zbp_group:n #1
  { \exp_args:NNo \exp_args:No \__zbp_group_back:n { \zbp_iterator:n {#1} } }
\cs_new:Npn \__zbp_group_back:n #1 { \__zbp_return:n { {#1} } }
% Output:
\cs_new:Npn \__zbp_return:n #1 #2 \q__zbp_stop #3
  { \__zbp_loop:w #2 \q__zbp_stop {#3 #1} }
\cs_new:Npn \__zbp_finish:w #1 \q__zbp_stop #2 { \exp_end: #2 }
%
% Hash table-based upper casing:
\cs_new:Npn \__zbp_do_token:N #1
  {
    \str_case:nnTF {#1}
      {
        {s}{ }
        {i}{ }
      }
      { \exp_args:Nf \__zbp_return:n { \tl_upper_case:n {#1} } }
      { \__zbp_return:n {#1} }
  }
% Interfaces
\cs_new:Npn \zIterator { \zbp_iterator:n }
\ExplSyntaxOff

\tt\zIterator{Keep spaces {\bf (and groups)} intact!}

\bye
| improve this answer | |
  • As I am still learning expl3, I appreciate the expl3-example. Thank you very much. :-) – Ulrich Diez Nov 19 '19 at 15:19
  • @UlrichDiez Glad you like it :-) expl3 is scary at first, but once you get used to the _ and :, it's not that different from normal TeX. It's great to see you're interested in the project! – Phelype Oleinik Nov 19 '19 at 19:29
  • @PhelypeOkeinik Nowadays I even have to get used to LaTeX 2e/source2e again. ;-> There is a lot of code to read.over the next months: source2e, expl3, xparse... ;-) Next thing I want to find out is why, when defining my own argument-processor usingr xparse's \ProcessList{<list>}{<function>} , I cannot do something like \ProcessList{<list>}{<function that processes 2 args>{<function's 1st arg>}}. No, don't tell me - I want to find out myself. ;-) – Ulrich Diez Nov 20 '19 at 9:22
  • @UlrichDiez Okay, won't tell. I'll just say it's possible (the functionality exists ;-) – Phelype Oleinik Nov 20 '19 at 10:35
3

The reason why your routine does not even replace s by S and i by I is in the lines:

\ifx#1s\zCapSwitch1\fi
\ifx#1i\zCapSwitch1\fi

In case TeX gathers a number digit-character-token by digit-character-token, it will keep expanding tokens. \fi is expandable, thus TeX does not stop gathering digits when having encountered \fi.

Thus TeX is still gathering digits of the number of the \zCapSwitch-assignment when carrying out the subsequent \ifnum-comparison.

Thus the new value is not yet assigned to \zCapSwitch when the subsequent \ifnum-comparison is carried out.

Just make sure to terminate the \zCapSwitch-assignment by having the digit-sequence that forms the number trailed by a space or by \relax:

\ifx#1s\zCapSwitch1 \fi
\ifx#1i\zCapSwitch1 \fi

TeX will take the spaces behind 1 for something that terminates the digit-sequence and therefore will discard them instead of keeping them and producing horizontal glue.

When you do that, the code looks almost the same, but replacing s and i works to some extent:

\documentclass[margin=5mm,varwidth]{standalone}
\begin{document}
  \newcount\zCapSwitch % UPPERCASE SWITCH
  \zCapSwitch0 % SET TO FALSE (NO UPPERCASE CONVERSION NEEDED)
  \def\zEnd{\zEnd}
  \def\zzIterator#1{%
      \ifx#1\zEnd
      \else
        % ------------ %
        % OUTPUT CHUNK %
        % ------------ %
        % CAPITALIZE "s" AND "i"
        % 
        % YOU NEED SPACES TO TERMINATE DIGIT-SEQUENCES.
        % 
        \ifx#1s\zCapSwitch1 \fi
        \ifx#1i\zCapSwitch1 \fi
        \ifnum\zCapSwitch=1
          \uppercase{#1}%
          \zCapSwitch0 %
        \else
          #1%
        \fi
        % ------------ %
        \expandafter\zzIterator
      \fi
    }
  \def\zIterator#1{%
      \zzIterator#1\zEnd
    }
  \zIterator{Keep spaces intact!}
\end{document}

enter image description here

But spaces are still not preserved.

The reason is that \zzIterator does process a non-delimited macro-argument.
While gathering the tokens that belong to a non-delimited macro-argument, TeX always drops/removes/discards explicit space-tokens that precede the non-delimited macro-argument.

Besides this the case of the argument of \zIterator containing brace-groups is not handled.

Here is an approach where \futurelet is used for "looking ahead" at the meaning of the next token and \afterassignment-\let is used for removing tokens whose meaning equals the meaning of the space-token.

This approach sort of preserves spaces.

But it does still not handle the case of the argument of \zIterator containing brace-groups:

\documentclass[margin=5mm,varwidth]{standalone}
\begin{document}
  \newcount\zCapSwitch % UPPERCASE SWITCH
  \zCapSwitch0 % SET TO FALSE (NO UPPERCASE CONVERSION NEEDED)
  \def\zEnd{\zEnd}%
  \long\def\foo#1{#1}%
  \long\def\fot#1#2{#1}%
  \long\def\sot#1#2{#2}%
  \foo{\let\zzSpace= } %
  \def\zzIterator{\futurelet\zzNext\zzSpacefork}%
  \def\zzSpacefork{%
    \ifx\zzNext\zzSpace
      \expandafter\fot
    \else
      \expandafter\sot
    \fi
    { \afterassignment\zzIterator\let\zzNext= }%
    {\zzIteratorA}%
  }%
  \def\zzIteratorA#1{%
      \ifx#1\zEnd
      \else
        % ------------ %
        % OUTPUT CHUNK %
        % ------------ %
        % CAPITALIZE "s" AND "i"
        % 
        % YOU NEED SPACES TO TERMINATE DIGIT-SEQUENCES.
        % 
        \ifx#1s\zCapSwitch1 \fi
        \ifx#1i\zCapSwitch1 \fi
        \ifnum\zCapSwitch=1
          \uppercase{#1}% 
          \zCapSwitch0 %
        \else
          #1%
        \fi
        % ------------ %
        \expandafter\zzIterator
      \fi
    }
  \def\zIterator#1{%
      \zzIterator#1\zEnd
    }
  \zIterator{Keep spaces intact!}
\end{document}

enter image description here


Handling brace-groups is an interesting subject because having TeX "look ahead" at the meaning of the next token via \futurelet or \let actually is not sufficient:

  1. This way you can find out whether the meaning of the next token equals, e.g., the meaning of the character-token {1, i.e., the meaning of that character-token whose character-code is 123—123 is the number of the code-point of the {-character in the TeX-engine's internal character representation scheme—and whose category-code is 1(begin group). But you cannot find out whether that next token is explicit, i.e., whether that next token is the explicit {1(begin group)-token, or is implicit, i.e., is something like the \bgroupcontrol word-token, with \let\bgroup={.
    "Explicitness/implicitness" of character-tokens of category-code 1(begin group) respectively category-code 2(end group) does matter because non-delimited macro-arguments being empty or having a leading explicit space-token or consisting of more than one token and delimited macro-arguments containing the argument-delimiter are to be nested into a pair of explicit character-tokens of category-code 1(begin group) respectively category-code 2(end group) while implicit character-tokens of category-code 1(begin group) respectively category-code 2(end group) will by TeX not be taken for markers for the begin or the end of a macro-argument.

  2. Besides this: Some day somebody might—in addition to the character { respectively }—assign category-code 1(begin group) respectively category-code 2(end group) to some other characters, which then will, e.g., in matters of beginning/ending a local scope and in matters of gathering macro-arguments and in matters of gathering ⟨balanced text⟩, be treated by TeX like { respectively }, but whose meaning will not equal the meaning of {1(begin group) respectively }2(end group) as the character-codes are different.

I can offer a \romannumeral0-expansion-based tail-recursive routine for replacing the explicit catcode-11(letter)-character-tokens s and i by explicit catcode-11(letter)-character-tokens S respectively I where

  • the result is delivered after two "hits" by \expandafter.
  • no temporay assignments and the like take place. E.g., things like \afterassignment/\let/\futurelet are not used. Therefore the routine can also be used in expansion-contexts, e.g., within \csname..\endcsname.
  • (unmatched) \if.. /\else/\fi in the argument do not disturb the routine as the routine is based on delimited arguments and therefore does not use \if..-tests at all.
  • although the token \UDSelDOm is used as "sentinel-token" in some places, that token can occur within the argument, thus there are no forbidden tokens for the argument. (Except that you generally cannot use \outer tokens in macro-arguments.)
  • the mechanism can cope with things being nested in curly braces.
  • you don't need to think about the question of distinguishing explicit brace- and/or space-tokens from their implicit pendants. (When "looking ahead" at the next token via \let or \futurelet, this question probably might cause headache because \let and \futurelet let you cope just with the meanings of tokens while the meaning of a character-token does not include information regarding whether that character-token is an explicit or an implicit character-token...)

A side-effect of the routine is that it replaces

  • all explicit-catcode 1(begin group)-character-tokens by explicit curly-opening-brace-character-tokens ({) of catcode 1(begin group).
  • all explicit-catcode 2(end group)-character-tokens by explicit curly-closing-brace-character-tokens (}) of catcode 2(end group).

Usually { is the only character whose catcode is 1(begin group).
Usually } is the only character whose catcode is 2(end group).
Therefore this usually should not be a problem.

The routine is just an example. You can enhance it to replace characters other than s and i by modifying the macros \UDsiSelect and \UDsiFork. If you do that, you also need to adjust the call to \UDsiFork within \zIteratorTailRecursiveLoop.

(This is a bunch of code. If you were not insisting in plain TeX, things could be shortened, e.g., using expl3.)

%%
%% As you wished plain TeX, don't use latex/pdflatex but use tex/pdftex for compiling.
%%
%%=============================================================================
%% Paraphernalia:
%%    \UDfirstoftwo, \UDsecondoftwo,
%%    \UDExchange, \UDPassFirstBehindThirdToSecond
%%    \UDremovespace, \UDCheckWhetherNull, \UCheckWhetherBrace,
%%    \UDCheckWhetherLeadingSpace, \UDExtractFirstArg
%%=============================================================================
\long\def\UDfirstoftwo#1#2{#1}%
\long\def\UDsecondoftwo#1#2{#2}%
\long\def\UDExchange#1#2{#2#1}%
\long\def\UDPassFirstBehindThirdToSecond#1#2#3{#2{#3}{#1}}%
\UDfirstoftwo{\def\UDremovespace}{} {}%
%%=============================================================================
%% Check whether argument is empty:
%%=============================================================================
%% \UDCheckWhetherNull{<Argument which is to be checked>}%
%%                     {<Tokens to be delivered in case that argument
%%                       which is to be checked is empty>}%
%%                     {<Tokens to be delivered in case that argument
%%                       which is to be checked is not empty>}%
%%
%% Due to \romannumeral0-expansion the result is delivered after two
%% expansion-steps/after two "hits" by \expandafter.
%%
%% The gist of this macro comes from Robert R. Schneck's \ifempty-macro:
%% <https://groups.google.com/forum/#!original/comp.text.tex/kuOEIQIrElc/lUg37FmhA74J>
%%
\long\def\UDCheckWhetherNull#1{%
  \romannumeral0\expandafter\UDsecondoftwo\string{\expandafter
  \UDsecondoftwo\expandafter{\expandafter{\string#1}\expandafter
  \UDsecondoftwo\string}\expandafter\UDfirstoftwo\expandafter{\expandafter
  \UDsecondoftwo\string}\UDfirstoftwo\expandafter{} \UDsecondoftwo}%
  {\UDfirstoftwo\expandafter{} \UDfirstoftwo}%
}%
%%=============================================================================
%% Check whether argument's first token is a catcode-1-character
%%=============================================================================
%% \UDCheckWhetherBrace{<Argument which is to be checked>}%
%%                      {<Tokens to be delivered in case that argument
%%                        which is to be checked has leading
%%                        catcode-1-token>}%
%%                      {<Tokens to be delivered in case that argument
%%                        which is to be checked has no leading
%%                        catcode-1-token>}%
%%
%% Due to \romannumeral0-expansion the result is delivered after two
%% expansion-steps/after two "hits" by \expandafter.
%%
\long\def\UDCheckWhetherBrace#1{%
  \romannumeral0\expandafter\UDsecondoftwo\expandafter{\expandafter{%
  \string#1.}\expandafter\UDfirstoftwo\expandafter{\expandafter
  \UDsecondoftwo\string}\UDfirstoftwo\expandafter{} \UDfirstoftwo}%
  {\UDfirstoftwo\expandafter{} \UDsecondoftwo}%
}%
%%=============================================================================
%% Check whether brace-balanced argument's first token is an explicit
%% space token
%%=============================================================================
%% \UDCheckWhetherLeadingSpace{<Argument which is to be checked>}%
%%                             {<Tokens to be delivered in case <argument
%%                               which is to be checked>'s 1st token is a
%%                               space-token>}%
%%                             {<Tokens to be delivered in case <argument
%%                               which is to be checked>'s 1st token is not
%%                               a space-token>}%
%%
%% Due to \romannumeral0-expansion the result is delivered after two
%% expansion-steps/after two "hits" by \expandafter.
%%
\long\def\UDCheckWhetherLeadingSpace#1{%
  \romannumeral0\UDCheckWhetherNull{#1}%
  {\UDfirstoftwo\expandafter{} \UDsecondoftwo}%
  {\expandafter\UDsecondoftwo\string{\UDInnerCheckWhetherLeadingSpace.#1 }{}}%
}%
\long\def\UDInnerCheckWhetherLeadingSpace#1 {%
  \expandafter\UDCheckWhetherNull\expandafter{\UDsecondoftwo#1{}}%
  {\UDExchange{\UDfirstoftwo}}{\UDExchange{\UDsecondoftwo}}%
  {\UDExchange{ }{\expandafter\expandafter\expandafter\expandafter
   \expandafter\expandafter\expandafter}\expandafter\expandafter
   \expandafter}\expandafter\UDsecondoftwo\expandafter{\string}%
}%
%%=============================================================================
%% Extract first inner undelimited argument:
%%=============================================================================
%%   \UDExtractFirstArg{ABCDE} yields  {A}
%%   \UDExtractFirstArg{{AB}CDE} yields  {AB}
%%
%% Due to \romannumeral0-expansion the result is delivered after two
%% expansion-steps/after two "hits" by \expandafter.
%%
\long\def\UDRemoveTillUDSelDOm#1#2\UDSelDOm{{#1}}%
\long\def\UDExtractFirstArg#1{%
  \romannumeral0%
  \UDExtractFirstArgLoop{#1\UDSelDOm}%
}%
\long\def\UDExtractFirstArgLoop#1{%
  \expandafter\UDCheckWhetherNull\expandafter{\UDfirstoftwo{}#1}%
  { #1}%
  {\expandafter\UDExtractFirstArgLoop\expandafter{\UDRemoveTillUDSelDOm#1}}%
}%
%%=============================================================================
%% Extract K-th inner undelimited argument:
%%=============================================================================
%% \UDExtractKthArg{<integer K>}{<list of undelimited args>} 
%% 
%% In case there is no K-th argument in <list of indelimited args> : 
%%   Does not deliver any token.
%% In case there is a K-th argument in <list of indelimited args> : 
%%   Does deliver that K-th argument with one level of braces removed.
%%
%% Examples:
%%
%%   \UDExtractKthArg{0}{ABCDE} yields: <nothing>
%%
%%   \UDExtractKthArg{3}{ABCDE} yields:  C
%%
%%   \UDExtractKthArg{3}{AB{CD}E} yields:  CD
%%
%%   \UDExtractKthArg{4}{{001}{002}{003}{004}{005}} yields: 004
%%
%%   \UDExtractKthArg{6}{{001}{002}{003}} yields: <nothing> 
%%
%% Due to \romannumeral0-expansion the result is delivered after two
%% expansion-steps/after two "hits" by \expandafter.
%%
\long\def\UDExtractKthArg#1{%
  \romannumeral0%
  % #1: <integer number K>
  \expandafter\UDExtractKthArgCheck
  \expandafter{\romannumeral\number\number#1 000}%
}%
\long\def\UDExtractKthArgCheck#1#2{%
  \UDCheckWhetherNull{#1}{ }{%
    \expandafter\UDExtractKthArgLoop\expandafter{\UDfirstoftwo{}#1}{#2}%
  }%
}%
\long\def\UDExtractKthArgLoop#1#2{%
  \expandafter\UDCheckWhetherNull\expandafter{\UDfirstoftwo#2{}.}{ }{%
    \UDCheckWhetherNull{#1}{%
      \expandafter\UDExchange
      \romannumeral0\UDExtractFirstArgLoop{#2\UDSelDOm}{ }%
    }{%
      \expandafter\UDExchange\expandafter{\expandafter{\UDfirstoftwo{}#2}}%
      {\expandafter\UDExtractKthArgLoop\expandafter{\UDfirstoftwo{}#1}}%
    }%
  }%
}%
%%=============================================================================
%% Fork whether argument either is an _explicit_ 
%% catcode 11(letter)-character-token of the set {s, i}
%% or is something else.
%%=============================================================================
%% \UDsiFork{<Argument to check>}{%
%%   {<tokens to deliver in case <Argument to check> is s>}%
%%   {<tokens to deliver in case <Argument to check> is i>}%
%%   {<tokens to deliver in case <Argument to check> is empty or something else>}%
%% }%
%%
%% Due to \romannumeral0-expansion the result is delivered after two
%% expansion-steps/after two "hits" by \expandafter.
%%
\long\def\UDGobbleToExclam#1!{}%
\long\def\UDCheckWhetherNoExclam#1{%
  \expandafter\UDCheckWhetherNull\expandafter{\UDGobbleToExclam#1!}%
}%
\long\def\UDsiSelect#1!!s!i!#2#3!!!!{#2}%
\long\def\UDsiFork#1#2{%
  \romannumeral
  \UDCheckWhetherNoExclam{#1}{%
    \UDsiSelect
    !#1!s1!i!{\expandafter\UDsecondoftwo\UDExtractKthArg{3}{#2}}% empty
    !!#1!i!{\expandafter\UDsecondoftwo\UDExtractKthArg{1}{#2}}% s
    !!s!#1!{\expandafter\UDsecondoftwo\UDExtractKthArg{2}{#2}}% i
    !!s!i!{\expandafter\UDsecondoftwo\UDExtractKthArg{3}{#2}}% something else without !
    !!!!%
  }{\expandafter\UDsecondoftwo\UDExtractKthArg{3}{#2}}% something else with !
}%
%%=============================================================================
%% The main routine which calls the main loop:
%%=============================================================================
%% \zIterator{<Argument where s respectively s to be replaced by S respectively I>}
%%
%% Due to \romannumeral0-expansion the result is delivered after two
%% expansion-steps/after two "hits" by \expandafter.
%%
\long\def\zIterator{%
  \romannumeral0\zIteratorTailRecursiveLoop{}%
}%
%%=============================================================================
%% The main loop:
%%=============================================================================
%% \zIteratorTailRecursiveLoop{<list of tokens where replacement
%%                              is already done>}%
%%                            {<remaining list of tokens where replacement of
%%                              s/i by S/I must still be performed>}%
%%
%% In case the <remaining list of tokens where replacement of s/i by S/I must
%% still be performed> is empty, you are done, thus deliver the <list of tokens
%% where replacement is already done>.   
%% Otherwise:
%%    Check if the <remaining list of tokens where replacement of s/i 
%%    by S/I must still be performed> has a leading space. 
%%    If so: Add a space-token to the <list of tokens  where replacement is 
%%           already done>.
%%           Remove the leading space token from the <remaining list of tokens 
%%           where  replacement of s/i by S/I must still be performed>
%%    Otherwise:
%%           Check if the <remaining list of tokens where replacement of s/i
%%           by S/I must still be performed> has a leading brace. 
%%           If so: Extract its first component/its first undelimited argument
%%                  and apply this routine to that extraction and add the
%%                  result (nested in braces) to the  <list of tokens  where
%%                  replacement is already done> .
%%           Otherwise:
%%                   Check if the <remaining list of tokens where replacement
%%                   of s/i by S/I must still be performed>'s first component
%%                   is s or i.
%%                   If so: add "S" respectively "I" to the <list of tokens
%%                          where replacement is already done> .
%%                   Otherwise:
%%                          Add the <remaining list of tokens where replacement
%%                          of s/i by S/I must still be performed>'s first 
%%                          component to the <list of tokens where replacement
%%                          is already done> .
%%           Remove the first compoinent/the first undelimited argument from
%%           the <remaining list of tokens where replacement of s/i by S/I 
%%           must still be performed>.
\long\def\zIteratorTailRecursiveLoop#1#2{%
  % #1 - list of tokens where replacement is already done
  % #2 - remaining list of tokens where replacement of s/i by S/I must
  %      still be performed
  \UDCheckWhetherNull{#2}{ #1}{%
    \UDCheckWhetherLeadingSpace{#2}{%
      \expandafter\UDPassFirstBehindThirdToSecond\expandafter{%
        \UDremovespace#2%
      }{%
        \UDPassFirstBehindThirdToSecond{#1 }{\UDsecondoftwo{}}%
      }%
    }{%
      \expandafter\UDPassFirstBehindThirdToSecond\expandafter{%
        \UDfirstoftwo{}#2%
      }{%
        \UDCheckWhetherBrace{#2}{%
          \expandafter\UDPassFirstBehindThirdToSecond\expandafter{%
            \romannumeral0\expandafter
            \UDExchange\expandafter{\expandafter{%
              \romannumeral0\expandafter\zIteratorTailRecursiveLoop
              \expandafter{\expandafter}%
              \romannumeral0\UDExtractFirstArgLoop{#2\UDSelDOm}%
            }}{ #1}%
          }{\UDsecondoftwo{}}%
        }{%
          \expandafter\UDsiFork
          \romannumeral0\UDExtractFirstArgLoop{#2\UDSelDOm}{%
            {\UDPassFirstBehindThirdToSecond{#1S}{\UDsecondoftwo{}}}%
            {\UDPassFirstBehindThirdToSecond{#1I}{\UDsecondoftwo{}}}%
            {%
              \expandafter\UDPassFirstBehindThirdToSecond\expandafter{%
                \romannumeral0\expandafter\UDExchange
                \romannumeral0\UDExtractFirstArgLoop{#2\UDSelDOm}{ #1}%
              }{\UDsecondoftwo{}}%
            }%
          }%
        }%
      }%
    }%
    {\zIteratorTailRecursiveLoop}%
  }%
}%
%%=============================================================================
%% Usage-Examples of \zIterator which show that 
%% - the result is delivered after two "hits" by \expandafter
%% - the mechanism is based on expansion only. No temporay assignments take
%%   place. Therefore \zIterator can also be used in expansion-contexts,
%%   e.g., within \csname..\endcsname,
%% - (unmatched) \if.. /\else/\fi in the argument do not disturb the mechanism.
%% - although the token \UDSelDOm is used as "sentinel-token" in some places,
%%   that token can occur within the argument, thus there are no forbidden
%%   tokens. (Except that you generally cannot use \outer tokens in
%%   macro-arguments.)
%% - the mechanism can cope with things being nested in curly braces.
%%=============================================================================

\expandafter\expandafter\expandafter\def
\expandafter\expandafter\expandafter\test
\expandafter\expandafter\expandafter{%
  \zIterator{A \TeX \is {\funny } {s sssi}i i \else \UDSelDOm {\fi } do ## not disturb me.}%
}

{\tt\meaning\test}%

\def\aSSbISSIIIISz{Yeah, that's it!}

\csname\zIterator{assbissiiiisz}\endcsname

\bye

enter image description here

| improve this answer | |
  • 2
    @bp2017 I tried to increase comprehensibility by splitting the code into sub-routines. Complexity depends on the requirements. E.g.: How shall the mechanism behave in case the argument contains unmatched \else or \fi ? Is it acceptable to have "forbidden tokens"? (E.g., with your code tokens whose meaning equals the meaning of \zEnd are forbidden within the argument of \zIterator.) What about curly braces? Should the thing also work in pure expansion-contexts, e.g., within \csname..\endcsname? – Ulrich Diez Nov 18 '19 at 21:39
  • 1
    @bp2017 About the termination-token when TeX gathers numbers digit-token-wise: I use the space-token because it gets silently discarded when terminating a number and thus won't disturb the look of the .pdf-output-file. TeX keeps expanding tokens when gathering numbers digit-token-wise. This is the gist of \romannumeral0-expansion: \romannumeral finds the digit 0, keeps on searching and hereby is tricked into doing a lot of expansion-work until finding a space. Thus the number 0 is found while \romannumeral silently swallows non-positive numbers, not delivering any token in return. – Ulrich Diez Nov 18 '19 at 21:49
  • 1
    @bp2017 TeXbook's chapter 24 "Summary of Vertical Mode" explains what a <number> is. Alone this appears to be amazingly complex at first glimpse. ;-) My experience is: Reading the TeXbook was kind of exhausting. Because I couldn't read it like a novel. You have to read it like an advocat who examines the meaning of each little phrase of a contract for pitfalls. ;-) When you have walked down that road for a while, it starts being fun. ;-) In case you need help looking through the code provided by me, don't hesitate to ask. – Ulrich Diez Nov 18 '19 at 22:17
2

Here is the example from my tokcycle package (https://ctan.org/pkg/tokcycle) that does this. The purpose of the package is to iterate through token lists.

Note that, in the MWE, it not only capitalizes after spaces, but ignores intervening catcode-12 tokens and the like, such as brackets, hyphens, and parens.

\documentclass{article}
\usepackage{tokcycle}
\newcommand\TitleCase[1]{%
  \def\capnext{T}
  \tokcycle
    {\addcytoks{\nextcap{##1}}}
    {\processtoks{##1}}
    {\addcytoks{##1}}
    {\addcytoks{##1\def\capnext{T}}}
    {#1}%
  \the\cytoks
}
\newcommand\nextcap[1]{%
  \edef\tmp{#1}%
  \tctestifx{-#1}{\def\capnext{T}}{}%
  \tctestifcon{\if T\capnext}%
    {\tctestifcon{\ifcat\tmp A}%
    {\uppercase{#1}\def\capnext{F}}%
    {#1}}%
  {#1}%
}
\begin{document}
\TitleCase{%
|here, {\bfseries\today{}, is [my]} really-big-test 
  (\textit{capitalizing} words).|\par
here, {\bfseries\today{}, is [my]} really-big-test 
  (\textit{capitalizing} words).
}
\end{document}

enter image description here

And here is a Plain-TeX version:

\input tokcycle.tex
\long\def\TitleCase#1{%
  \def\capnext{T}
  \tokcycle
    {\addcytoks{\nextcap{##1}}}
    {\processtoks{##1}}
    {\addcytoks{##1}}
    {\addcytoks{##1\def\capnext{T}}}
    {#1}%
  \the\cytoks
}
\long\def\nextcap#1{%
  \edef\tmp{#1}%
  \tctestifx{-#1}{\def\capnext{T}}{}%
  \tctestifcon{\if T\capnext}%
    {\tctestifcon{\ifcat\tmp A}%
    {\uppercase{#1}\def\capnext{F}}%
    {#1}}%
  {#1}%
}
\TitleCase{%
|here, {\bf today, is [my]} really-big-test 
  ({\it capitalizing} words).|\par
here, {\bf today, is [my]} really-big-test 
  ({\it capitalizing} words).
}
\bye

enter image description here

| improve this answer | |

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.