11

I take the following question for a moot thing.

Nonetheless it makes me sick that I don't see a good and robust solution myself. ;-)


Assume something like this:

\def\unfold{\umbrella My bald head always dries so slowly.}
\outer\def\umbrella{No raindrops on my head, please! }

Toplevel-expanding \unfold yields the following list of tokens:

\umbrella My bald head always dries so slowly.

\umbrella is defined in terms of \outer.
So it is not a good idea to pass the toplevel-expansion of \unfold as argument towards another macro.

How can you check, using methods only which

  • can also be applied in expansion-only contexts, e.g., with \csname..\endcsname-, \edef- or \write-expansion,
  • don't require some sort of forbidden sentinel-token,
  • also work when the \escapechar-parameter has unusual values,

whether an arbitrary token-sequence also contains \outer-tokens?

I am happy about solutions both in plain TeX and in LaTeX.
With solutions using Lua and/or expl3 I will probably need some time for retracing every step. ;-)

I am grateful for pointers towards the right direction.

5
  • 3
    the best answer is to build the format with \def\outer{} as soon as you have the catcodes of {} set, but I suppose that isn't the answer you want:-) Commented Feb 6, 2020 at 15:15
  • luatex solutions OK? Commented Feb 6, 2020 at 15:17
  • @DavidCarlisle I'm not that familiar with lua-extensions, but sure, why not. ;-) (As I said: This is a moot thing. So don't wate too much time on it.) Commented Feb 6, 2020 at 15:20
  • 1
    you could expendably iterate over the token list in Lua (I may post an answer if I do that:-) and check for outer tokens although actually simpler, in the tex side of luatex you can set \suppressoutererror = 1 and then \outer tokens do not generate an error. Commented Feb 6, 2020 at 15:26
  • 1
    I totally rewrote my [deleted] answer in a better way. Have a look. Commented Feb 8, 2020 at 20:59

5 Answers 5

13

What did I get myself into. . .

There are three possible ways:

  1. \def\outer{} and go on with your life. Hands down the best choice (in all aspects).
  2. I bet it's easier to implement \suppressoutererror in TeX than to write TeX code to do that.
  3. It's 2020, the year Knuth will address reported bugs in TeX. Submit \outer for consideration.
  4. I said three!
  5. I suppose if you are still reading you expect me to show some code. Read on, then :-)

The problem with \outer macros is that they are meant to be used only. You are not supposed to do things with them. Anything fancy you try to do, TeX will yell at you. This rules out one of TeX's most powerful features: macros. You simply can't use them on an \outer macro.

And then you want expandability, so you just ruled out most of TeX's primitives as well (including the most useful contender, \let, which can not only look at an \outer macro, but can also remove it). Not even ε-TeX is helpful here, as \detokenize and friends cannot have an \outer control sequence in them (so yes, the code below works in Knuth TeX).

This leaves you with minimal resources. Only one primitive might help you now: \meaning. The code below misuses \meaning to try to find out if there is an \outer control sequence in the argument. . .


What It can do:

There are two macros, \ifoutertl and \ifouterarg. The first one expands the argument token list once, to expose its contents, and the second requires braces (catcode 1 and 2) around the argument. In your example you'd use them like:

\ifoutertl\unfold{with}{without} outer
\ifouterarg{\umbrella My bald head always dries so slowly.}{with}{without} outer
\ifouterarg{My bald head always dries so slowly.}{with}{without} outer

and it would print: with outer with outer without outer.

How It works:

Abandon all hope, ye who enter here.

Basics

When you do:

\ifouterarg{This is \outer: \umbrella}{T}{F}

the code will start by removing the leading { and will hit the first token with \meaning:

\some@macro the letter This is \outer: \umbrella}{T}{F}

and then the \some@macro will do some checks for the \escapechar (essentially ignored in the process), then will look at the immediate next t and will try to find out what to do with it. A t will be either a letter (the letter <something>) or a character (the character <something>). The code then processes that and moves on, doing the same with the rest. Then the code arrives at \outer, and hits it with \meaning:

\some@macro \outer: \umbrella}{T}{F}

(\outer is a primitive, so \meaning\outer is \12o12u12t12e12r12). In this case, the code will see the \ and will do its thing with \outer[note 1]. Later, it arrives at \umbrella and hits it with \meaning:

\some@macro \outer macro:->No raindrops on my head, please! }{T}{F}

and this time it will see the sequence of character tokens \outer macro:->, in which case it will understand that an \outer control sequence was in the argument, and then a bunch of expansion steps later it will leave T as a result. If no \outer control sequence was found F would be left as result. This “bunch of expansion steps” means hitting all the tokens in No raindrops on my head, please! with \meaning[note 2] and removing them as described earlier.

Analysing each token (after \meaninging it)

After a token is hit with \meaning (remember, we cannot look at a token before hitting it with \meaning) the code proceeds to analyse it to find out what's to be done. We're interested in the prefix of a control sequence, so the very first thing the code looks for is for \protected \long and \outer. Lucky us, \outer is always the last prefix to appear in the list, so the code just skips the other two and keeps looking for more. If it finds \outer, we look for \outer macro:->, which is guaranteed to be there[note 3].

In case the token is not a macro (thus no macro:-> text), then we keep looking if it's one of TeX's 10 character tokens (with catcode 1, 2, 3, 4, 6, 7, 8, 10, 11, or 12). If the token is one of these, then the code simply removes it and proceeds scanning.

If it's not (for instance, a primitive, whose meaning is itself, a \count register, whose meaning is \count<number>, and a couple others), then the code gives up and starts again[note 4] (it's not an \outer control sequence anyway, so we don't care). This time, it will start by doing \meaning again (for example, in \count123) and then the first token will be a the character \, and it will find its way out.

Nesting

I'm not sure you noticed, but never in the process we told the macro when to end.
It is doing \meaning on things and looking at it. So here comes the two exceptions in the process above: If the token was a begin-group character, then the code starts a (sort of) new level of expansion, which will end at the next end-group character. Any \outer control sequence found in there will be reported to the upper expansion level, so you can nest braces to your heart's content :-)
If braces are balanced and an end-group character is found, the end of the token list is reached, and it stops, selecting the proper conditional branch.

To keep track of nesting levels and whether an \outer control sequence was found, the macro that commands the scanning process does something like (pseudocode):

\romannumeral
  \expandafter \after_egroup_action
  \expandafter \outer_found_boolean
    \romannumeral \scan

when \scan finds another begin-group character, a new layer (\action\boolean\scan) is placed, and the code keeps scanning. When an end-group character is found, an \exp_end: \ud@exp@end is inserted stopping \romannumeral, and the control passes to the \after_egroup_action macro, which will either: keep scanning the token list with the \outer_found_boolean, or will end the process and use \outer_found_boolean to pick the proper conditional branch.

Notes:

  • 1: Since the code cannot look at the actual control sequence, the process is not completely robust. Suppose here:
    %                  V------V
    \ifouterarg{\outer macro:->}{T}{F}
    
    if you managed to change the catcode of the marked tokens to 12, then yes, you managed to fool the code into thinking that you had an \outer control sequence. I doubt it is possible to overcome this problem: we can't examine \outer (and find out it's the primitive \outer) beforehand without the risk of grabbing an \outer control sequence. And once we hit it with \meaning, it is indistinguishable from an \outer macro. So yes, it's not fool proof, sorry.
  • 2: Yes, the code is slow. Awfully slow. It has to hit each token with \meaning, so if you have \def\a{\b\b\b\b. . . lots . . .\b\b\b\b\b} and \def\b{<something awfully long>}, then yes, it will do \meaning in every \b, and then \meaning in every token in \b, which might escalate quickly. Again, I doubt this can be optimised. Don't take it wrong, though: the code will not expand forever. Any macro in the argument is expanded with \meaning, however everything else are characters, which will end up being scanned and removed.

  • 3: That is, if the \outer macro:-> comes from the \meaning of an \outer control sequence. If you take the example in note 1 and make it read \ifouterarg{\outer macro:>}{T}{F} instead, then the code will expand to F.

  • 4: Yes, the code could be optimised to know more primitive tokens other than the character tokens, so that (taking the example of \count123) instead of consuming each of \, c, o, . . . one by one (with \meaning and a large loop), it would see that it's \count<something> and take a shortcut. Implementing this is left as an exercise for the reader ;-)

The code is probably not what can be called robust, but it gets the job done. At this point I'm not really sure there is a better way to do this. Though much of it was written while I was changing my mind about how it would work, so there are probably redundancies and it could be improved a bit for speed. Not much, though, I think. Proceed with caution!

And if you managed to read all the way to here, congrats! Here's the code:

\catcode`\@=11
% Utilities
\def\@empty{}
\long\def\@gobble#1{}
\long\def\@firstoftwo#1#2{#1}
\long\def\@secondoftwo#1#2{#2}
\long\def\ud@usetwo#1#2{#1#2}
\def\ud@zap@space#1{\ud@@zap@space#1 \@empty}
\def\ud@@zap@space#1 #2{#1%
  \ifx#2\@empty\else\expandafter\ud@@zap@space\fi#2}
\ud@usetwo{\let\ud@sptoken= }{ }
\chardef\ud@exp@end=0
\chardef\ud@false=0
\chardef\ud@true=1
% User-level macros
\def\ifoutertl#1{%
  \romannumeral
  \iffalse{\fi\ud@ifouter\ud@false\ud@ifouter@TF#1}}
\def\ifouterarg{%
  \romannumeral
  \expandafter\expandafter\expandafter\ud@ifouterarg@aux
  \expandafter\@gobble\string}
% Internal macros
\def\ud@ifouter#1#2#3{%
  \expandafter\ud@if@outer@scan\expandafter#1\expandafter#2#3}
\def\ud@ifouterarg@aux{%
  \ud@ifouter\ud@false\ud@ifouter@TF\@empty}
\def\ud@ifouter@TF#1{%
  \ifodd#1%
    \expandafter\expandafter\expandafter
    \ud@exp@end\expandafter\@firstoftwo
  \else
    \expandafter\expandafter\expandafter
    \ud@exp@end\expandafter\@secondoftwo
  \fi}
\def\ud@if@outer@scan#1#2{%
  \expandafter\ud@if@outer@decypher
    \expandafter#1\expandafter#2\meaning}
\def\ud@if@outer@decypher#1#2{%
  \expandafter\ud@rearrange
    \expandafter#1\expandafter#2%
      \romannumeral\ud@if@outer@escapechar}
\def\ud@if@outer@escapechar#1{%
  \ifnum\ifnum\escapechar<0   0\else 1\fi
        \ifnum\escapechar>255 0\else 1\fi=0
    \ud@decypher@noescape
  \else
    \ifnum`#1=\escapechar
      \ud@decypher@escape
    \else
      \ud@decypher@noescape
    \fi
  \fi#1}
\def\ud@rearrange#1#2#3{#3#1#2}
\def\ud@decypher@noescape#1\fi\fi{\fi\fi\ud@decypher@cs@prefix}
\def\ud@decypher@escape\else#1\fi\fi#2{\fi\fi
  \ud@decypher@cs@prefix}
\def\ud@decypher@cs@prefix#1{%
  \ifcase0\if #1p1\fi \if #1l2\fi
          \if #1o3\fi \if #1m4\fi \ud@sptoken
      \expandafter\ud@scan@token@keyword%
  \or \expandafter\ud@scan@string@p % \protected
  \or \expandafter\ud@scan@string@l % \long
  \or \expandafter\ud@scan@string@o % \outer
  \or \expandafter\ud@scan@string@m % macro :->
  \fi#1}
\def\ud@return@same@scanner{\ud@exp@end\ud@if@outer@scan}
\def\ud@return@true@scanner{\ud@exp@end\ud@return@true@outer}
\def\ud@return@true@outer#1{\ud@if@outer@scan\ud@true}
\def\ud@newstring#1#2{%
  \ifx\relax#2%
    \expandafter\@gobble
  \else
    \edef\ud@tmp@tl{\ud@tmp@tl#1}%
    \edef\ud@test@tokn{\string#1}%
    \expandafter\edef\csname ud@scan@string@\ud@tmp@tl\endcsname##1{%
      \noexpand\ifx ##1\ud@test@tokn
        \noexpand\expandafter\expandafter\noexpand
          \csname ud@scan@string@\ud@tmp@tl#2\endcsname
      \noexpand\else
        \noexpand\expandafter\noexpand\ud@return@same@scanner
      \noexpand\fi}%
    \expandafter\ud@newstring
  \fi{#2}}
\def\ud@new@scan@string#1{%
  \def\ud@tmp@tl{}%
  \ud@newstring #1{end}\relax
  \expandafter\def\csname ud@scan@string@\ud@zap@space{#1}end\endcsname}
\ud@new@scan@string{protected}{\ud@if@outer@escapechar}
\ud@new@scan@string{long}{\ud@if@outer@escapechar}
\ud@new@scan@string{macro:->}{\ud@return@same@scanner}
\ud@new@scan@string{outer macro:->}{\ud@return@true@scanner}
\def\ud@scan@bgroup{\ud@exp@end\ud@scan@bgroup@aux}
\def\ud@scan@bgroup@aux#1#2{%
  \expandafter\ud@after@group@continue
    \expandafter#1\expandafter#2%
      \romannumeral\ud@if@outer@scan#1\ud@after@egroup}
\def\ud@after@egroup{\ud@exp@end}
\def\ud@after@group@continue#1#2#3{%
  \ud@if@outer@scan#3#2}
\def\ud@scan@egroup{\ud@exp@end\ud@scan@egroup@aux}
\def\ud@scan@egroup@aux#1#2{#2#1}
\def\ud@gobble@char@return#1{\ud@return@same@scanner}
\def\ud@gobble@char@do#1#2{#1}
\ud@usetwo{\def\ud@gobble@two@spaces}{ } {}
\ud@new@scan@string{begin-group character}{\ud@gobble@char@do\ud@scan@bgroup}
\ud@new@scan@string{end-group character}{\ud@gobble@char@do\ud@scan@egroup}
\ud@new@scan@string{math shift character}{\ud@gobble@char@return}
\ud@new@scan@string{alignment tab character}{\ud@gobble@char@return}
\ud@new@scan@string{macro parameter character}{\ud@gobble@char@return}
\ud@new@scan@string{superscript character}{\ud@gobble@char@return}
\ud@new@scan@string{subscript character}{\ud@gobble@char@return}
\ud@new@scan@string{blank space}{\expandafter\ud@return@same@scanner\ud@gobble@two@spaces}
\ud@new@scan@string{the letter}{\ud@gobble@char@return}
\ud@new@scan@string{thec haracter}{\ud@gobble@char@return}
\def\ud@scan@token@keyword#1{%
  \expandafter\ifx\csname ud@scan@string@#1\endcsname\relax
    \expandafter\ud@return@same@scanner
  \else
    \csname ud@scan@string@#1\expandafter\endcsname
  \fi#1}
% keyword forks
\def\ud@strip@prefix#1>{}
\def\ud@detokenize#1#2{\def#1{#2}\edef#1{\expandafter\ud@strip@prefix\meaning#1}}
\def\ud@tl@head{\expandafter\@firstoftwo}
\def\ud@set@fork@string#1#2#3{%
  \begingroup \escapechar-1
    \def\x{ud@scan@string@#1}%
    \expandafter\ud@set@fork@string@aux
      \csname\x#2\expandafter\endcsname
      \csname\x#2_\expandafter\endcsname
      \csname\x#3\expandafter\expandafter\expandafter\endcsname
      \expandafter\string\csname#2\expandafter\expandafter\expandafter\endcsname
      \expandafter\string\csname#3\endcsname}
\def\ud@set@fork@string@aux#1#2#3#4#5{%
  \endgroup
  \let#2#1%
  \def#1##1{%
    \ifx ##1#4\expandafter#2%
    \else \ifx ##1#5\expandafter\expandafter\expandafter#3%
          \else \expandafter\expandafter\expandafter\ud@return@same@scanner
          \fi
    \fi##1}}
\ud@set@fork@string{ma}{c}{t} % macro / math
\ud@set@fork@string{macro}{p}{:} % macro parameter / macro:->
\ud@set@fork@string{the}{c}{l} % the character / the letter
\ud@set@fork@string{b}{l}{e} % blank / begin
\catcode`\@=12

% -----
% Tests
% -----

\def\rain{My bald head is still wet.}
\def\unfold{\umbrella My bald {head always} dries so slowly.}
\outer\def\umbrella{No raindrops on my head, please! }
\newcount\abc
\tt
0\ifouterarg{\newcount}{T}{F} (T)\par
1\ifouterarg{\abc}{T}{F} (F)\par
2\ifouterarg{\zzz}{T}{F} (F)\par
3\ifoutertl\unfold{T}{F} (T)\par
4\ifoutertl\rain{T}{F} (F)\par
5\ifouterarg{No raindrops on my head, please! }{T}{F} (F)\par
6\ifouterarg{\umbrella My bald {head always} dries so slowly.}{T}{F} (T)\par
7\edef\tmpa{\ifoutertl\unfold{T}{F}}\meaning\tmpa (T)\par
8\edef\tmpa{\ifouterarg{\umbrella corp.}{T}{F}}\meaning\tmpa (T)\par
9\edef\tmpa{\ifouterarg{\zombies!}{T}{F}}\meaning\tmpa (F)\par
\bye
7
  • Remark, in the "not completely robust" version you need a space before the "macro:->" as well.
    – user202729
    Commented Jul 5, 2022 at 12:51
  • Actually I think it's possible to overcome the \meaning\outer = \outer problem, same way how \__tl_analysis i.e. by doubling the token list, remember info from the first pass and use it in the second pass. Would be terrible to implement though.
    – user202729
    Commented Jul 5, 2022 at 12:54
  • Bug • if there's weird space the code doesn't work • if there's a (catcode other, charcode space) token immediately followed by a outer macro the code doesn't work either.
    – user202729
    Commented Jul 5, 2022 at 13:05
  • @user202729 Thanks for the comments. I'll look into that... eventually :) (after two years I have no clue what this code does anymore :) Commented Jul 5, 2022 at 13:47
  • More bug • if it starts with \outer\umbrellathen the part that check for "\outer macro:" will grab the umbrella → error. • if \escapechar=-1 then you can't distinguish between "end-group character }" and "⟨\end⟩-group character }" where the ⟨\end⟩ is the primitive. (it's difficult...)
    – user202729
    Commented Jul 5, 2022 at 17:20
8

Let's try a Lua solution. We could use \suppressoutererror to make scanning easier but that wouldn't be much fun, so we scan individual tokens instead and manually try to keep track of nested braces.

For every scanned token, Lua can access the "command id". This is kind of a generalization of catcodes. Especially, every catcode 1 ({) token has id 1, every catcode 2 (}) token has id 2 and every token which would invoke a \outer macro has he id returned by token.command_id'outer_call' or token.command_id'long_outer_call'. So for every token we only have to check if it has any of these command ids. For id 1 we increase the nesting level, for 2 we decrease it and if one of the other two ids are found we remember to return true at the end:

\documentclass{article}
\begin{document}
\directlua{
  local i = luatexbase.new_luafunction'hasouter'
  % The following creates a table outer_cmd, such that
  % outer_cmds[i] is true iff i is a id corresponding to
  % a call to an \outer macro 
  local outer_cmds = {
    [token.command_id'outer_call'] = true,
    [token.command_id'long_outer_call'] = true,
  }
  lua.get_functions_table()[i] = function() % This function will be executed if we use `\hasouter`
    local tok = token.scan_token() % scan_token applies full expansion until the first non-expandable token is found. This allows e.g. \hasouter\expandafter{...}
    local cmd = tok.command % Look at the command code
    if cmd \csstring\~= 1 then % \csstring\ must makes sure that TeX does not expand ~.
      token.put_next(firsttok) % If we read a wrong character, putting it back ensures that TeX gets less confused if the user decides to continue after the error.
      error[[Argument must start with \csstring\{]]
    end
    local nesting = 0
    local result = false % This will become true if we find an \outer call
    while true do % An endless loop. This will still terminate because we return early if nesing becomes 0 again
      if cmd == 1 then % tok is equivalent to `{`. Increase the nesting level.
        nesting = nesting + 1
      elseif cmd == 2 then % tok is equivalent to `}`. Decrease the nesting level.
        nesting = nesting - 1
        if nesting == 0 then
          % We want to expand to the first or second parameter depending on result, so we insert @first/secondoftwo
          token.put_next(token.create(result and '@firstoftwo' or '@secondoftwo'))
          return
        end
      else
        result = result or outer_cmds[cmd] % If result is already true, don't change anything. Otherwise make it true if cmd corresponds to an outer call
      end
      tok = token.get_next() % Continue with the next token. get_next applies no expansion.
      cmd = tok.command
    end
  end
  token.set_lua('hasouter', i) % Define \hasouter to execute the function above
}

\def\unfold{\umbrella My bald head always dries so slowly.}
\def\unfoldX{My bald head always dries so slowly.}
\outer\def\umbrella{No raindrops on my head, please! }
\hasouter\expandafter{\unfold}{with}{without} outer
\hasouter\expandafter{\unfoldX}{with}{without} outer
\hasouter{\umbrella My bald head always dries so slowly.}{with}{without} outer
\hasouter{My bald head always dries so slowly.}{with}{without} outer
\end{document}

As Phelype Oleinik mentioned in a comment, this does not actually work inside \edef because there get_next enforces the restriction that no outer macros should be contained in an macro.

In modern LuaTeX versions (e.g. starting with TeXLive 2020 or when compiled with lualatex-dev in TeXLive 2019), this can be worked around by using tex.runtoks: Executing \lowercase{} or something similar doesn't do anything, but it returns TeX's scanner to a normal state where \outer macros are accepted. Of course \ŀowercase isn't expandable, but runtoks allows to use non-expandable things in an expandable context.

\documentclass{article}
\begin{document}
\directlua{
  local i = luatexbase.new_luafunction'realhasouter'
  local j = luatexbase.new_luafunction'hasouter'
  local outer_cmds = {
    [token.command_id'outer_call'] = true,
    [token.command_id'long_outer_call'] = true,
  }
  lua.get_functions_table()[i] = function()
    local delayed_tok = token.get_next()
    local tok = token.scan_token()
    local cmd = tok.command
    if cmd \csstring\~= 1 then
      token.put_next(firsttok)
      error[[Argument must start with \csstring\{]]
    end
    local nesting = 0
    local result = false
    while true do
      if cmd == 1 then
        nesting = nesting + 1
      elseif cmd == 2 then
        nesting = nesting - 1
        if nesting == 0 then
          token.put_next(token.create(result and '@firstoftwo' or '@secondoftwo'))
          token.put_next(delayed_tok)
          return
        end
      else
        result = result or outer_cmds[cmd]
      end
      tok = token.get_next()
      cmd = tok.command
    end
  end
  local call_realhasouter_toks = {
    token.new(0, token.command_id'case_shift'), token.new(0, 1), token.new(0, 2), % This is similar to \lowercase{}. It doesn't do anything, but it changes the status of TeX's scanner to allow outer tokens
    token.new(i, token.command_id'lua_expandable_call')
  }
  lua.get_functions_table()[j] = function()
    tex.runtoks(function()
      token.put_next(call_realhasouter_toks)
    end)
  end
  token.set_lua('hasouter', j)
}
\def\unfold{\umbrella My bald head always dries so slowly.}
\def\unfoldX{My bald head always dries so slowly.}
\outer\def\umbrella{No raindrops on my head, please! }
\hasouter\expandafter{\unfold}{with}{without} outer
\hasouter\expandafter{\unfoldX}{with}{without} outer
\hasouter{\umbrella My bald head always dries so slowly.}{with}{without} outer
\hasouter{My bald head always dries so slowly.}{with}{without} outer
\edef\xxx{\hasouter{My bald head always dries so slowly.}{with}{without} outer}
\show\xxx
\edef\xxx{\hasouter{\umbrella My bald head always dries so slowly.}{with}{without} outer}
\show\xxx
\end{document}
5
  • What is the difference between an expandable TeX macro and an “expandable luacall” that the latter cannot appear in an \edef? (talking about the fourth example in your answer here:) \edef expands as it sees the tokens, so \hasouter should expand to \@firstoftwo before \edef has the chance to see \umbrella... which doesn't seem to be the case. Commented Feb 8, 2020 at 21:12
  • @PhelypeOleinik Interesting. This seems to be a bug: get_next doesn't change the status of TeX's scanner, so if TeX currently is in a context where outer isn't allowed, it also isn't allowed in get_next. If I write it like this it also could be considered a feature... Commented Feb 8, 2020 at 23:39
  • 1
    Ah, of course, with Lua you can use get_next to look at the token: you're not restrained to the expansion methods of Mortal Men :-) It makes sense now, thanks! Though I think it would make sense to change scanner_status in the scope of \hasouter (to normal, I think...) Commented Feb 8, 2020 at 23:59
  • @PhelypeOleinik I added a workaround Commented Feb 8, 2020 at 23:59
  • @PhelypeOleinik I agree that it would make sense to change it. Actually there is already code to reset the scanner_status afterwards, so the only necessary change would be adding scanner_status = normal. Commented Feb 9, 2020 at 0:01
5

This introduces \countouters{<macro>}, which will count the occurrences of \outer macros contained in the argument. The result is stored in the counter outercnt. I had to make an adjustment to the logic of tokcycle token-digesting package (inserting an extra trapping level to look for pre-digested \outer macros) to accomplish this.

The macro to be tested can itself contain macros and groups...however, \outer macros are not allowed within groups inside the defined macro. So, for example in the MWE below, the macro \unfold could not be defined as \def\unfold{\textit{\umbrella}}, if \umbrella were later defined as an outer macro.

\documentclass{article}
\usepackage{listofitems,tokcycle}
\newcounter{outercnt}

\makeatletter
\let\detect@CANTabsorbA\detect@CANTabsorb
\long\def\detect@CANTabsorb{%
  \expandafter\def\expandafter\mytmp\expandafter{\meaning\tc@next}%
  \expandafter\setsepchar\expandafter{\detokenize{\outer macro:->}}%
  \readlist\mylist{\mytmp}%
  \tctestifnum{\listlen\mylist[]>1}%
    {\stepcounter{outercnt}\expandafter\@tokcycle\string}%
    {\detect@CANTabsorbA}%
}
\makeatother

\tokcycleenvironment\countouterenv{}{\processtoks{##1}}{}{}

\newcommand\countouters[1]{\setcounter{outercnt}{0}%
  \expandafter\countouterenv#1\endcountouterenv}

\begin{document}
\def\unfold{1. \umbrella My bald head \textit{always} dries so slowly.
  My \umbrella}
\outer\def\umbrella{No raindrops on my head, please! }

\countouters{\unfold} 

Outer occurences = \theoutercnt 
\end{document}

enter image description here

2
  • 1
    If I put \outer in the argument the code counts it (which it should not). Changing the \setsepchar line to \expandafter\setsepchar\expandafter{\detokenize{\outer macro:->}} seems to be a safer test Commented Feb 8, 2020 at 21:17
  • 1
    Thank you @PhelypeOleinik. Good suggestion. Commented Feb 8, 2020 at 21:24
2

As Phelype Oleinik, I am thinking that \meaning is a lead for this task. But, in my way, there is no checking all characters in \meaning result still only searching macro: sequence and following analysis previous characters for looking \outer. Perhaps, this solution is not so complete but it still work.

\outertest implements basic work. It analyses getting control sequence that may content of token list. As result, it sets three conditions: \ifoutermacro for itself control sequence checking, \ifouterin for its top-level expansion and auxiliary \ifmacro to distinguish between macro and primitive. There are two stages in its working process:

  1. Checking for getting control sequence being outer macro. For preserve against using argument \futurelet syntax is used. Macro \@ut...\@nd@ut puts text before macro: sequence in \meaning output if it exists (or full \meaning output if not) in \p@fix macro. If control sequence is a macro its expansion (after macro:) is saved if \s@fix macro, else \s@fix is empty. Then it is compare with \outer or \long\outer sequences. If match is find, \outermacrotrue is set and no more analysis of getting control sequence is carried out.
  2. If getting control sequence is not outer, checking is carried out whether that is a macro or a primitive. In first case, its expansion is checked out token by token in same manner. Solution for \@outertokstest scanning command is based on the example from TeXbook Appendix D. When first outer macro have been find in expansion \outertokstest sets \outerintrue and stops its work.

Getting results for \ifoutermacro, \ifouterin and \ifmacro might be used in following implementation as in \isouter macro.

\catcode`\@=11
\newif\ifouterin
\newif\ifoutermacro
\newif\ifmacro
% Token list for \outer in catcodes 12
\edef\outertoks{\string\outer\space}
% Token list for \long\outer in catcodes 12
\edef\longoutertoks{\string\long\outertoks}

{% Defining command for reading \meaning output
    \let\tp=\catcode % temporary \catcode
    \tp`\m=12 \tp`\a=12 \tp`\c=12
    \tp`\r=12 \tp`\o=12
    \gdef\@ut#1macro:#2\@nd@ut{\def\p@fix{#1}\def\s@fix{#2}}
    \gdef\t@il{macro:} % tail for no-macro meanings
}

\def\outertest{\futurelet\m@cro\m@crotest }
\def\m@crotest{\outermacrofalse\outerinfalse\macrofalse
    \edef\t@stmeaning{\meaning\m@cro\t@il}%
    \expandafter\@ut\t@stmeaning\@nd@ut
    \ifx\p@fix\outertoks\outermacrotrue\macrotrue\else
        \ifx\p@fix\longoutertoks\outermacrotrue\macrotrue\else
            \outermacrofalse
            \ifx\s@fix\empty\relax\else\macrotrue\@outertokstest\fi
        \fi
    \fi
    \n@xtlet
}
\def\n@xtlet{\let\n@@xt= } % gobble the token without using

% \@outertokstest is based on example from Appendix D from TeXbook
\def\@outertokstest{\expandafter\@test\m@cro\closetest}
\def\@test{\afterassignment\@@test\let\checkt@k= }
\def\n@test#1\closetest{}
\def\@@test{\ifx\checkt@k\closetest \let\n@xt\relax\else
        \edef\t@stmeaning{\meaning\checkt@k\t@il}%
        \expandafter\@ut\t@stmeaning\@nd@ut
        \ifx\p@fix\outertoks\outerintrue \let\n@xt\n@test\else
            \ifx\p@fix\longoutertoks\outerintrue \let\n@xt\n@test\else
                \outerinfalse \let\n@xt\@test
            \fi
        \fi
    \fi
    \n@xt
}

\outer\def\isouter{\futurelet\m@cr@\t@stresults }
\def\t@stresults{%
    \outertest\m@cr@
    \ifoutermacro outer macro.\else
        \ifmacro
            not outer macro
            \ifouterin but contains \else
                and contains no
            \fi
            outer macro in expansion.
        \else
            not a macro.
        \fi
    \fi
    \n@xtlet
}

\catcode`\@=12

% -- Tests --

\def\unfold{\umbrella My bald head always dries so slowly.}
\outer\def\umbrella{No raindrops on my head, please! }

\tt
1. \string\relax\ is \isouter\relax \par
2. \string\newif\ is \isouter\newif \par
3. \string\ifouterin\ is \isouter\ifouterin \par
4. \string\empty\ is \isouter\empty \par
5. \string\isouter\ is \isouter\isouter \par
6. \string\unfold\ is \isouter\unfold \par
7. \string\outertoks\ is \isouter\outertoks \par
8. \string\umbrella\ is \isouter\umbrella \par
9. \string\undef\ is \isouter\undef \par

\bye
2

I notice some issues with Phelype Oleinik's answer, so I try to find out a better solution.

There's an implementation below. However absolutely do not try to reverse engineer what the code does. Read the English description instead, or see other options below.

Stage 0. Preparation

How to take a detokenized token from the input stream

(token equal to its own \string, i.e. either explicit space or non-space with other catcode)

(handled by the remove_next_stringified_character function.)

Apply \meaning to that token, then the following content is either blank space or the character ⟨non-space token⟩.

An alternative (which should be faster), if e-TeX is available, is to expand \unless \if \relax ⟨the character to be removed⟩ once, then expand \fi.

Expand the following tokens in the input stream while holding data

(implemented into the compiler engine)

This part is just standard tricks. If we hold a number instead there are 2 ways

  • note that \number ⟨some number⟩ \expandafter ⟨explicit space token⟩ ⟨some other tokens⟩ expands in 1 step to ⟨some number⟩ ⟨once expansion of these other tokens⟩.
  • wrap the number into a \csname, do the necessary expansion, then \cs_to_str:N it back. (issue: pollute the hash table)
  • write some code to insert one \expandafter token before each token (it's easy because it's just pure numbers), then do an expansion. (okay this is the third way.)

Stage 1. Handle the input size

(implemented in the count_string_len function)

Background

This algorithm is multiple-pass, it needs to take the container macro as input.

First, there's a little problem with \meaning.

Let's say we know \escapechar=-1 and the following token is either...

  1. char c cat other
  2. the token \the, followed by either c or some outer token
  3. some outer token
  4. char e cat }
  5. the primitive \end

The only way(*) to proceed in this situation is to \meaning the next token, but then the following token might be...

  • end-group character e comes from case 4 or 5 above
  • the character c comes from case 1 or 2 above
  • the ⟨some outer token⟩ comes from case 2 above
  • end-⟨some outer token⟩ comes from case 4 or 5 above

The first 2 bullet points make it hard enough to distinguish already, then there's still the possibility that some following token might be outer so we need to \meaning it again, etc..

(*) (I believe it's possible to apply \noexpand before applying \meaning. That approach could work as well, but it appears to be more complex.)

Side note, LuaTeX appears to have some bug where the \meaning of a \noexpand token is [unknown command code! (0, 1)] instead of the usual \relax. So the following description assumes the meaning of a noexpand token is ⟨escape character⟩relax.

We want to use \string here, as repeatedly apply \string to something will not increase its size unbounded i.e. \string is idempotent.

Approach

We will compute the total length of the stringified representation of all tokens.

First, put ⟨token list⟩A in the input stream, then repeatedly

  • stringify the next token. The stringfication must be nonempty.
  • absorb one stringified token.

This will eventually absorb the whole token list.

Let's say after 50 characters, we hit A as the 51th character. There are two possibilities

  • either we reached the end of the ⟨token list⟩, in that case we report the result is 50,
  • or there's some A within the stringfication of some token in the ⟨token list⟩.

To double check, we put ⟨token list⟩B in the input stream (note that if the second case is true, there will remain some garbage in the input stream which we will clean up later), then stringify-grab exactly 50 tokens, stringify the next one, then check if it's A or B.

If it's A, We have not reached the end of the token list yet, so we recursively call the function knowing that the length is at least 51 characters long.

The inner function would initially absorb 51 characters, before checking for any A or B; after some steps it should finally return the result.

Assuming it's determined that the actual length is 100, then the outer function would need to clean up about 50 remaining tokens after stringifying them. (the exact number is left as an exercise for the implementor.)

Stage 2. Apply \meaning to one token

(also handled by count_string_len, by passing suitable preprocess values this can be done)

Let's say, at this point we know that the concatenated stringfication of all tokens in the ⟨token list⟩ has length 100.

We iterate from 0 to 99, for each number X:

  • we would stringify/remove first X tokens, apply \meaning to the following token (without \stringify it), then use the A/B flip idea above to grab the remaining tokens.

    That way, we can compute the total length of the stringfication of all tokens, minus the length of the token that is generated after first X stringify/remove, plus the length of its \meaning.

  • Do the same as the above, but instead of \meaning we apply \noexpand followed by \meaning.

    Note that, in this way there are 2 important properties

    • only N-type tokens will have the length changed (the meaning of {, }, are unchanged under \noexpand)
    • all outer tokens will have the length changed (because outer macro is strictly longer than \relax, regardless of the value of \escapechar)

Stage 3. Compute the result

Given the above, while iterating over values X, after stringify/removing first X tokens, we can simply \noexpand the next token, grab it as an argument, then it's not difficult to process it to determine whether or not it's \outer.

When I said "simply" here, I mean you need to do the following...

  • first check if the string "macro:" appear in the meaning.
  • if it is, then check if "mark" appear before the first occurrence of "macro".
  • if it isn't, then check if "outer" appear before the first occurrence of "macro".

Reference: Every possible \meaning that a token can have

We also know the total stringfication length, so if we "copy" instead of absorb the token, we can continue stringifying/grabbing N-X remaining tokens after checking.

Notes

  • Instead of \numexpr, this code stores "numbers" in unary. Overall the time complexity is the same anyway (this method might even be faster because of various reasons).
  • As far as I can see, this method is completely robust. There's only minimal test suite below, but see the tests in the source file linked below.

Alternative approach

If you have \unexpanded available, simply do the normal "tail recursion" thing to process the token list and \unexpanded the argument after every step.

This requires the arguments, if given as a token list instead of the container macro, to be initially wrapped in two layers of braces (one for \unexpanded, the other for absorbing the token list).

Actually the alternative approach won't work, \unexpanded doesn't accept outer tokens in its input as well. The main approach does work though.


Actual implementation here.

%! TEX program = pdflatex
\documentclass{article}
\usepackage{filecontentsdef}
\begin{document}
\ExplSyntaxOn

% ======== some auxiliary macros ========
\def\__process_char #1 #2 {
    %\prettye:n{\expandafter \expandafter \expandafter \noexpand \char_generate:nn {`#2} {"#1}}
    \expandafter \expandafter \expandafter \noexpand \char_generate:nn {`#2} {"#1}
    \__process_s
}

\def\__process_space_other_cat #1 {
    \expandafter \expandafter \expandafter \noexpand \char_generate:nn {32} {"#1}
    \__process_s
}

\def\__process_cs #1 / {
    \expandafter \noexpand \csname #1 \endcsname
    \__process_s
}

\def\__process_s#1{
    \token_if_eq_charcode:NNTF #1 0 { % 0 <name> / → the control sequence
        \__process_cs
    } {
        \token_if_eq_charcode:NNTF #1 s { ~   \__process_s
        } {
            \token_if_eq_charcode:NNTF #1 S { % S <cat> → a space
                \__process_space_other_cat
            } {
                \token_if_eq_charcode:NNF #1 . { % . → end
                    \__process_char #1
                }
            }
        }
    }
}

% main handler function, will exec the resulting token list.
\def\__process_all#1{
    \begingroup \exp_last_unbraced:Nx \endgroup {\__process_s #1}
}
\ExplSyntaxOff
\begin{filecontentsdefmacro}{\data}
0def/0ifescapenull/1{0texconditional/1{0ifnum/C0C>0escapechar/2}2}0def/0remove_next_stringified_character/1{0expandafter/0stzz31/0meaning/2}0def/0stzz31/6#6#C11{0if/Bb6#6#C10use_ii_to_i:w/0fi/0use_ii:nn/0stzz33/0stzz35/2}0def/0stzz35/ChCesCcChCaCrCaCcCtCeCrs6#6#C11{0exp_end:/2}0def/0stzz33/ClCaCnCksCsCpCaCcCess1{0exp_end:/2}0def/0stringify_until_A/1{0expandafter/0stzz56/0string/2}0def/0stzz56/1{0expandafter/0stzz57/0meaning/2}0def/0stzz57/6#6#C11{0ifx/6#6#C1Cb0use_ii_to_i:w/0fi/0use_ii:nn/0stzz59/0stzz61/2}0def/0stzz61/ChCesCcChCaCrCaCcCtCeCrs6#6#C11{0ifx/6#6#C1CA0use_ii_to_i:w/0fi/0use_ii:nn/0stzz63/0stzz67/2}0def/0stzz63/1{0exp_end:/BA2}0def/0stzz59/ClCaCnCksCsCpCaCcCess1{0expandafter/0exp_end:/0expandafter/C.0romannumeral/0stringify_until_A/2}0def/0stzz67/1{0expandafter/0exp_end:/0expandafter/C.0romannumeral/0stringify_until_A/2}0def/0replicate_helper/6#6#C11{0stzz80/6#6#C12}0def/0stzz80/6#6#C16#6#C21{0stzz83/6#6#C26#6#C12}0def/0stzz83/6#6#C16#6#C21{0ifx/6#6#C10relax/0use_ii_to_i:w/0fi/0use_ii:nn/0stzz85/1{0stzz88/6#6#C22}2}0def/0stzz88/6#6#C11{0expandafter/0stzz103/0expandafter/6#6#C10exp:w/0replicate_helper/6#6#C12}0def/0stzz85/1{0exp_end:/0relax/2}0def/0replicate/6#6#C16#6#C21{0expandafter/0stzz102/0exp:w/0replicate_helper/6#6#C26#6#C10relax/2}0def/0stzz102/6#6#C10relax/1{0exp_end:/6#6#C12}0def/0stzz103/6#6#C11{0exp_end:/6#6#C12}0def/0stringify_remove_next_repeated/1{0expandafter/0stzz125/0exp:w/2}0def/0stzz125/1{0expandafter/0remove_next_stringified_character/0string/2}0def/0stringify_remove_next_repeated_count/6#6#C11{0expandafter/0stzz149/0expandafter/1{0exp:w/0replicate/1{6#6#C12}0stringify_remove_next_repeated/2}2}0def/0remove_next_repeated/1{0expandafter/0stzz143/0exp:w/2}0def/0stzz143/6#6#C11{0exp_end:/2}0def/0remove_next_repeated_count/6#6#C11{0expandafter/0stzz149/0expandafter/1{0exp:w/0replicate/1{6#6#C12}0remove_next_repeated/2}2}0def/0stzz149/6#6#C11{0expandafter/0exp_end:/0romannumeral/6#6#C10exp_end:/2}0def/0exp_forward/1{0expandafter/0stzz157/0exp:w/2}0def/0stzz157/1{0exp_end:/0exp_forward/2}0def/0exp_forward_end/1{0expandafter/0stzz164/2}0def/0stzz164/1{0exp_end:/0exp_forward_end/2}0def/0r_function_do_nothing/1{0exp_end:/2}0def/0r_function_meaning_noexpand/1{0expandafter/0r_function_meaning/0noexpand/2}0def/0r_function_meaning/1{0expandafter/0exp_end:/0meaning/2}0def/0stzz191/6#6#C1CmCaCcCrCoC:6#6#C20relax/1{1{6#6#C12}1{6#6#C22}2}0def/0stzz205/6#6#C1CmCaCrCk1{2}0def/0stzz209/6#6#C1CoCuCtCeCr1{2}0def/0check_outer_aux/1{0expandafter/0stzz184/0noexpand/2}0def/0stzz184/6#6#C11{0expandafter/0stzz187/0expandafter/1{0meaning/6#6#C12}2}0def/0stzz187/6#6#C11{0expandafter/0stzz195/0expandafter/1{0stzz191/6#6#C1CmCaCcCrCoC:0relax/2}6#6#C12}0def/0stzz195/6#6#C11{0stzz195a/6#6#C12}0def/0stzz195a/6#6#C16#6#C21{0ifempty_simple/1{6#6#C22}0exp_end:/1{0stzz205b/1{6#6#C12}2}2}0def/0stzz205b/6#6#C11{0expandafter/0stzz206/0expandafter/1{0stzz205/6#6#C1CmCaCrCk2}1{6#6#C12}2}0def/0stzz206/6#6#C16#6#C21{0ifempty_simple/1{6#6#C12}1{0stzz206b/1{6#6#C22}2}0exp_end:/2}0def/0stzz206b/6#6#C11{0expandafter/0stzz212/0expandafter/1{0stzz209/6#6#C1CoCuCtCeCr2}2}0def/0stzz212/6#6#C11{0ifempty_simple/1{6#6#C12}0exp_end:/0stzz215/2}0def/0stzz215/1{0exp_end:/C12}0def/0count_string_len/6#6#C16#6#C26#6#C36#6#C41{0stzz232/6#6#C11{6#6#C22}1{6#6#C42}1{6#6#C32}2}0def/0stzz232/6#6#C16#6#C26#6#C36#6#C41{0stzz233/6#6#C41{6#6#C22}1{6#6#C32}6#6#C12}0def/0stzz233/6#6#C16#6#C26#6#C36#6#C41{0stzz234/6#6#C3C*1{6#6#C22}6#6#C16#6#C42}0def/0stzz234/6#6#C1C*6#6#C26#6#C36#6#C41{0stzz236/6#6#C1C*6#6#C46#6#C36#6#C2C*2}0def/0stzz236/6#6#C1C*6#6#C26#6#C36#6#C4C*1{0stzz237/6#6#C26#6#C4C*6#6#C36#6#C1C*6#6#C2BA2}0def/0stzz237/6#6#C16#6#C2C*6#6#C36#6#C4C*1{0expandafter/0stzz240/0expandafter/6#6#C10exp:w/6#6#C20expandafter/0exp_end:/0expandafter/C*0expandafter/6#6#C30exp:w/6#6#C40expandafter/0exp_end:/0expandafter/C*2}0def/0stzz240/6#6#C16#6#C2C*6#6#C36#6#C4C*1{0stzz240a/6#6#C2C*6#6#C16#6#C4C*6#6#C30exp:w/0stringify_remove_next_repeated_count/1{6#6#C22}2}0def/0stzz240a/6#6#C1C*6#6#C26#6#C3C*6#6#C41{0expandafter/0stzz241/0exp:w/6#6#C10expandafter/0exp_end:/0expandafter/C*0expandafter/6#6#C20exp:w/6#6#C30expandafter/0exp_end:/0expandafter/C*0expandafter/6#6#C42}0def/0stzz241/6#6#C1C*6#6#C26#6#C3C*6#6#C41{0stzz241a/6#6#C46#6#C26#6#C3C*6#6#C1C*0exp:w/6#6#C42}0def/0stzz241a/6#6#C16#6#C26#6#C3C*6#6#C4C*1{0expandafter/0stzz242/0expandafter/6#6#C10expandafter/6#6#C20exp:w/6#6#C30expandafter/0exp_end:/0expandafter/C*0exp:w/6#6#C40expandafter/0exp_end:/0expandafter/C*2}0def/0stzz242/6#6#C16#6#C26#6#C3C*6#6#C4C*1{0stzz242a/6#6#C3C*6#6#C26#6#C16#6#C4C*0exp:w/0stringify_remove_next_repeated_count/1{6#6#C32}2}0def/0stzz242a/6#6#C1C*6#6#C26#6#C36#6#C4C*1{0expandafter/0stzz246/0exp:w/6#6#C10expandafter/0exp_end:/0expandafter/C*0expandafter/6#6#C20expandafter/6#6#C30exp:w/6#6#C40expandafter/0exp_end:/0expandafter/C*2}0def/0stzz246/6#6#C1C*6#6#C26#6#C36#6#C4C*1{0stzz246a/6#6#C1C*6#6#C26#6#C36#6#C4C*0exp:w/0stringify_until_A/2}0def/0stzz246a/6#6#C1C*6#6#C26#6#C36#6#C4C*1{0expandafter/0stzz249/0exp:w/6#6#C10expandafter/0exp_end:/0expandafter/C*0expandafter/6#6#C20expandafter/6#6#C30exp:w/6#6#C40expandafter/0exp_end:/0expandafter/C*2}0def/0stzz249/6#6#C1C*6#6#C26#6#C36#6#C4C*6#6#C5BA1{0stzz252/6#6#C4C*6#6#C1C*6#6#C26#6#C31{6#6#C52}2}0def/0stzz252/6#6#C1C*6#6#C2C*6#6#C36#6#C46#6#C51{0expandafter/0stzz253/0expandafter/1{0exp:w/0replicate/1{6#6#C52}0exp_forward/2}6#6#C2C*6#6#C36#6#C46#6#C1C*2}0def/0stzz253/6#6#C16#6#C2C*6#6#C36#6#C46#6#C5C*1{0stzz256/6#6#C5C*6#6#C2C*6#6#C36#6#C46#6#C1C*2}0def/0stzz256/6#6#C1C*6#6#C2C*6#6#C36#6#C46#6#C5C*1{0stzz257/6#6#C1C*6#6#C2C*6#6#C36#6#C46#6#C5C*6#6#C3BB2}0def/0stzz257/6#6#C1C*6#6#C2C*6#6#C36#6#C46#6#C5C*1{0expandafter/0stzz261/0exp:w/6#6#C10expandafter/0exp_end:/0expandafter/C*0exp:w/6#6#C20expandafter/0exp_end:/0expandafter/C*0expandafter/6#6#C30expandafter/6#6#C40exp:w/6#6#C50expandafter/0exp_end:/0expandafter/C*2}0def/0stzz261/6#6#C1C*6#6#C2C*6#6#C36#6#C46#6#C5C*1{0stzz261a/6#6#C1C*6#6#C46#6#C36#6#C2C*6#6#C5C*0exp:w/0stringify_remove_next_repeated_count/1{6#6#C12}2}0def/0stzz261a/6#6#C1C*6#6#C26#6#C36#6#C4C*6#6#C5C*1{0expandafter/0stzz262/0exp:w/6#6#C10expandafter/0exp_end:/0expandafter/C*0expandafter/6#6#C20expandafter/6#6#C30exp:w/6#6#C40expandafter/0exp_end:/0expandafter/C*0exp:w/6#6#C50expandafter/0exp_end:/0expandafter/C*2}0def/0stzz262/6#6#C1C*6#6#C26#6#C36#6#C4C*6#6#C5C*1{0stzz262a/6#6#C1C*6#6#C26#6#C36#6#C4C*6#6#C5C*0exp:w/6#6#C22}0def/0stzz262a/6#6#C1C*6#6#C26#6#C36#6#C4C*6#6#C5C*1{0expandafter/0stzz263/0exp:w/6#6#C10expandafter/0exp_end:/0expandafter/C*0expandafter/6#6#C20expandafter/6#6#C30exp:w/6#6#C40expandafter/0exp_end:/0expandafter/C*0exp:w/6#6#C50expandafter/0exp_end:/0expandafter/C*2}0def/0stzz263/6#6#C1C*6#6#C26#6#C36#6#C4C*6#6#C5C*1{0stzz263a/6#6#C1C*6#6#C26#6#C36#6#C4C*6#6#C5C*0exp:w/0stringify_remove_next_repeated_count/1{6#6#C46#6#C52}2}0def/0stzz263a/6#6#C1C*6#6#C26#6#C36#6#C4C*6#6#C5C*1{0expandafter/0stzz265/0exp:w/6#6#C10expandafter/0exp_end:/0expandafter/C*0expandafter/6#6#C20expandafter/6#6#C30exp:w/6#6#C40expandafter/0exp_end:/0expandafter/C*0exp:w/6#6#C50expandafter/0exp_end:/0expandafter/C*2}0def/0stzz265/6#6#C1C*6#6#C26#6#C36#6#C4C*6#6#C5C*1{0stzz265a/6#6#C1C*6#6#C26#6#C36#6#C4C*6#6#C5C*0string/2}0def/0stzz265a/6#6#C1C*6#6#C26#6#C36#6#C4C*6#6#C5C*1{0expandafter/0stzz271/0exp:w/6#6#C10expandafter/0exp_end:/0expandafter/C*0expandafter/6#6#C20expandafter/6#6#C30exp:w/6#6#C40expandafter/0exp_end:/0expandafter/C*0exp:w/6#6#C50expandafter/0exp_end:/0expandafter/C*2}0def/0stzz271/6#6#C1C*6#6#C26#6#C36#6#C4C*6#6#C5C*6#6#C61{0stzz272/1{6#6#C62}6#6#C26#6#C1C*6#6#C36#6#C4C*6#6#C5C*2}0def/0stzz272/6#6#C16#6#C26#6#C3C*6#6#C46#6#C5C*6#6#C6C*1{0ifx/6#6#C1CA0use_ii_to_i:w/0fi/0use_ii:nn/1{0stzz284c/6#6#C3C*6#6#C26#6#C46#6#C5C*2}0stzz284/6#6#C6C*2}0def/0stzz284/6#6#C1C*1{0exp_end:/6#6#C12}0def/0stzz284c/6#6#C1C*6#6#C26#6#C36#6#C4C*6#6#C5C*1{0stzz274/6#6#C1C*6#6#C4C*6#6#C36#6#C26#6#C5C*2}0def/0stzz274/6#6#C1C*6#6#C2C*6#6#C36#6#C46#6#C5C*1{0expandafter/0stzz275/0expandafter/1{0exp:w/0count_string_len/6#6#C31{6#6#C12}6#6#C41{6#6#C26#6#C50exp_forward/2}2}6#6#C5C*2}0def/0stzz275/6#6#C16#6#C2C*1{0stzz280/6#6#C1C*6#6#C2C*2}0def/0stzz280/6#6#C1C*6#6#C2C*1{0stzz280a/6#6#C1C*6#6#C2C*0exp:w/0stringify_remove_next_repeated_count/1{6#6#C16#6#C10exp_forward/0exp_forward/2}2}0def/0stzz280a/6#6#C1C*6#6#C2C*1{0expandafter/0stzz281/0exp:w/6#6#C10expandafter/0exp_end:/0expandafter/C*0exp:w/6#6#C20expandafter/0exp_end:/0expandafter/C*2}0def/0stzz281/6#6#C1C*6#6#C2C*1{0exp_end:/6#6#C20exp_forward/6#6#C12}0def/0ifempty_simple/6#6#C11{0ifx/0relax/6#6#C10relax/0expandafter/0use_i:nn/0else/0expandafter/0use_ii:nn/0fi/2}0def/0ifnotempty_simple/6#6#C11{0ifx/0relax/6#6#C10relax/0expandafter/0use_ii:nn/0else/0expandafter/0use_i:nn/0fi/2}0def/0iflenequal_simple/6#6#C16#6#C21{0expandafter/0stzz312/0exp:w/0remove_next_repeated_count/1{6#6#C22}6#6#C1C?6#6#C2C+2}0def/0stzz312/6#6#C16#6#C2C+1{0texconditional/1{0ifx/6#6#C1C?2}2}0def/0if_outer/6#6#C11{0stzz318/6#6#C12}0def/0stzz318/6#6#C11{0expandafter/0stzz321/0expandafter/1{0exp:w/0count_string_len/6#6#C11{2}0r_function_do_nothing/1{2}2}6#6#C12}0def/0stzz321/6#6#C16#6#C21{0expandafter/0stzz326/0expandafter/1{0use_none:n/6#6#C12}1{6#6#C22}2}0def/0stzz322a/6#6#C16#6#C21{0expandafter/0stzz326/0expandafter/1{0use_none:n/6#6#C22}1{6#6#C12}2}0def/0stzz326/6#6#C16#6#C21{0expandafter/0stzz327/0expandafter/1{0exp:w/0count_string_len/6#6#C21{6#6#C12}0r_function_meaning_noexpand/1{2}2}1{6#6#C22}1{6#6#C12}2}0def/0stzz327/6#6#C16#6#C26#6#C31{0expandafter/0stzz328/0expandafter/1{0exp:w/0count_string_len/6#6#C21{6#6#C32}0r_function_meaning/1{2}2}1{6#6#C22}1{6#6#C32}1{6#6#C12}2}0def/0stzz328/6#6#C16#6#C26#6#C36#6#C41{0iflenequal_simple/1{6#6#C42}1{6#6#C12}1{0stzz336f/1{6#6#C32}1{6#6#C22}2}1{0stzz332/1{6#6#C22}1{6#6#C32}1{6#6#C12}2}2}0def/0stzz332/6#6#C16#6#C26#6#C31{0expandafter/0stzz333/0expandafter/1{0exp:w/0count_string_len/6#6#C11{6#6#C22}0check_outer_aux/1{2}2}1{6#6#C12}1{6#6#C22}1{6#6#C32}2}0def/0stzz333/6#6#C16#6#C26#6#C36#6#C41{0iflenequal_simple/1{6#6#C12}1{6#6#C42}1{0stzz336f/1{6#6#C32}1{6#6#C22}2}0use_i:nn/2}0def/0stzz336f/6#6#C16#6#C21{0ifnotempty_simple/1{6#6#C12}1{0stzz322a/1{6#6#C22}1{6#6#C12}2}0use_ii:nn/2}0def/0debug/C1C21{0stzz352/2}0def/0stzz352/C3C41{0exp_end:/2}.
\end{filecontentsdefmacro}
\ExplSyntaxOn
\exp_args:NV \__process_all \data

\def \use_ii_to_i:w \fi \use_ii:nn { \fi \use_i:nn }
\def \texconditional #1 {#1 \use_ii_to_i:w \fi \use_ii:nn}

\ExplSyntaxOff


% -----
% Tests
% -----

\def\rain{My bald head is still wet.}
\def\unfold{\umbrella My bald {head always} dries so slowly.}
\outer\def\umbrella{No raindrops on my head, please! }
\newcount\abc
\tt
3\csname if_outer\endcsname\unfold{T}{F} (T)\par
4\csname if_outer\endcsname\rain{T}{F} (F)\par
7\edef\tmpa{\csname if_outer\endcsname\unfold{T}{F}}\meaning\tmpa (T)\par

\end{document}

It's compatible with all engines, I think.

As mentioned above. Absolutely do not try to reverse engineer what the code does.

I compiled that code from the source code at https://github.com/user202729/TeXlib/blob/main/test_imperative2.tex using one of my unreleased libraries.

This one (imperative, name might change later subject to CTAN maintainers) is made to make expandable programming/input stream parsing macros easier to write, and reduce the number of meaningless _aux, _auxi, _auxii etc. macro. (or worse, \macro@, \macro@@, \m@cro, m@cr@)

No documentation so far, unfortunately...

1
  • I believe this approach is completely robust, and I also believe that it's impossible to implement the function given only the braced group following in the input stream (because of the \meaning issue above. It would be interesting to see what the situation would be if the input is given wrapped inside two layers of braces and \unexpanded and \expanded is available)
    – user202729
    Commented Jul 5, 2022 at 19:55

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .