# Is there a token not affectable by \uppercase/\lowercase and not (re-)definable in terms of \outer?

Is there a token not affectable by \uppercase/\lowercase and not (re-)definable in terms of \outer?

(If so I would like to use it as argument delimiter with things that may be used inside \uppercase/\lowercase.)

• I am not sure what you mean by not definable in terms of outer? please show a small test document. All non-character tokens are not affected by uppercase and lowercase. – David Carlisle Nov 11 '20 at 10:47
• Is ! an answer, as a catcode 12 token it is not definable, and it is not affected by uppercase or lowercase with the default lccode and uccode tables. But I do not understand the question enough to be sure if this is what you mean? – David Carlisle Nov 11 '20 at 10:49
• Are we allowed to use LuaMetaTeX? – Joseph Wright Nov 11 '20 at 10:57
• You should really open an account. – egreg Nov 11 '20 at 12:32

Afaik there is no such token:

Each token belongs into at least one of the following two classes of tokens—active character tokens belong into both classes simultaneously:

1. Control sequences. (Control word tokens, control symbol tokens, active character tokens.) All control sequences are (re)definable in terms of \outer.
2. Explicit character tokens. Each explicit character token is affectable by \uppercase/\lowercase (provided its \uccode/\lccode is set accordingly).

In many situations you can avoid delimited arguments entirely by instead doing with undelimited arguments and checking for emptiness.

E.g., for extracting the first unelimited argument from a brace-balanced token-list I often use something like this:

%%   \romannumeral\UD@ExtractFirstArgLoop{<argument>\UD@SelDOm}%
%%   yields <argument>'s 1st undlimited argument.
%%   <argument> must not be blank, i.e., must neither be empty nor consist
%%   only of explicit character tokens of catcode 10 and charcode 32.
%%
%%   \UD@SelDOm must not be defined in terms of \outer !
%%.............................................................................
\@ifdefinable\UD@RemoveTillUD@SelDOm{%
\long\def\UD@RemoveTillUD@SelDOm#1#2\UD@SelDOm{{#1}}%
}%
\newcommand\UD@ExtractFirstArgLoop[1]{%
\expandafter\UD@CheckWhetherNull\expandafter{\@firstoftwo{}#1}%
{\expandafter\z@\@secondoftwo{}#1}%
{\expandafter\UD@ExtractFirstArgLoop\expandafter{\UD@RemoveTillUD@SelDOm#1}}%
}%


(\UD@CheckWhetherNull being defined as

%%-----------------------------------------------------------------------------
%% Check whether argument is empty:
%%.............................................................................
%% \UD@CheckWhetherNull{<Argument which is to be checked>}%
%%                     {<Tokens to be delivered in case that argument
%%                       which is to be checked is empty>}%
%%                     {<Tokens to be delivered in case that argument
%%                       which is to be checked is not empty>}%
\newcommand\UD@CheckWhetherNull[1]{%
\romannumeral\expandafter\@secondoftwo\string{\expandafter
\@secondoftwo\expandafter{\expandafter{\string#1}\expandafter
\@secondoftwo\string}\expandafter\@firstoftwo\expandafter{\expandafter
\@secondoftwo\string}\expandafter\z@\@secondoftwo}%
{\expandafter\z@\@firstoftwo}%
}%


or as

\newcommand\UD@CheckWhetherNull[1]{%
\romanumeral\ifcat$\detokenize{#1}$%
\expandafter\expandafter\expandafter\z@\expandafter\@firstoftwo\else
\expandafter\expandafter\expandafter\z@\expandafter\@secondoftwo\fi
}%


. )

With this you get:

\romannumeral\UD@ExtractFirstArgLoop{{A}{B}CDE\UD@SelDOm}%A.

In order to decrease the amount of iterations needed for removing everything but the first non-delimited argument I used an argument delimited by \UD@SelDOm. (The amount of iterations needed is: "Amount of \UD@SelDOm within the argument that are not nested in braces"+1).

If you don't like the \UD@SelDOm-delimiter as it might be defined in terms of \outer, then you can as follows do without—at the cost of needing more iterations until obtaining the result—the amount of iterations needed is: "Amount of undelimited arguments within the argument that themselves are not nested in braces"+1:

% Syntax now is: \romannumeral\UD@ExtractFirstArgLoop{<argument>{}}%
\newcommand\UD@GrabFirst[2]{{#1}}%
\renewcommand\UD@ExtractFirstArgLoop[1]{%
\expandafter\UD@CheckWhetherNull\expandafter{\@firstoftwo{}#1}%
{\expandafter\z@\@secondoftwo{}#1}%
{\expandafter\UD@ExtractFirstArgLoop\expandafter{\UD@GrabFirst#1}}%
}%


E.g.,

\romannumeral\UD@ExtractFirstArgLoop{{A}{BC}DE{}}%A

There are many situations where you can do similar things in order to entirely avoid delimited arguments.

Sometimes people use as argument-delimiter the macro-token which processes the argument in question, e.g., \def\macro#1\macro{...}. Whether this is feasible/reasonable depends on the question whether \macro could be nested within its own argument and thus erroneously match up the delimiter or whether something like \let\macrob=\macro \outer\def\macro... could occur.

Since you are already thinking about the question how to make macro arguments safe, I would like to note that there are other traps besides \outer\def... and \uppercase/\lowercase:

1. For example the question whether a macro-mechanism based on delimited arguments, which shall handle (almost) arbitrary arguments given by the user, should work inside a tabular environment/within an alignment.

Assume, e.g., a macro \grabdelimited which processes a delimited argument and which the user uses for gathering and detokenizing the characters B and &.

\documentclass{article}
\def\grabdelimited#1\delimiter{Arguments grabbed: \detokenize{#1}}%
\begin{document}
\grabdelimited B&\delimiter

\makeatletter
\begin{tabular}{|l|l|}
%A&\relax\grabdelimited B&\delimiter\\
A&\relax\expandafter\@firstofone\expandafter{\grabdelimited B&\delimiter}
\end{tabular}
\end{document}


The first/commented line inside the tabular-environment would yield an error, the second does not because here the & belonging to the delimited argument is hidden inside the curly braces of \@firstofone.

2. Yet another problem might be passing unbalanced \if.../\else/\fi as macro-arguments which might erroneously match up some \if.../\else that occur in the definitions of the macros that process those arguments.

Same with unbalanced \csname/\endcsname.

Extended comment:

If you're really afraid that your end-of-argument marker might get affected by \uppercase/\lowercase you just have to make sure that it doesn't appear on top level so that it is never seen by \uppercase or \lowercase. This could be achieved by making sure that it is only inserted in a context where expansion can't stop in between, e.g., by using a \romannumeral expansion context. The following sets up a macro which uses the character D as the end-of-argument marker, but since it expands further that marker is never left in the input stream such that it could be affected by \lowercase or \uppercase:

% first insert a \romannumeral such that the following is expanded as far as possible
\newcommand*\mymacro{\romannumeral\mymacro@a}
% only after \romannumeral has started input the delimiter
\newcommand\mymacro@a[1]{\mymacro@b #1D}


Then \mymacro@b could process the argument in an expandable manner and use D as the delimiter. And you could end the \romannumeral expansion context with \z@. Of course you could still expand \mymacro once and then expand \mymacro@a with \expandafter without starting \romannumeral such that the D could get affected by \lowercase (with \expandafter\expandafter\expandafter\lowercase\expandafter\expandafter\expandafter{\expandafter\expandafter\mymacro{}}), but at least now this has to be created with malicious intent.

You can never protect yourself against an \outer redefinition but people redefining internals of other people's code as \outer seem to not want to have working code, so maybe this is no case you have to protect yourself against.