6

If \lccode does not change category codes, shouldn't the code based on the lowercase trick work for numbers too? For that matter, any character?

\documentclass{article}
\usepackage{fontspec}% xelatex

\long\def\clownaround#1{% based on https://tex.stackexchange.com/a/219497/13552
  \begingroup% only needed to limit scope of formatting macros
  \ttfamily%
  \begingroup\lccode`~=`0\lowercase{\endgroup\def~}{YOUVEBEENPSYCHED0}%
  \begingroup\lccode126=48\lowercase{\endgroup\def~}{YOUVEBEENPSYCHED0} % equivalent of prev line
  \begingroup\lccode`~=`/\lowercase{\endgroup\def~}{YOUVEBEENFOOLED/}%
  \begingroup\lccode`~=`a\lowercase{\endgroup\def~}{YOUVEBEENDUPEDa}%
  \catcode`/=\active\catcode`.=\active% what does this do?
  \scantokens{#1\noexpand}%
  \endgroup%
}
\begin{document}

\clownaround{01/a}% Expected output: YOUVEBEENPSYCHED01YOUVEBEENFOOLED/YOUVEBEENDUPEDa

\end{document}
  • 2
    You have only made / and . active, not 0, 1 or a – Joseph Wright Mar 22 '18 at 14:41
9

To answer the question asked, “code based on the \lowercase trick” does work for numbers or for any characters. There are a few different things going on in the question; introducing them separately may help clarify matters:


  1. You can give a definition for any character, but the definition is used only if the character is an active character when TeX encounters it.

Giving a definition to a character simply stores that definition in a table somewhere. When TeX encounters a character, if it's of category code 11 (“letter”) or 12 (“other char”) TeX simply typesets that character, if it's of category code 3 (“math shift”, like $) TeX enters or exits math mode, etc. It's only if the character has catcode 13 (“active char”) that TeX looks up the character's definition and uses it.

This is the main thing going on in the question as asked, and here's a simpler example:

\documentclass{article}
\begin{document}

\catcode`Z=13  % Makes Z an active character
\defZ{Duck}    % Gives Z the definition "Duck"
\catcode`Z=11  % Makes Z a letter again

Z Z Goose      % Output: Z Z Goose

\catcode`Z=13  % Makes Z an active character
Z Z Goose      % Output: Duck Duck Goose
\end{document}

  1. The \lowercase trick is, in this example, simply a way to avoid setting the catcode and back again. (Because ~ inside \lowercase results in a token having the catcode of ~, but having the actual character the lowercase version of ~.)

For example, the following already illustrates many of the features of the question, without the complication of groups and \scantokens:

\documentclass{article}

\begin{document}

\ttfamily
\lccode`~=`0 \lowercase{\def~}{YOUVEBEENPSYCHED0} % Gives the character 0 the definition "YOUVEBEENPSYCHED0"
\lccode`~=`/ \lowercase{\def~}{YOUVEBEENFOOLED/}  % Gives the character / the definition "YOUVEBEENFOOLED/"
\lccode`~=`a \lowercase{\def~}{YOUVEBEENDUPEDa}   % Gives the character a the definition "YOUVEBEENDUPEDa"
\catcode`/=\active % Makes the character / active (so that / will be replaced by its definition)
\catcode`0=\active % Makes the character 0 active (so that 0 will be replaced by its definition)

01/a               % Output: YOUVEBEENPSYCHED01YOUVEBEENFOOLED/a

\end{document}

output without a active

Note that here, we have given the character a a definition, but it's never used (because the character's catcode was not \active = 13).


  1. If we want to restrict the effect of setting \catcode or \lccode to a certain region, that's when we need the further complications of \begingroup and \endgroup (as in the question).

When the character a is active, \relax would be interpreted as \rel followed by a followed by x, so TeX would complain about undefined control sequence \rel. (And \scantokens amusingly doesn't result in a error but just does the wrong thing.) So we need to be careful and enclose it in a group: in the document just mentioned previously, add the following before \end{document}:

% We need to be more careful with the character 'a', because
% otherwise \relax will be interpreted as \rel, then a, then x.
\def\clownaround{
  \begingroup
  \catcode`a=\active
  \scantokens{01/a}
  \endgroup
}
\clownaround

output with a active

Here the definitions of 0, / and a are all used. The \begingroup and \endgroup restrict the scope of \catcode`a = 13 so that things like \relax will still work later. And \scantokens is used so that “01/a” is scanned with the catcodes as of that point (when the macro \clownaround is expanded and \scantokens is encountered), rather than with the catcodes as of the time the macro \clownaround was defined.

Another trick in the question is the \begingroup and \endgroup in

\begingroup\lccode`~=`0\lowercase{\endgroup\def~}{YOUVEBEENPSYCHED0}

which restrict the scope of \lccode as much as possible, just in case we later ask for the lowercase version of ~ and get 0 when we were expecting something else.


  1. The \noexpand at the end of \scantokens is not needed here; it's just general practice with \scantokens for avoiding the space that would otherwise occur.

You got the example in your question from here which got it from here, but see here and here for stuff about \noexpand at the end of \scantokens.

  • 1
    Doesn’t \begingroup \lccode~=0 \lowercase{\def~}{YOUVEBEENPSYCHED0}\endgroup make a local definition, that will not last past the end of the group? – GuM Mar 22 '18 at 22:45
  • @GuM Ah you're right! Will delete the incorrect version. (Deleted.) – ShreevatsaR Mar 22 '18 at 22:47
8

When you rescan material, e-TeX applies the currently active category codes to the input. In the example, the 'normal' category codes apply with the exception of the two affected by the line

\catcode`/=\active\catcode`.=\active

Thus the rescanning leaves 0, etc. as 'other' characters, and the fact that the \lowercase 'trick' has defined an active 0 to expand to something makes no difference. You need to make every character you are interested in 'active':

\catcode`0=\active
\catcode`a=\active
% Others

which leads to a minimal example

\documentclass{article}

\long\def\clownaround#1{% based on https://tex.stackexchange.com/a/219497/13552
  \begingroup% only needed to limit scope of formatting macros
  \ttfamily%
  \begingroup\lccode`~=`0\lowercase{\endgroup\def~}{YOUVEBEENPSYCHED0}%
  \begingroup\lccode`~=`/\lowercase{\endgroup\def~}{YOUVEBEENFOOLED/}%
  \begingroup\lccode`~=`a\lowercase{\endgroup\def~}{YOUVEBEENDUPEDa}%
  \catcode`/=\active
  \catcode`\0=\active
  \catcode`\a=\active
  \endlinechar=-1 %
  \scantokens{#1}%
  \endgroup%
}
\begin{document}

\clownaround{01/a}% Expected output: YOUVEBEENPSYCHED1YOUVEBEENFOOLEDYOUVEBEENDUPED

\end{document}
  • I would venture to guess that you did not try running it. \catcode`a=\active yields Undefined control sequence. – Jonathan Komar Mar 22 '18 at 15:14
  • That was fast! Cool beans. I noticed you changed 0 to \0 and a to \a. It seems to work both ways. I am wondering why you did that. – Jonathan Komar Mar 22 '18 at 15:26
  • 2
    @JonathanKomar (Can't answer for Joseph, but this is a reason one may do it.) Both \catcode`0 and \catcode`\0 are equivalent to \catcode48. The backslashes are sometimes needed (e.g. \catcode`\% mentioned at this question), and sometimes not (as here), but they are always ok. So the habit of always including the backslash saves having to think about whether it's needed or not. – ShreevatsaR Mar 22 '18 at 16:33
  • @ShreevatsaR Just to add a nitpick, sometimes (1% of the time) I found safer to use the non backslash version if the code coulde end up being expanded in an \edef (I don't remember the particular place that happened, but I rememer more than once that the code was in an edef). But you are right! – Manuel Mar 22 '18 at 17:13
  • @JosephWright \everyeof{\noexpand} is redundant in your example, but you may replace it by \endlinechar=-1 in order to suppress the unwanted space created at the end of \scantokens parameter. – wipet Mar 22 '18 at 18:16

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.