2

I want to make ^ unspecial in a command definition, i. e. “^” in the input should give the “^” character in the output. Setting the catcode to 11 or 12 works—but not for “^^”. Instead LaTeX replaces “^^!” by “a” and “^^h” by “(”. What’s happening here and what can I do against that?

Hints:

  • The ASCII codes of !, a; h, ( are 0x21, 0x61; 0x28, 0x68. The same problem for other characters: ^^ vanishes and 0x40 is added to or subtracted from the next character. My personal WTF moment.
  • The problem is the same for LaTeX, pdfLaTeX, LuaLaTeX, XeLaTeX.
\documentclass{article}

\newcommand{\func}[1]{{%
    \catcode94=12\relax%^
    \catcode95=12\relax%_
    \scantokens{#1}%
}}

\begin{document}
OK:\func{_!_h}

OK:\func{^!^h}

OK:\func{__!__h}

NOT OK:\func{^^!^^h}

\catcode94=12\relax
\catcode95=12\relax
OK:__!__h

OK:^^!^^h
\end{document}

Sorry for that not-minimal code example; I wanted to show that __ (two _) doesn’t have that problem and that the problem occurs only when I try that function definition.

2
  • if you make ^ catcode 12 then ^ will act as a normal character but you need to do it before you scan the argument, as is usual with tex macros and catcode changes Aug 14 '19 at 20:57
  • 2
    About your "WTF moment", see The TeXBook, Appendix C: Character Codes. Quote: "Here ^^ doesn't necessarily mean two circumflex characters; it means two identical characters whose current \catcode is 7. In such cases TeX simply adds or subtracts '100 [64 in decimal] from the internal code of the character that immediately follows. For example, * can also be typed as ^^j; j can also be typed as ^^*." This is the way Knuth implemented so that users can type all the 128 ASCII characters, not just the visible (or available in your keybord) range. Aug 14 '19 at 21:11
5

enter image description here

You need to change the catcode before you scan the argument:

\documentclass{article}

\usepackage[T1]{fontenc}

\newcommand{\func}{\bgroup%
    \catcode94=12\relax%^
    \catcode95=12\relax%_
    \xfunc}
\newcommand\xfunc[1]{\scantokens{#1}\egroup}

\begin{document}
OK:\func{_!_h}

OK:\func{^!^h}

OK:\func{__!__h}

NOT OK:\func{^^!^^h}

\catcode94=12\relax
\catcode95=12\relax
OK:__!__h

OK:^^!^^h
\end{document}

The difference between _ and ^ is that __! still makes three tokens so you can "correct" the tokens to have catcode12 even after the argument has been parsed by using \scantokens but as ^^! just parses as a single token, \scantokens can not be used to get three catcode 12 tokens.

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.