11

What would happen if catcodes were changed to 11 (letter) instead of 12 (other) in verbatim environment or something similar? Will any problems happen? Or, are there any advantages to use catcode12 instead of catcode11?

  • Well changing the 12 to 11 in \def\@makeother#1{\catcode‘#111\relax} causes verbatim to crash. That doesn't really answer the question, but gives a clue. – Steven B. Segletes Jun 27 '18 at 15:09
  • 2
    @StevenB.Segletes: The reason is that the \ character in the \end{verbatim} text used to delimit the argument is no longer recognized, having \catcode 11. Perhaps I’ll write an answer to this question, after all…! ;-) – GuM Jun 27 '18 at 15:12
  • From the verbatim package docs: Here comes the tricky part: During the definition of the macros we need to use the special characters \, {, and } not only with their normal category codes, but also with category code 12 (other). We achieve this by the following trick: first we tell TEX that \, {, and } are the lowercase versions of !, [, and ]. Then we replace every occurrence of \, {, and } that should be read with category code 12 by !, [, and ], respectively, and give the whole list of tokens to \lowercase, knowing that category codes are not altered by this primitive! – Steven B. Segletes Jun 27 '18 at 15:18
  • @Gum You'll get my upvote if you do. – Steven B. Segletes Jun 27 '18 at 15:19
  • @StevenB.Segletes: Yes, of course I was forgetting that, in order for \end{verbatim} to be recognized as the delimiter, the \catcodes of { and of } matter too! – GuM Jun 27 '18 at 15:33
7

Well, characters from A to Z and from a to z do retain their \catcode (11) in the verbatim environment. I see no particular contraindications to setting also the \catcodes of other symbols to 11, except for the following ones (of which the first is obvious):

  1. Why should one want to have symbols treated as letters?

  2. The text to be typeset verbatim is absorbed by TeX as an argument delimited by the string \end{verbatim}. In order for this delimiting string to be recognized, both character codes and category codes of its characters must match (see The TeXbook, p. 203, lines -3 to -2). The code that sets up the verbatim environment stores this string with the \catcodes of \, {, and } equal to 12, and with the \catcodes of v, e, r, … ,m equal to 11. If you set the \catcodes of \, {, and } to 11, hence, the delimiting string will no longer be recognized.

The following little program can be used to verify the above claims; try changing the definition of the \MyCatcode macro from

\newcommand*\MyCatcode{12}

to

\newcommand*\MyCatcode{11}

and check that everything works as expected.

% My standard header for TeX.SX answers:
\documentclass[a4paper]{article} % To avoid confusion, let us explicitly 
                                 % declare the paper format.

\usepackage[T1]{fontenc}         % Not always necessary, but recommended.
% End of standard header.  What follows pertains to the problem at hand.

\usepackage{etoolbox}

\makeatletter

\newcommand*\MyCatcode{12}
\newcommand*\@makemychar[1]{\catcode`#1=\MyCatcode\relax}

\patchcmd{\@verbatim}
    {\let\do\@makeother \dospecials}
    {\let\do\@makemychar \dospecials \@makeother\\\@makeother\{\@makeother\}}
    {}
    {}

\newcommand*{\testverbatim}{%
    \begin{verbatim}%
        \typeout{}%
        \typeout{--------------------------------}%
        \typeout{}%
        \typeout{A few examples:}%
        \typeout{}%
        \typeoutcatcode{\\}%
        \typeoutcatcode{\{}%
        \typeoutcatcode{\}}%
        \typeoutcatcode{\$}%
        \typeoutcatcode{\&}%
        \typeoutcatcode{\#}%
        \typeoutcatcode{\^}%
        \typeoutcatcode{\_}%
        \typeoutcatcode{\%}%
        \typeoutcatcode{\ }%
        \typeoutcatcode{0}%
        \typeoutcatcode{9}%
        \typeoutcatcode{@}%
        \typeoutcatcode{A}%
        \typeoutcatcode{Z}%
        \typeoutcatcode{a}%
        \typeoutcatcode{z}%
        \typeoutcatcode{.}%
        \typeoutcatcode{,}%
        \typeoutcatcode{?}%
        \typeoutcatcode{!}%
        \typeout{}%
        \typeout{--------------------------------}%
        \typeout{}%
}
\newcommand*{\typeoutcatcode}[1]{%
    \typeout{\string\catcode `\string #1 = \number \catcode `#1}%
}

\makeatother



\begin{document}

Non-verbatim text.

\testverbatim
Verbatim text: \{}$&#^_%

% My standard header for TeX.SX answers:
\documentclass[a4paper]{article} % To avoid confusion, let us explicitly 
                                 % declare the paper format.

\usepackage[T1]{fontenc}         % Not always necessary, but recommended.
% End of standard header.  What follows pertains to the problem at hand.

\usepackage{etoolbox}

\makeatletter

\newcommand*\MyCatcode{12}
\newcommand*\@makemychar[1]{\catcode`#1=\MyCatcode\relax}

\patchcmd{\@verbatim}
    {\let\do\@makeother \dospecials}
    {\let\do\@makemychar \dospecials \@makeother\\\@makeother\{\@makeother\}}
    {}
    {}

\newcommand*{\testverbatim}{%
    \begin{verbatim}%
        \typeout{}%
        \typeout{--------------------------------}%
        \typeout{}%
        \typeout{A few examples:}%
        \typeout{}%
        \typeoutcatcode{\\}%
        \typeoutcatcode{\{}%
        \typeoutcatcode{\}}%
        \typeoutcatcode{\$}%
        \typeoutcatcode{\&}%
        \typeoutcatcode{\#}%
        \typeoutcatcode{\^}%
        \typeoutcatcode{\_}%
        \typeoutcatcode{\%}%
        \typeoutcatcode{\ }%
        \typeoutcatcode{0}%
        \typeoutcatcode{9}%
        \typeoutcatcode{@}%
        \typeoutcatcode{A}%
        \typeoutcatcode{Z}%
        \typeoutcatcode{a}%
        \typeoutcatcode{z}%
        \typeoutcatcode{.}%
        \typeoutcatcode{,}%
        \typeoutcatcode{?}%
        \typeoutcatcode{!}%
        \typeout{}%
        \typeout{--------------------------------}%
        \typeout{}%
}
\newcommand*{\typeoutcatcode}[1]{%
    \typeout{\string\catcode `\string #1 = \number \catcode `#1}%
}

\makeatother



\begin{document}

...

\end{document}
\end{verbatim}

Non-verbatim text again.

\end{document}

With

\newcommand*\MyCatcode{12}

the diagnostic messages read

--------------------------------

A few examples:

\catcode`\\ = 12
\catcode`\{ = 12
\catcode`\} = 12
\catcode`\$ = 12
\catcode`\& = 12
\catcode`\# = 12
\catcode`\^ = 12
\catcode`\_ = 12
\catcode`\% = 12
\catcode`\  = 13
\catcode`0 = 12
\catcode`9 = 12
\catcode`@ = 12
\catcode`A = 11
\catcode`Z = 11
\catcode`a = 11
\catcode`z = 11
\catcode`. = 12
\catcode`, = 13
\catcode`? = 12
\catcode`! = 12

--------------------------------

showing that letters remain of \catcode 11; on the other hand,

\newcommand*\MyCatcode{11}

shows that the verbatim environment is not disrupted, provided that the three crucial \catcodes of \, {, and } are not tampered with.


Addition

For further evidence, try this:

% My standard header for TeX.SX answers:
\documentclass[a4paper]{article} % To avoid confusion, let us explicitly 
                                 % declare the paper format.

\usepackage[T1]{fontenc}         % Not always necessary, but recommended.
% End of standard header.  What follows pertains to the problem at hand.

\usepackage{etoolbox}

\makeatletter

\begingroup \catcode `|=0 \catcode `[= 1
\catcode`]=2
% Beware:
\catcode `\{=11 \catcode `\}=11 \catcode`\\=11
% ------
|gdef|@xverbatim#1\end{verbatim}[#1|end[verbatim]]
|gdef|@sxverbatim#1\end{verbatim*}[#1|end[verbatim*]]
|endgroup


\newcommand*\MyCatcode{11}
\newcommand*\@makemychar[1]{\catcode`#1=\MyCatcode\relax}

\patchcmd{\@verbatim}
    {\let\do\@makeother}
    {\let\do\@makemychar}
    {}
    {}

\newcommand*{\testverbatim}{%
    \begin{verbatim}%
        \typeout{}%
        \typeout{--------------------------------}%
        \typeout{}%
        \typeout{A few examples:}%
        \typeout{}%
        \typeoutcatcode{\\}%
        \typeoutcatcode{\{}%
        \typeoutcatcode{\}}%
        \typeoutcatcode{\$}%
        \typeoutcatcode{\&}%
        \typeoutcatcode{\#}%
        \typeoutcatcode{\^}%
        \typeoutcatcode{\_}%
        \typeoutcatcode{\%}%
        \typeoutcatcode{\ }%
        \typeoutcatcode{0}%
        \typeoutcatcode{9}%
        \typeoutcatcode{@}%
        \typeoutcatcode{A}%
        \typeoutcatcode{Z}%
        \typeoutcatcode{a}%
        \typeoutcatcode{z}%
        \typeoutcatcode{.}%
        \typeoutcatcode{,}%
        \typeoutcatcode{?}%
        \typeoutcatcode{!}%
        \typeout{}%
        \typeout{--------------------------------}%
        \typeout{}%
}
\newcommand*{\typeoutcatcode}[1]{%
    \typeout{\string\catcode `\string #1 = \number \catcode `#1}%
}

\makeatother



\begin{document}

Non-verbatim text.

\testverbatim
Verbatim text: \{}$&#^_%

% My standard header for TeX.SX answers:
\documentclass[a4paper]{article} % To avoid confusion, let us explicitly 
                                 % declare the paper format.

\usepackage[T1]{fontenc}         % Not always necessary, but recommended.
% End of standard header.  What follows pertains to the problem at hand.

\usepackage{etoolbox}

\makeatletter

\newcommand*\MyCatcode{12}
\newcommand*\@makemychar[1]{\catcode`#1=\MyCatcode\relax}

\patchcmd{\@verbatim}
    {\let\do\@makeother \dospecials}
    {\let\do\@makemychar \dospecials \@makeother\\\@makeother\{\@makeother\}}
    {}
    {}

\newcommand*{\testverbatim}{%
    \begin{verbatim}%
        \typeout{}%
        \typeout{--------------------------------}%
        \typeout{}%
        \typeout{A few examples:}%
        \typeout{}%
        \typeoutcatcode{\\}%
        \typeoutcatcode{\{}%
        \typeoutcatcode{\}}%
        \typeoutcatcode{\$}%
        \typeoutcatcode{\&}%
        \typeoutcatcode{\#}%
        \typeoutcatcode{\^}%
        \typeoutcatcode{\_}%
        \typeoutcatcode{\%}%
        \typeoutcatcode{\ }%
        \typeoutcatcode{0}%
        \typeoutcatcode{9}%
        \typeoutcatcode{@}%
        \typeoutcatcode{A}%
        \typeoutcatcode{Z}%
        \typeoutcatcode{a}%
        \typeoutcatcode{z}%
        \typeoutcatcode{.}%
        \typeoutcatcode{,}%
        \typeoutcatcode{?}%
        \typeoutcatcode{!}%
        \typeout{}%
        \typeout{--------------------------------}%
        \typeout{}%
}
\newcommand*{\typeoutcatcode}[1]{%
    \typeout{\string\catcode `\string #1 = \number \catcode `#1}%
}

\makeatother



\begin{document}

...

\end{document}
\end{verbatim}

Non-verbatim text again.

\end{document}
  • I just would like to confirm that: does it mean that verbatim environment works well even if each special character's catcode changed 11 as long as consistency of catcodes in the environment is kept? – agnagic Jun 28 '18 at 14:14
  • 1
    @agnagic: I repeat, unless I’m forgetting something, changing the \catcode of specials character to 11 does not affect the outcome of a verbatim environment, provided, of course, that the delimiting string \end{verbatim} is also stored with all \catcodes set to 11. – GuM Jun 29 '18 at 16:30
9

Have a look at how the verbatim environment is implemented in the commented source code of LaTeX 2ε which can be found in source2e.pdf (https://ctan.org/pkg/source2e).The verbatim-environment is in File y: ltmiscen.dtx, section 53.3 Verbatim.

There you can see that the macro \verbatim, which is called by\begin{verbatim},

  • first does call the command \@verbatim which in turn does a lot of preparation like performing required catcode-changes (by means of \let\do=\@makeother and \dospecials), adding vertical space and the like,
  • then calls \frenchspacing for having all spaces in equal width,
  • then does call \@vobeyspaces for turning spaces into something where no line-breaks occur,
  • then does call \@xverbatim.

\@xverbatim in turn does "catch" the text that is to be typeset verbatim in terms of a delimited argument, whereby the delimiter consists of a sequence of character tokens \, e, n, d, {, v, e, r, b, a, t, i, m, }, whereby the catcode-régime at the time of tokenizing the delimiter within the definition-text of \@xverbatim is set as follows:

\begingroup \catcode ‘|=0 \catcode ‘[= 1
\catcode‘]=2 \catcode ‘\{=12 \catcode ‘\}=12
\catcode‘\\=12 |gdef|@xverbatim#1\end{verbatim}[#1|end[verbatim]]
|endgroup

This means \@xverbatim "expects" the character-tokens \ and { and } from the delimiting phrase \end{verbatim} to have catcode 12 (other).

\@xverbatim will spit out its argument trailed by the sequence |end[verbatim] where |end is the control word \end and [ and ] are of category code 1 respective 2 and thus serve as opening braces/closing braces so that the entire thing is the \end-command for the environment.

For fun you can say

\def\@makeother#1{\catcode`#1=11\relax}

and

\begingroup
\catcode`|=0  \catcode`[=1   \catcode`]=2
\catcode`{=11 \catcode`\}=11 \catcode`\\=11 
|@firstofone[%
  |endgroup
  |def|@xverbatim#1\end{verbatim}[#1|end[verbatim]]%
]%

and see that now the verbatim-environment works with catcode-11-characters also:

\documentclass{article}
\begin{document}
\begingroup
\makeatletter
\def\@makeother#1{\catcode`#1=11\relax}%
\begingroup
\catcode`|=0  \catcode`[=1   \catcode`]=2
\catcode`{=11 \catcode`\}=11 \catcode`\\=11 
|@firstofone[%
  |endgroup
  |def|@xverbatim#1\end{verbatim}[#1|end[verbatim]]%
]%
\makeatother
\begin{verbatim}
\{}[]~^_-+#'blabla
\{}[]~^_-+#'blabla
\{}[]~^_-+#'blabla
\end{verbatim}
\endgroup
\end{document}

enter image description here

  • +1 for the reference to the LaTeX sources (I was to lazy to write all that stuff! :-) – GuM Jun 27 '18 at 16:14

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.