6

To which specific examples from answer to exercise 20.7 in TeXbook refers the following part of the answer?

Category codes are not shown, but a character of category 6 always appears twice in succession. A parameter token in the replacement text uses the character code of the final parameter in the parameter text.

  • The whole question 20.7 is about strange charcode combinations for catcodes 1, 2 and 6: isn't it 'the whole question'? (The question has catcode-6 ! and #, then has e.g. !# which will be equivalent to the usual ##.) – Joseph Wright May 15 at 9:59
7

That part of the answer refers to the output of \tracingmacros=1. It also contains a mistake. (Edit: I got carried away by another discrepancy in the answer. The OP pointed out the correct interpretation; there is no mistake in this sentence.)


Exercise 20.7 on page 205, marked with a double “dangerous bend” symbol, asks:

Suppose that ‘[’, ‘]’, and ‘!’ have the respective catcodes 1, 2, and 6, as do ‘{’, ‘}’, and ‘#’. See if you can guess what the following definition means:

\def\!!1#2![{!#]#!!2}

What token list will result when ‘\! x{[y]][z}’ is expanded?

As mentioned on page 37, catcode 1 is “beginning of group” (like {). Catcode 2 is “end of group” (like }). Catcode 6 is parameter (like #). So to answer the question first:

\def\!!1#2![{!#]#!!2}
┗━┳┛┗┫┗┫┗┫┃┃┃┗┫┃┗┫┗┫┃
  ┃  ┃ ┃ ┃┃┃┃ ┃┃ ┃ ┃┃
  ┗━━╋━╋━╋╋╋╋━╋╋━╋━╋╋━━━━━ \def primitive defines a macro.
     ┗━╋━╋╋╋╋━╋╋━╋━╋╋━━━━━  \!  means that the command being defined is \! (like the \foo in \def\foo{bar} etc).
       ┗━╋╋╋╋━╋╋━╋━╋╋━━━━━  !1  is like #1 and means parameter 1, the first parameter passed to the macro when used.
         ┗╋╋╋━╋╋━╋━╋╋━━━━━  #2  means parameter 2, the second parameter passed to the macro when used.
          ┗╋╋━╋╋━╋━╋╋━━━━━  !   (as it's of catcode 6 and followed by a catcode 1 token) means, by the special rule mentioned after 20.5,
           ┃┃ ┃┃ ┃ ┃┃           that a token [ namely catcode 1, char 91 is inserted at end of both parameter text and replacement text.
           ┗╋━╋╋━╋━╋╋━━━━━  [   as it has catcode 1 is like { so this marks end of parameter text / start of replacement text of the macro
            ┗━╋╋━╋━╋╋━━━━━  {   means the token { namely catcode 1, char 123
              ┗╋━╋━╋╋━━━━━  !#  as it is two consecutive chars with catcode 6, means the token # namely catcode 6, char 35 -- HERE THE LATTER CHAR IS USED
               ┗━╋━╋╋━━━━━  ]   means the token ] namely catcode 2, char 93
                 ┗━╋╋━━━━━  #!  as it is two consecutive chars with catcode 6, means the token ! namely catcode 6, char 33 -- HERE THE LATTER CHAR IS USED
                   ┗╋━━━━━  !2  is like #2 and means parameter 2, the second parameter passed to the macro when used.
                    ┗━━━━━  }   marks the end of the replacement text and macro definition, but recall there's a { at end, by the special rule.

In case your browser doesn't display that properly:

answer

When you invoke the expansion mentioned in the question, e.g. with

\catcode`[=1
\catcode`]=2
\catcode`!=6
\def\!!1#2![{!#]#!!2}

\tracingmacros=1
\message{Expansion gives \! x{[y]][z} ok?}

\end

You get the first parameter set to x, then the second parameter set to [y] (namely {[y]] without the surrounding begin-group / end-group tokens), and the [ is consumed, and all this is replaced with the sequence of tokens {, #, ], !, [y], [. The log file shows

\!!1#2[->{##]!!#2[
!1<-x
#2<-[y]

(not #1<-x as the exercise mentions: this is a bug either in TeX or in The TeXbook, not sure which one will DEK will consider it to be). Here, in this \tracingmacros output:

  • “Category codes are not shown, but a character of category 6 always appears twice in succession.” — For example, the ## and !! that are shown, each representing a single catcode-6 token with char code that of # and ! respectively.

  • “A parameter token in the replacement text uses the character code of the final parameter in the parameter text.” — This refers to the fact that the output parameter in the first log line (on the right-hand side of the -> in \!!1#2[->{##]!!#2[) is shown as #2 rather than !2. If we have \def\foo#1!2{#1#2} then we see \foo #1!2->!1!2, and if we have \def\foo!1#2{#1#2} then we see \foo !1#2->#1#2. That is, every parameter token shown in the replacement text by \tracingmacros uses the last catcode-6 character that was used for a parameter in the parameter-text part of the macro definition. [Edit: This interpretation was pointed out by the OP; removed incorrect explanation that was here.]

  • The answer to ex. 20.7 is now completely clear to me. The last sentence may be understood by comparing the following two examples: 1) \def\!!1#2![{!#]#!!2#2!1#1} gives #2 and #1 in all cases in the trace line with macro expansion, after ->; 2) \def\!!1!2![{!#]#!!2#2!1#1} gives !2 and !1 in all cases in the trace line with macro expansion, after ->. As for the trace lines with macro arguments, there is no precision in the answer about the charcode (BTW, I stumbled upon this discrepancy too, and reported it yesterday). – Igor Liferenko May 16 at 1:15
  • @IgorLiferenko Ah that makes more sense! So there is no mistake in that sentence after all. Your understanding fits the “replacement text” part of the sentence better. Thanks! – ShreevatsaR May 16 at 1:31
7

The exercise is about interpreting

\def\!!1#2![{!#]#!!2}

when [, ] and ! have category code 1, 2 and 6, respectively.

The paragraph before the quoted one reads

Incidentally, if you display this with \tracingmacros=1, TeX says

\!!1#2[->{##]!!#2[
 #1<-x
 #2<-[y]

The duplication of the category code 6 characters is evident in the top line: the expansion of \! is shown to be

{##]!!#2

which means that the input stream will receive

{1#6]2!6[1y11]2

and a left brace is expected.

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.