8

This is a couple of questions that stem from the answer to Problems understanding definition of \newif by @egreg.

In that @egreg says

Let's look at
\expandafter\expandafter\expandafter
\def\@if\iffoo{true}{\let\iffoo=\iftrue}%
recalling the definition of \@if:
\def\@if#1#2{\csname\expandafter\if@\string#1#2\endcsname}
The first \expandafter expands the third one, which in turn causes the expansion of \@if, which has two arguments; in this case they are
#1<-\iffoo
#2<-true

But from my understanding the way that \def works is that it is sensitive to the non-argument characters (i.e., characters other than #1,#2,…,#9) that follow the command's name, such that (to take from p202 of the TeXbook) if you have \def\cs #1. #2\par{…} then \cs will need arguments that are given in the format #1. #2 where you need a .␣ separating the two, such that

\cs You owe \$5.00. Pay it.\par  

gives

#1 <- You owe \$5.00 
#2 <- Pay it. 

Where (to quote the TeXbook)

the period in \5.00 doesn't stop #1, because TeX keeps going until finding a period that is immediately followed by a space.

And in the above \def\@if#1#2 is not \def\@if#1{#2}, and I would imagine that the latter wouldn't work since it would interpret #2 as the definition of \@if#1 thinking that \@if only took a single argument #1.

My second difficulty was in understanding the following section

So we're left with
\expandafter\def\csname\expandafter\if@\string\iffoo true\endcsname
Now the remaining \expandafter triggers the expansion of \csname that, I recall, does exhaustive expansion until finding the matching \endcsname. So, in order to know what control sequence name will be produced, we have to follow the expansion of all tokens after \csname.
The first token is \expandafter, that causes the expansion of \string; since \escapechar is -1, we're left with
\if@ iffootrue\endcsname

But I do not understand why \string has that effect both in the situation where \escapechar=-1 or where \escapechar has not been reassigned.

Acc. to the TeXbook

\string<token> TeX first reads the <token> without expansion. If a control sequence token appears, its \string expansion consists of the control sequence name (including \escapechar as an escape character, if the control sequence isn't simply an active character).

So I'm left to surmise that if it replaced \ before iffoo in \iffoo then

  1. why does \string have that effect when \iffoo is a control sequence and so (acc. to my likely misunderstanding of the TeXbook) should just be turned into its name, which is \iffoo (or is it iffoo, in which case why did the \escapechar need to be changed). Or,
  2. If \ was replaced as it is a "control sequence token" as opposed to a "control sequence", per se, where did the -1 go?

And finally, in the next section @egreg says

Because of its definition, \if@ removes i and f and leaves footrue; so we have
\def\footrue{\let\iffoo=\iftrue}

I believe that \if@ is defined in the following:

{\uccode‘1=‘i \uccode‘2=‘f \uppercase{\gdef\if@12{}}} % ‘if’ required

And I understand that the first part sets the uppercase value of 1 to be i and 2 to be f, but i do not understand why this section is enclosed in a separate set of brackets nor why this definition acts to remove i and f since it would seem to either be undefined if \uppercase was applied initially since \GDEF is not a valid command or would be expecting 1 and 2 as fixed arguments (not i and f) that it would eliminate (in that it returns an empty string).

I apologize if this seems entirely esoteric, but I am really trying to understand why these definitions work as they do, and I genuinely don't know where I'm going awry. I had originally intended to ask this in the comments of the original answer, but @egreg helpfully pointed out that my questions were extensive enough that comments were an inappropriate place for that discussion.

5

Saying \newif\iffoo generates the following list of tokens:

\count@\escapechar \escapechar\m@ne
\expandafter\expandafter\expandafter\def\@if\iffoo{true}{\let\iffoo=\iftrue}
\expandafter\expandafter\expandafter\def\@if\iffoo{false}{\let\iffoo=\iffalse}
\@if\iffoo{false}
\escapechar\count@

In what follows, spaces (or newlines) are added just for separating tokens; there is none in the definitions, so none is generated.

I recall also the definition of the auxiliary macros \@if

\def\@if#1#2{\csname\expandafter\if@\string#1#2\endcsname}

and \if@

{\uccode‘1=‘i \uccode‘2=‘f \uppercase{\gdef\if@12{}}} % ‘if’ required

What does \if@ do?

TeX opens a group, where new \uccode values for 1 and 2 are set. Then it executes \uppercase. The operation transforms character tokens in their uppercase equivalents, without changing category codes. The resulting token list is put back in the input stream and it is

\gdef\if@ if{}

where i and f have category code 12 (like 1 and 2). The space is there just for clarity, there's no space token. So the macro \if@ wants to see after it that pair of character tokens or an error is issued and swallows them producing nothing. Next the } restores the \uccode values of i and f.

Step 1

The assignments \count@\escapechar and \escapechar\m@ne are performed. In particular, the escapechar is set to –1, which means that \string\cs will expand to cs (each character with category code 12), because –1 is not a valid character code.

Step 2

\expandafter\expandafter\expandafter\def\@if\iffoo{true}{\let\iffoo=\iftrue}

TeX jumps over a token and expands the next one, which is \expandafter, so another token is jumped over and the next is expanded. It is \@if, which takes two arguments. The macro is not followed by {, so the first argument is the following (non space) token, which is \iffoo; a { follows, so the second argument is all tokens up to and including the matching }, so {true}. Recall that TeX strips off a pair of braces from an undelimited argument (and also from a delimited one, if this doesn't leave unbalanced braces). So argument #2 is true. Substitute \if@\iffoo{true} with the replacement text

\csname\expandafter\if@\string\iffoo true\endcsname

Again, the space is not there, I leave it just for separating tokens. There are ten tokens in this token list.

Step 3

Two \expandafter's are gone, the replacement text has been put in place of \if@ and its arguments, so we have

\expandafter\def\csname\expandafter\if@\string\iffoo true\endcsname{\let\iffoo=\iftrue}

TeX jumps over \def and expands \csname. This will ultimately expand to a symbolic token, that's in turn built up by expanding whatever comes up to the matching \endcsname

Step 4

Let's look what's the token list up to \endcsname:

\expandafter\if@\string\iffoo true

(the space is not there, remember). TeX jumps over \if@ and expands \string. This produces the five character tokens iffoo (because \escapechar is –1) all with category code 12. Thus we're left with

\if@ iffootrue

and \if@ is expanded. The required tokens are found and swallowed, so we get footrue. Now TeX finalizes the \csname...\endcsname construction, producing the token \footrue.

Step 5

The input stream has

\def\footrue{\let\iffoo\iftrue}

that's executed and removed.

Step 6

TeX now finds

\expandafter\expandafter\expandafter\def\@if\iffoo{false}{\let\iffoo=\iffalse}

and does exactly the same as before, but the final token list is

\def\foofalse{\let\iffoo\iffalse}

that's executed and removed.

Step 7

Now we have

\@if\iffoo{false}

that does the same as before, producing just

\let\iffoo\iffalse

so the conditional starts out false.

Step 8

The final tokens

\escapechar\count@

reset \escapechar to the value it had before starting \newif\iffoo.

  • Step 2 was absolutely crucial. Did not know that the brackets would just be swallowed in that way and that what might be thought of as the output of the command sequence is delimited with {} and that arguments that are expected can also use {} to group characters to input as that argument. In Step 4, is the \ at the front of footrue due to the way that \csname works since it is meant to produce a valid control sequence as its output? And in steps 1 and 8, count@ is just acting as a holding register for whatever charcode \escapechar has before running the code. – mpacer Dec 4 '14 at 1:00
  • @mpacer in step 4 there isn't a \ in the expansion of \csname it is the single token with name footrue which is conventionally written as \footrue but it is not 8 tokens starting \ it is 1. – David Carlisle Dec 4 '14 at 1:17
  • @mpacer You need to remember that while we tend to write \foo it means 'the control sequence foo': the escape char is needed as part of the input but isn't part of the name. Try setting \escapechar to something else to see this, for example: \def\foo{Definition}\def\firstofone#1{#1}\begingroup\firstofone{\escapechar`\a\show\foo\endgroup}\bye and see what the log shows! – Joseph Wright Dec 4 '14 at 8:33
9

working backwards:

{\uccode‘1=‘i \uccode‘2=‘f \uppercase{\gdef\if@12{}}} % ‘if’ required

\uppercase works on tokens and any token other than a character token is left unchanged. Character tokens produce new character tokens with the same catcode as before but with character code obtained by looking up the uccode (if that is non zero)

so \gdef uppercases to \gdef not to \GDEF and you get

\gdef\if@ if{}

which defines \if@ to take no argument and expand to nothing but gobble a catcode 12 i and f (or generate an error).

It's like this:

\catcode`i=12
\catcode`f=12
\def\if@if{}

except that doesn't work as the f in \def needs to be catcode 11 and as do the i and f in \if@

the construct is enclosed in brackets as you do not want the uppercase of 1 to be i for the rest of the document.

I'm not sure I understand your \string question. \string\iffoo makes the sequence of catcode 12 tokens \ i f f o o except that the first \ is whatever character is specified by \escapechar or nothing if that is not a legal character code, so as it is -1 here \string\iffoo produces iffoo.

  • I apologize if this is me just being dense, but why isn't \gdef equivalent to the catcode 0 token \ followed by catcode 11 tokens g d e f? and so \uppercase, when it applies to those character tokens, why does it not transform those character tokens to G D E F? And the gobbling you describe I think is what I meant by i and f being fixed arguments, but I'm still confused. Normally you are using \newif to pass a command with catcode 11 tokens(letters), how does it know to treat the catcode 11 tokens passed into it as valid catcode 12 tokens to be gobbled up? – mpacer Dec 4 '14 at 0:24
  • @mpacer \gdef is one token a command token with name gdef the argument to \newif isn't catcode 11 letters it is again a single token such as \ifwibble \string takes a token and always makes a sequence of catcode 12 tokens (or catcode 10 in case of space), so \ifwibble is one token \string\wibble expands (if escapechar is -1) to 6 catcode 12 character tokens – David Carlisle Dec 4 '14 at 0:27
  • Oi. Where in the TeXbook is this explained? I thought i'd read this stuff pretty thoroughly, which is why I thought that \string would treat \foo as a single token, but then it didn't make sense that it could treat the \ separately. And the definition of \string doesn't say anything about it changing it into catcode 12 tokens, so I'm not sure where that's established… And just to make sure I understand, setting \escapechar to -1 makes its \charcode not within the 0-255 valid codes, and it doesn't modularize it to make -1 → 255? – mpacer Dec 4 '14 at 0:35
  • Just found page 40 in the TeXbook… "Each character in this token list automatically gets category code 12("other"). Including the backslash that \string inserts to represent an escape character." And \string looks for what to represent the escape character as and then the nonexistence of \charcode of \m@ne(or -1) in the tables means that it treats it as an ignored character (or catcode 9, <null>). Can't find a reference for that last bit, but I think that fits… – mpacer Dec 4 '14 at 0:50
  • @mpacer it doesn't make a catcode 9 character it simply makes nothing (sometimes tex-the-program is more informative than the texbook:-=) – David Carlisle Dec 4 '14 at 1:10
3

The Knuth's macro \newif is very cryptic. I spent full page 402 in my "TeXbook naruby" with explanation of this. But we can ask if there isn't more straightforward and more understandable solution of this task with equivalent result. I mean that it exists. For example:

\def\newif#1{\expandafter\newifA\string#1\relax#1}
\edef\tmp{\string\if}
\expandafter\def\expandafter\newifA\tmp#1\relax#2{%
   \newifB#2{\csname#1true\endcsname}{\csname#1false\endcsname}%
}
\def\newifB#1#2#3{% #1=\iffoo  #2=\footrue  #3=\foofalse
   \let#1=\iffalse
   \expandafter\def#2{\let#1=\iftrue}%
   \expandafter\def#3{\let#1=\iffalse}%
}
  • 1
    While I'd agree Knuth's definition is odd, I'm not sure this answers the question in hand (it would be fine for 'Is there a simpler alternative to Knuth's definition of \newif?' for example). Perhaps ask a separate question and answer it yourself? – Joseph Wright Dec 4 '14 at 9:24
  • 1
    I did mean (from the name of this thread) that it is the "discussion about \newif" and my answer is new point to such discussion. I see that debaters are drown in the problem of cryptic macro and that they are unable to see the problem from another point of view. My answer shows that the Knuth's macros is not good educational material about macro programming. Many pieces of TeX macro code (especially from LaTeX) are very cryptic with redundant code and this makes (unfortunately) TeX as less favourable for many users. – wipet Dec 4 '14 at 10:13
  • This fails if \escapechar is not 92 at the time of the \newif. – egreg Dec 4 '14 at 11:07
  • There is also a simpler implementation of newif in latex. – David Carlisle Dec 4 '14 at 11:17
  • LaTeX allows to use \newif\foo without error message which defines \otrue and \ofalse. It is not full optimal. – wipet Dec 4 '14 at 12:38

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.