Why in the discussion of the definition of \newif are the following things true

Question

This is a couple of questions that stem from the answer to Problems understanding definition of \newif by @egreg.

In that @egreg says

Let's look at
\expandafter\expandafter\expandafter
\def\@if\iffoo{true}{\let\iffoo=\iftrue}%
recalling the definition of \@if:
\def\@if#1#2{\csname\expandafter\if@\string#1#2\endcsname}
The first \expandafter expands the third one, which in turn causes the expansion of \@if, which has two arguments; in this case they are
#1<-\iffoo
#2<-true

But from my understanding the way that \def works is that it is sensitive to the non-argument characters (i.e., characters other than #1,#2,…,#9) that follow the command's name, such that (to take from p202 of the TeXbook) if you have \def\cs #1. #2\par{…} then \cs will need arguments that are given in the format #1. #2 where you need a .␣ separating the two, such that

\cs You owe \$5.00. Pay it.\par  

gives

#1 <- You owe \$5.00 
#2 <- Pay it. 

Where (to quote the TeXbook)

the period in \5.00 doesn't stop #1, because TeX keeps going until finding a period that is immediately followed by a space.

And in the above \def\@if#1#2 is not \def\@if#1{#2}, and I would imagine that the latter wouldn't work since it would interpret #2 as the definition of \@if#1 thinking that \@if only took a single argument #1.

My second difficulty was in understanding the following section

So we're left with
\expandafter\def\csname\expandafter\if@\string\iffoo true\endcsname
Now the remaining \expandafter triggers the expansion of \csname that, I recall, does exhaustive expansion until finding the matching \endcsname. So, in order to know what control sequence name will be produced, we have to follow the expansion of all tokens after \csname.
The first token is \expandafter, that causes the expansion of \string; since \escapechar is -1, we're left with
\if@ iffootrue\endcsname

But I do not understand why \string has that effect both in the situation where \escapechar=-1 or where \escapechar has not been reassigned.

Acc. to the TeXbook

\string<token> TeX first reads the <token> without expansion. If a control sequence token appears, its \string expansion consists of the control sequence name (including \escapechar as an escape character, if the control sequence isn't simply an active character).

So I'm left to surmise that if it replaced \ before iffoo in \iffoo then

1. why does \string have that effect when \iffoo is a control sequence and so (acc. to my likely misunderstanding of the TeXbook) should just be turned into its name, which is \iffoo (or is it iffoo, in which case why did the \escapechar need to be changed). Or,
2. If \ was replaced as it is a "control sequence token" as opposed to a "control sequence", per se, where did the -1 go?

And finally, in the next section @egreg says

Because of its definition, \if@ removes i and f and leaves footrue; so we have
\def\footrue{\let\iffoo=\iftrue}

I believe that \if@ is defined in the following:

{\uccode‘1=‘i \uccode‘2=‘f \uppercase{\gdef\if@12{}}} % ‘if’ required

And I understand that the first part sets the uppercase value of 1 to be i and 2 to be f, but i do not understand why this section is enclosed in a separate set of brackets nor why this definition acts to remove i and f since it would seem to either be undefined if \uppercase was applied initially since \GDEF is not a valid command or would be expecting 1 and 2 as fixed arguments (not i and f) that it would eliminate (in that it returns an empty string).

I apologize if this seems entirely esoteric, but I am really trying to understand why these definitions work as they do, and I genuinely don't know where I'm going awry. I had originally intended to ask this in the comments of the original answer, but @egreg helpfully pointed out that my questions were extensive enough that comments were an inappropriate place for that discussion.

Answer 1

Saying \newif\iffoo generates the following list of tokens:

\count@\escapechar \escapechar\m@ne
\expandafter\expandafter\expandafter\def\@if\iffoo{true}{\let\iffoo=\iftrue}
\expandafter\expandafter\expandafter\def\@if\iffoo{false}{\let\iffoo=\iffalse}
\@if\iffoo{false}
\escapechar\count@

In what follows, spaces (or newlines) are added just for separating tokens; there is none in the definitions, so none is generated.

I recall also the definition of the auxiliary macros \@if

\def\@if#1#2{\csname\expandafter\if@\string#1#2\endcsname}

and \if@

{\uccode‘1=‘i \uccode‘2=‘f \uppercase{\gdef\if@12{}}} % ‘if’ required

What does \if@ do?

TeX opens a group, where new \uccode values for 1 and 2 are set. Then it executes \uppercase. The operation transforms character tokens in their uppercase equivalents, without changing category codes. The resulting token list is put back in the input stream and it is

\gdef\if@ if{}

where i and f have category code 12 (like 1 and 2). The space is there just for clarity, there's no space token. So the macro \if@ wants to see after it that pair of character tokens or an error is issued and swallows them producing nothing. Next the } restores the \uccode values of i and f.

Step 1

The assignments \count@\escapechar and \escapechar\m@ne are performed. In particular, the escapechar is set to –1, which means that \string\cs will expand to cs (each character with category code 12), because –1 is not a valid character code.

Step 2

\expandafter\expandafter\expandafter\def\@if\iffoo{true}{\let\iffoo=\iftrue}

TeX jumps over a token and expands the next one, which is \expandafter, so another token is jumped over and the next is expanded. It is \@if, which takes two arguments. The macro is not followed by {, so the first argument is the following (non space) token, which is \iffoo; a { follows, so the second argument is all tokens up to and including the matching }, so {true}. Recall that TeX strips off a pair of braces from an undelimited argument (and also from a delimited one, if this doesn't leave unbalanced braces). So argument #2 is true. Substitute \if@\iffoo{true} with the replacement text

\csname\expandafter\if@\string\iffoo true\endcsname

Again, the space is not there, I leave it just for separating tokens. There are ten tokens in this token list.

Step 3

Two \expandafter's are gone, the replacement text has been put in place of \if@ and its arguments, so we have

\expandafter\def\csname\expandafter\if@\string\iffoo true\endcsname{\let\iffoo=\iftrue}

TeX jumps over \def and expands \csname. This will ultimately expand to a symbolic token, that's in turn built up by expanding whatever comes up to the matching \endcsname

Step 4

Let's look what's the token list up to \endcsname:

\expandafter\if@\string\iffoo true

(the space is not there, remember). TeX jumps over \if@ and expands \string. This produces the five character tokens iffoo (because \escapechar is –1) all with category code 12. Thus we're left with

\if@ iffootrue

and \if@ is expanded. The required tokens are found and swallowed, so we get footrue. Now TeX finalizes the \csname...\endcsname construction, producing the token \footrue.

Step 5

The input stream has

\def\footrue{\let\iffoo\iftrue}

that's executed and removed.

Step 6

TeX now finds

\expandafter\expandafter\expandafter\def\@if\iffoo{false}{\let\iffoo=\iffalse}

and does exactly the same as before, but the final token list is

\def\foofalse{\let\iffoo\iffalse}

that's executed and removed.

Step 7

Now we have

\@if\iffoo{false}

that does the same as before, producing just

\let\iffoo\iffalse

so the conditional starts out false.

Step 8

The final tokens

\escapechar\count@

reset \escapechar to the value it had before starting \newif\iffoo.

Answer 2

working backwards:

{\uccode‘1=‘i \uccode‘2=‘f \uppercase{\gdef\if@12{}}} % ‘if’ required

\uppercase works on tokens and any token other than a character token is left unchanged. Character tokens produce new character tokens with the same catcode as before but with character code obtained by looking up the uccode (if that is non zero)

so \gdef uppercases to \gdef not to \GDEF and you get

\gdef\if@ if{}

which defines \if@ to take no argument and expand to nothing but gobble a catcode 12 i and f (or generate an error).

It's like this:

\catcode`i=12
\catcode`f=12
\def\if@if{}

except that doesn't work as the f in \def needs to be catcode 11 and as do the i and f in \if@

the construct is enclosed in brackets as you do not want the uppercase of 1 to be i for the rest of the document.

I'm not sure I understand your \string question. \string\iffoo makes the sequence of catcode 12 tokens \ i f f o o except that the first \ is whatever character is specified by \escapechar or nothing if that is not a legal character code, so as it is -1 here \string\iffoo produces iffoo.

Answer 3

The Knuth's macro \newif is very cryptic. I spent full page 402 in my "TeXbook naruby" with explanation of this. But we can ask if there isn't more straightforward and more understandable solution of this task with equivalent result. I mean that it exists. For example:

\def\newif#1{\expandafter\newifA\string#1\relax#1}
\edef\tmp{\string\if}
\expandafter\def\expandafter\newifA\tmp#1\relax#2{%
   \newifB#2{\csname#1true\endcsname}{\csname#1false\endcsname}%
}
\def\newifB#1#2#3{% #1=\iffoo  #2=\footrue  #3=\foofalse
   \let#1=\iffalse
   \expandafter\def#2{\let#1=\iftrue}%
   \expandafter\def#3{\let#1=\iffalse}%
}