# Why don't unexpandable active characters work in \csname…\endcsname?

Page 213 of The TeXbook has a definitive explanation of `\csname...\endcsname`:

When TeX expands `\csname` it reads to the matching `\endcsname`, expanding tokens as it goes; only character tokens should remain after this expansion has taken place. […]

I noticed that active characters can be made unexpandable by giving a definition like

``````\chardef~="16
``````

Then `~` will stand for itself in commands like `\message{~}` and `\edef\foo{~}`. But why doesn't `\csname~\endcsname` work?

A `\chardef` token is not a character, but a command to print a character.

As such, it is not allowed inside `\csname...\endcsname` where only character tokens (after macro expansion) are permitted.

The fact that `\edef\foo{~}` yields `~` is because a `\chardef` token is unexpandable. Similarly for `\message{~}`.

The fact that TeX assigns special internal codes to active characters is irrelevant. The relevant aspect is whether an active character's definition is a macro or not. If it is a macro, it is expanded; otherwise it isn't and it's behavior depends on the context.

So, for instance, if you have

``````\chardef~="16
\csname\ifnum~="16 \string~\else foo\fi\endcsname
``````

is perfectly good and would end up with `\~`. But it's not really "using `~` inside `\csname...\endcsname`, of course.

On the other hand, `\&` is defined by `\chardef\&="26` and `\&` is not allowed ”naked” inside `\csname...\endcsname`, exactly like `~` would be if it's a `\chardef` token.

Another similar problem is with implicit character tokens. If you do

``````\let~=a
``````

you're not allowed to use `~` inside `\csname...\endcsname` as well.

• Characters and commands are not contradictory. Ordinary characters like `L` are also commands when passed to TeX’s “stomach”, according to page 267 of The TeXbook. – Hu Yajie Jan 7 at 13:29
• @HuYajie The first sentence is about `\chardef` tokens not being character tokens. – egreg Jan 7 at 15:40

Page 47 of The TeXbook states that

If TeX sees a character of categories 1, 2, 3, 4, 6, 8, 11, 12, or 13, or a character of category 7 that is not the first of a special sequence as just described, it converts the character to a token by attaching the category code, and goes into state M.

but this is a white lie. Characters of category 13 are actually converted to a special kind of control sequences by the following code in tex.web:

353. ⟨ Process an active-character control sequence and set statemid_line 353 ⟩ ≡
begin cur_cscur_chr + active_base; cur_cmdeq_type(cur_cs); cur_chrequiv(cur_cs);
statemid_line;
if cur_cmdouter_call then check_outer_validity;
end

The variable cur_cs is 0 for character tokens, 1–256 for active characters of character code 0–255, and > 256 for real control sequences. Unfortunately, the code in ⟨ Manufacture a control sequence name 372 ⟩ only accepts tokens with cur_cs = 0 when collecting characters for the `\csname...\endcsname` construction.

`\message{~}` and `\edef\foo{~}` work essentially like `\message{\\$}` and `\edef\foo{\\$}`.

• The last line seems wrong, `\message{~}` shows ~ as you say in the question (if it is a chardef token) but that is not essentially like \message{\undefined} (which gives an undefined command error) – David Carlisle Jan 7 at 17:09
• @DavidCarlisle Sorry, fixed. I probably meant something like `\unexpandable`. – Hu Yajie Jan 8 at 2:41
• The sentence you quoted from The TeXbook isn't a white lie as you describe it: active characters and even control sequences are tokens too (e.g. `\relax` is a single token), so the quoted sentence seems accurate (`~` is a token, just one with nonzero `cur_cs`). I think you need to look elsewhere in the TeXbook to see what's documented about `\csname … \endcsname` . – ShreevatsaR Jan 8 at 3:14
• @ShreevatsaR The white lie is the “attaching the category code” part. Active characters like `~` are stored as tokens, but not in the form of (character code, category code) pairs. – Hu Yajie Jan 8 at 6:05