TeX - LaTeX Stack Exchange is a question and answer site for users of TeX, LaTeX, ConTeXt, and related typesetting systems. It only takes a minute to sign up.
Sign up to join this community
Page 213 of The TeXbook has a definitive explanation of \csname...\endcsname:
When TeX expands
\csnameit reads to the matching\endcsname, expanding tokens as it goes; only character tokens should remain after this expansion has taken place. […]
I noticed that active characters can be made unexpandable by giving a definition like
\chardef~="16
Then ~ will stand for itself in commands like \message{~} and \edef\foo{~}. But why doesn't \csname~\endcsname work?
A \chardef token is not a character, but a command to print a character.
As such, it is not allowed inside \csname...\endcsname where only character tokens (after macro expansion) are permitted.
The fact that \edef\foo{~} yields ~ is because a \chardef token is unexpandable. Similarly for \message{~}.
The fact that TeX assigns special internal codes to active characters is irrelevant. The relevant aspect is whether an active character's definition is a macro or not. If it is a macro, it is expanded; otherwise it isn't and it's behavior depends on the context.
So, for instance, if you have
\chardef~="16
\csname\ifnum~="16 \string~\else foo\fi\endcsname
is perfectly good and would end up with \~. But it's not really "using ~ inside \csname...\endcsname, of course.
On the other hand, \& is defined by \chardef\&="26 and \& is not allowed ”naked” inside \csname...\endcsname, exactly like ~ would be if it's a \chardef token.
Another similar problem is with implicit character tokens. If you do
\let~=a
you're not allowed to use ~ inside \csname...\endcsname as well.
L are also commands when passed to TeX’s “stomach”, according to page 267 of The TeXbook.
\chardef tokens not being character tokens.
Page 47 of The TeXbook states that
If TeX sees a character of categories 1, 2, 3, 4, 6, 8, 11, 12, or 13, or a character of category 7 that is not the first of a special sequence as just described, it converts the character to a token by attaching the category code, and goes into state M.
but this is a white lie. Characters of category 13 are actually converted to a special kind of control sequences by the following code in tex.web:
353. ⟨ Process an active-character control sequence and set state ← mid_line 353 ⟩ ≡
begin cur_cs ← cur_chr + active_base; cur_cmd ← eq_type(cur_cs); cur_chr ← equiv(cur_cs);
state ← mid_line;
if cur_cmd ≥ outer_call then check_outer_validity;
end
The variable cur_cs is 0 for character tokens, 1–256 for active characters of character code 0–255, and > 256 for real control sequences. Unfortunately, the code in ⟨ Manufacture a control sequence name 372 ⟩ only accepts tokens with cur_cs = 0 when collecting characters for the \csname...\endcsname construction.
\message{~} and \edef\foo{~} work essentially like \message{\$} and \edef\foo{\$}.
\message{~} shows ~ as you say in the question (if it is a chardef token) but that is not essentially like \message{\undefined} (which gives an undefined command error)
Jan 7, 2021 at 17:09
\unexpandable.
\relax is a single token), so the quoted sentence seems accurate (~ is a token, just one with nonzero cur_cs). I think you need to look elsewhere in the TeXbook to see what's documented about \csname … \endcsname .
Jan 8, 2021 at 3:14
~ are stored as tokens, but not in the form of (character code, category code) pairs.