TeX - LaTeX Stack Exchange is a question and answer site for users of TeX, LaTeX, ConTeXt, and related typesetting systems. It only takes a minute to sign up.
Sign up to join this community
Page 213 of The TeXbook has a definitive explanation of \csname...\endcsname:
When TeX expands
\csnameit reads to the matching\endcsname, expanding tokens as it goes; only character tokens should remain after this expansion has taken place. […]
I noticed that active characters can be made unexpandable by giving a definition like
\chardef~="16
Then ~ will stand for itself in commands like \message{~} and \edef\foo{~}. But why doesn't \csname~\endcsname work?
A \chardef token is not a character, but a command to print a character.
As such, it is not allowed inside \csname...\endcsname where only character tokens (after macro expansion) are permitted.
The fact that \edef\foo{~} yields ~ is because a \chardef token is unexpandable. Similarly for \message{~}.
The fact that TeX assigns special internal codes to active characters is irrelevant. The relevant aspect is whether an active character's definition is a macro or not. If it is a macro, it is expanded; otherwise it isn't and it's behavior depends on the context.
So, for instance, if you have
\chardef~="16
\csname\ifnum~="16 \string~\else foo\fi\endcsname
is perfectly good and would end up with \~. But it's not really "using ~ inside \csname...\endcsname, of course.
On the other hand, \& is defined by \chardef\&="26 and \& is not allowed ”naked” inside \csname...\endcsname, exactly like ~ would be if it's a \chardef token.
Another similar problem is with implicit character tokens. If you do
\let~=a
you're not allowed to use ~ inside \csname...\endcsname as well.
Page 47 of The TeXbook states that
If TeX sees a character of categories 1, 2, 3, 4, 6, 8, 11, 12, or 13, or a character of category 7 that is not the first of a special sequence as just described, it converts the character to a token by attaching the category code, and goes into state M.
but this is a white lie. Characters of category 13 are actually converted to a special kind of control sequences by the following code in tex.web:
353. ⟨ Process an active-character control sequence and set state ← mid_line 353 ⟩ ≡
begin cur_cs ← cur_chr + active_base; cur_cmd ← eq_type(cur_cs); cur_chr ← equiv(cur_cs);
state ← mid_line;
if cur_cmd ≥ outer_call then check_outer_validity;
end
The variable cur_cs is 0 for character tokens, 1–256 for active characters of character code 0–255, and > 256 for real control sequences. Unfortunately, the code in ⟨ Manufacture a control sequence name 372 ⟩ only accepts tokens with cur_cs = 0 when collecting characters for the \csname...\endcsname construction.
\message{~} and \edef\foo{~} work essentially like \message{\$} and \edef\foo{\$}.
\message{~} shows ~ as you say in the question (if it is a chardef token) but that is not essentially like \message{\undefined} (which gives an undefined command error)
– David Carlisle
Jan 7 at 17:09
\unexpandable.
– Hu Yajie
Jan 8 at 2:41
\relax is a single token), so the quoted sentence seems accurate (~ is a token, just one with nonzero cur_cs). I think you need to look elsewhere in the TeXbook to see what's documented about \csname … \endcsname .
– ShreevatsaR
Jan 8 at 3:14
~ are stored as tokens, but not in the form of (character code, category code) pairs.
– Hu Yajie
Jan 8 at 6:05
