6

Page 213 of The TeXbook has a definitive explanation of \csname...\endcsname:

When TeX expands \csname it reads to the matching \endcsname, expanding tokens as it goes; only character tokens should remain after this expansion has taken place. […]

I noticed that active characters can be made unexpandable by giving a definition like

\chardef~="16 

Then ~ will stand for itself in commands like \message{~} and \edef\foo{~}. But why doesn't \csname~\endcsname work?

0
7

A \chardef token is not a character, but a command to print a character.

As such, it is not allowed inside \csname...\endcsname where only character tokens (after macro expansion) are permitted.

The fact that \edef\foo{~} yields ~ is because a \chardef token is unexpandable. Similarly for \message{~}.

The fact that TeX assigns special internal codes to active characters is irrelevant. The relevant aspect is whether an active character's definition is a macro or not. If it is a macro, it is expanded; otherwise it isn't and it's behavior depends on the context.

So, for instance, if you have

\chardef~="16
\csname\ifnum~="16 \string~\else foo\fi\endcsname

is perfectly good and would end up with \~. But it's not really "using ~ inside \csname...\endcsname, of course.

On the other hand, \& is defined by \chardef\&="26 and \& is not allowed ”naked” inside \csname...\endcsname, exactly like ~ would be if it's a \chardef token.

Another similar problem is with implicit character tokens. If you do

\let~=a

you're not allowed to use ~ inside \csname...\endcsname as well.

2
  • Characters and commands are not contradictory. Ordinary characters like L are also commands when passed to TeX’s “stomach”, according to page 267 of The TeXbook. – Hu Yajie Jan 7 at 13:29
  • @HuYajie The first sentence is about \chardef tokens not being character tokens. – egreg Jan 7 at 15:40
1

Page 47 of The TeXbook states that

If TeX sees a character of categories 1, 2, 3, 4, 6, 8, 11, 12, or 13, or a character of category 7 that is not the first of a special sequence as just described, it converts the character to a token by attaching the category code, and goes into state M.

but this is a white lie. Characters of category 13 are actually converted to a special kind of control sequences by the following code in tex.web:

353. ⟨ Process an active-character control sequence and set statemid_line 353 ⟩ ≡
  begin cur_cscur_chr + active_base; cur_cmdeq_type(cur_cs); cur_chrequiv(cur_cs);
  statemid_line;
  if cur_cmdouter_call then check_outer_validity;
  end

The variable cur_cs is 0 for character tokens, 1–256 for active characters of character code 0–255, and > 256 for real control sequences. Unfortunately, the code in ⟨ Manufacture a control sequence name 372 ⟩ only accepts tokens with cur_cs = 0 when collecting characters for the \csname...\endcsname construction.

\message{~} and \edef\foo{~} work essentially like \message{\$} and \edef\foo{\$}.

4
  • 2
    The last line seems wrong, \message{~} shows ~ as you say in the question (if it is a chardef token) but that is not essentially like \message{\undefined} (which gives an undefined command error) – David Carlisle Jan 7 at 17:09
  • @DavidCarlisle Sorry, fixed. I probably meant something like \unexpandable. – Hu Yajie Jan 8 at 2:41
  • The sentence you quoted from The TeXbook isn't a white lie as you describe it: active characters and even control sequences are tokens too (e.g. \relax is a single token), so the quoted sentence seems accurate (~ is a token, just one with nonzero cur_cs). I think you need to look elsewhere in the TeXbook to see what's documented about \csname … \endcsname . – ShreevatsaR Jan 8 at 3:14
  • @ShreevatsaR The white lie is the “attaching the category code” part. Active characters like ~ are stored as tokens, but not in the form of (character code, category code) pairs. – Hu Yajie Jan 8 at 6:05

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.