2

This question is about categories of explicit character tokens.

If you apply \string to an explicit character token, you get the category 12 pendant of that explicit character token.

(Exception: If the character code of the character token is 32, which denotes a space character, then you get the category 10 pendant of that explicit character token.)

So \string can also be seen as a means of expandably obtaining the category-12-pendant of an arbitrary explicit character token (whose character code is not 32).

Are there means for expandably obtaining pendants of other categories of an arbitrary explicit character token?

So that, e.g., you can easily turn an explicit opening curly brace { of category 12 into one of category 1?
So that, e.g., you can easily turn an explicit closing curly brace } of category 12 into one of category 2?
So that, e.g., you can easily turn an explicit hash # of category 6 into one of category 3?

If not: What is the reason for not providing such means?
Would such means allow to run into ambiguities/problems/troubles, probably distorting fundamental concepts of TeX?

(If such means are implemented, probably a test goes along with them for finding out if a token is an explicit character token at all?)

5
  • Are you using LaTeX? There's \char_generate:nn (implemented with \Ucharcat in XeTeX, read source code for other implementations)
    – user202729
    Commented Mar 27, 2022 at 15:46
  • Remark, most answers to "why TeX does not have feature X" are "stability and historical reason".
    – user202729
    Commented Mar 27, 2022 at 15:46
  • I add this one into the "common idiom list" tex.stackexchange.com/a/638529/250119 (disclaimer, my answer)
    – user202729
    Commented Mar 27, 2022 at 16:14
  • Character tokens can only have one of the categories 1, 2, 3, 4, 6, 7, 8, 10, 11, 12, 13. Implementing a check for an explicit character token is hard/not feasible in a way which is 100% reliable. Implementing a check whether an arbitrary argument forms a valid TeX-<number>-quantity denoting one of the values 1, 2, 3, 4, 6, 7, 8, 10, 11, 12, 13 also is hard/not feasible in a way which is 100% reliable. So the question arises: To what extend do you need to have cranked out specifications which don't make sense? Commented Mar 27, 2022 at 16:44
  • @UlrichDiez No idea, it should be (not that I tested it exhaustively though). File bug report or ask the team, then. (if it's not expandable then isn't it mostly useless though?)
    – user202729
    Commented Mar 27, 2022 at 17:19

1 Answer 1

1

There is ambiguos behavior of categories which are interpreted at token processor level. What happens when you want to have category 0 or 5 or 14 created by your expandable macro?

But you can do this (for other "normal" categories) by LuaTeX extensions:

\def\tokencatcode#1#2{%
   \immediateassignment\edef\tmp{\the\catcode`#1}%
   \immediateassignment\catcode`#1=#2
   \scantextokens\expandafter{\csstring#1}%
   \immediateassignment\catcode`#1=\tmp\space
}

$a\tokencatcode !{7}2 + b^2 = c^2$

\message{.......\tokencatcode !{7}} % yes, it is expandable

\bye

or more simple is to do it with \directlua:

\def\tokencatcode#1#2{\directlua{tex.cprint(#2,"\csstring#1")}}

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .