Allowing underscores in expl3

Question

It has been argued to me that it is dangerous to change the hyphen - as a normal character in style files because one might break a lot of code, negative numbers, overbar accents, table markup, ... all likely to break.

Doesn't the underscore with \ExplSyntaxOff also lead to similar problems ?

I am starting to think that originally disallowing underscores _ in names was in actuality a design mistake.

Answer 1

TeX sets the category code of a character when it is tokenized. That means that if we have code using a 'letter' from outside a-zA-Z, we only need to worry about what's happening where the catcode is set. Thus for example in expl3:

\cs_new:Npn \foo #1 { \__mypkg_internal_foo:n {#1} }

defines \foo, which will use \__mypkg_internal_foo:n even if we have changed the catcode of _.

The only issue in normal TeX usage with _ begin a 'letter' is that it's used for subscripts. So if we want to produce a subscript within expl3 code we need to 'generate' the correct character token: \char_generate:nn { `\_ } { 8 }

We could do the same with other chars: the obvious one is @, used by plain TeX and LaTeX2e. Using - is more 'interesting' as in an expression such as \count10=-5\relax, TeX requires - is 'other'. That is a real issue, and is one reason in expl3 we've not yet found a way to 'set' the catcode of all potentially-active chars.

Answer 2

Not really. The main two purposes of underscores in the LaTeX syntax are:

1. Math subscripts, e.g. \alpha_i
2. File address and URLs such as my_file.tex

As expl3 is mainly intended for actual programming, math content is rare, and in the few cases where you need to write something like α_i, you can type\alpha\sb{i} instead of \alpha_i. So the first use of underscores doesn’t really cause issues within expl3. And as far as I know, the second use of underscores is not affected by the changing of catcodes within expl3, so this doesn’t really cause issues either.