# Do XeTeX and LuaTeX always use Unicode?

I am writing some functions to manipulate strings. A typical way to produce a character with a given character code is the \lowercase primitive. The following code produces a character with code 234, and displays it to the terminal.

\lccode*=234
\lowercase{\message{*}}


If I replace 234 by 345 above, then pdfTeX raises an error: Invalid code (345), should be in the range 0..255. On the other hand, LuaTeX and XeTeX are happy up to 1114111, as expected from a Unicode-compliant engine.

Now, I want my macros to be as robust as possible. Is it always true that the maximum lccode is 255 in pdfTeX and 1114111 in LuaTeX and XeTeX? Or can a user prevent in any way the engine from using the full range, hence confusing my macros?

-
AFAIK, XeTeX and LuaTeX can use a different input encoding. But the two engines both always use Unicode internally. –  Leo Liu Nov 25 '11 at 22:21
I'd say yes. Why shouldn't it be true? As Leo Liu remarks, there's \XeTeXinputencoding that does a conversion of the input into Unicode, but that's all; when a character enters XeTeX it's changed into its Unicode correspondent in a way similar to the ^^ and ^^^^ conventions: the mouth will act on the changed character and so \lowercase and \uppercase need the whole set. –  egreg Nov 25 '11 at 22:35
@LeoLiu: that's also my thought, but could there be some command-line switch? (I'm thinking of the difference between pdftex and pdfetex, where the former disables some commands.) –  Bruno Le Floch Nov 25 '11 at 23:19
I've no idea. You'd better ask the maintainers. –  Leo Liu Nov 26 '11 at 7:22
The luatex manual says quite clearly that \char now accepts values between 0 and 1114111 and extends this statement to the other similar commands like \lccode. As far as I know this is true in xetex too. (But the etex extension can be disabled in both engines). –  Ulrike Fischer Nov 26 '11 at 9:23

The LuaTeX manual says quite clearly that \char now accepts values between 0 and 1114111 and extends this statement to the other similar commands like \lccode. As far as I know this is true in XeTeX too. And IMHO a command line to change this seems rather senseless.

But the eTeX extension can be disabled in both engines. Or more precisely: XeTeX has a command line option to enable the eTeX extension (which is used by default by all TeX Systems) and LuaTeX has a similar feature. So it is possible to build a format manually which doesn't use them. But this affects only the eTeX relevant commands.

-

I had a lot of 'old' documents and I didn't want to change to utf-8.

I used the following code in my preamble to use the 'old' encoding:

\ifxetex
\XeTeXinputencoding "cp1252" % actual document
\XeTeXdefaultencoding "cp1252" % sequent documents (\input)
\fi
\usepackage[latin1]{luainputenc}


I don't know if this will make problems with your code - but you can try it.

-
-1. I'm sure that Bruno knows that input encodings can be changed. But the question is about internal encodings. \XeTeXinputencoding does nothing with \lccode. Moreover, \XeTeXdefaultencoding "cp1252"` is dangerous because LaTeX itself writes some auxiliary files and they're always UTF-8 files. BTW, UTF-8 is compatible with Latin-1 encoding, but Latin-1 and cp1252 are different, cp1252 is superset of latin-1. –  Leo Liu Nov 26 '11 at 7:25
@LeoLiu: you were maybe a little bit harsh. I indeed knew/expected that XeTeX would be able to somehow change input encodings, and here knut gives me a precise example of use. So the answer is off topic, but still useful for me. –  Bruno Le Floch Nov 26 '11 at 8:46
@Leo - A quibble, I'm not sure what you wrote is what you meant to write: latin-1 is not compatible with utf-8. For example, the hex-encoded two-byte string C3 B6 corresponds to the character ö in utf-8 and the two characters Ã¶ in latin-1. –  Charles Stewart Nov 28 '11 at 20:42
@Charles: I was wrong. ASCII is compatible with UTF-8. Say, UTF-8 is superset of ASCII. –  Leo Liu Nov 29 '11 at 1:48