1

Is it possible to write a Latex document independent of the input language (Latin, Arabic, Japanese, Hindi)?

I though \usepackage[utf8]{inputenc} was there to do exactly that, but for instance Japanese would not appear when mixed with Latin characters. My current solution is importing the necessary language packages like \usepackage{xeCJK} for Japanese and \usepackage{CJK} for Chinese. Isn't there a way though to define UTF32 and not worry about all the language specific packages like xeCJK and CJK?

  • 3
    inputenc is for pdflatex, when using it with xetex it does nothing. Beside this: the problem is not the input but the output: the glyphs are in various fonts. – Ulrike Fischer Mar 31 '18 at 12:14
  • @UlrikeFischer Thanks for the hint about inputenc and pdflatex. What exactly did you mean with the glyphs within various fonts? Did you mean like for Japanese input to take \usepackage{xeCJK}? – Socrates Mar 31 '18 at 12:27
  • (nearly) no font has all glyphs. If you want to write english and japanese you normally will have to switch fonts. – Ulrike Fischer Mar 31 '18 at 12:29
  • 4
    This has nothing to do with utf8. You are still confusing input and output. You must tell xelatex which font to use. See e.g. here for similar discussions: tex.stackexchange.com/questions/323575/… – Ulrike Fischer Mar 31 '18 at 12:39
  • 1
    You presumably do not want to input utf32 input (very few editors support utf32) xetex and luatex (and pdftex with inputenc) support utf8 but whether or not there is a glyph is not related to the input encoding. – David Carlisle Mar 31 '18 at 13:29
6

XeTeX does have built in support for UTF-32 (and it could be coded in Lua for luatex)

For example

enter image description here

was produced from

\XeTeXinputencoding UTF-32
XXX\XXXdXXXoXXXcXXXuXXXmXXXeXXXnXXXtXXXcXXXlXXXaXXXsXXXsXXX{XXXaXXXrXXXtXXXiXXXcXXXlXXXeXXX}XXX\XXXbXXXeXXXgXXXiXXXnXXX{XXXdXXXoXXXcXXXuXXXmXXXeXXXnXXXtXXX}XXXhXXXeXXXlXXXlXXXoXXX XXXwXXXoXXXrXXXlXXXdXXX\XXXeXXXnXXXdXXX{XXXdXXXoXXXcXXXuXXXmXXXeXXXnXXXtXXX}

in which each X was replaced by a 0 byte, ascii null (control -@) (I had to make that replacement to X just to post to this site, as posting null bytes is tricky)

Very few editors support this format. even emacs only supports utf-8 and utf-16 by default, not utf-32.

In UTF-32 every character takes four bytes and so every ascii character is preceded by three zero bytes, this means that (in contrast to UTF-8) it is tricky to switch between UTF-8 and a legacy encoding as the string \XeTeXinputencoding is represented by a different byte sequence in UTF-32 than it is in UTF-8 or ASCII.

However UTF-8, UTF-16 and UTF-32 are file encodings for the same underlying Unicode character set so whichever of the encodings you use, the same characters can be represented. Whether that character is typeset depends not on the file encoding of the source but rather on the font being used. However you specify the code points for Chinese, if the current font is latin modern, you will get missing glyph warnings as Latin modern does not have those characters.

It is possible using fontspec to specify a different font to be used for different ranges of Unicode, but normally in LaTeX you need more than just a font switch, hyphenation and spacing (and other things) are language specific and best controlled by explicit commands in the document (which can then also select a suitable font) so a mechanism that just switches fonts when it finds a missing glyph would perhaps always produce poor output although at first it might seem an improvement on the current behaviour of no output at all.

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.