I'm currently analyzing and evaluating a migration process from pdfTeX to LuaTeX for our current TeX workflow. With LuaTeX being a fork of pdfTeX and packages like luainputenc at hand it seemed promising to reproduce pdfTeX's output using LuaTeX when sacrificing (some or most of) LuaTeX's new features.

Currently, however, I'm stuck and need your help to decide whether it's worth digging deeper or accepting what I found out. There are two problems I'm facing.

Here's the first problem. Engines: pdfTeX 3.1415926-2.4-1.40.13 and LuaTeX beta-0.70.2-2012052410 (both from TeX Live 2012) with --output-format=pdf. When using UTF-8 input ([utf8x]{inputenc} for pdfTeX, [utf8x]{luainputenc} for LuaTeX) and T1 encoded fonts ([T1]{fontenc}) the output from both engines differs for some non-T1 characters.


Some T1 characters:     & \# & \$ & \% & Ă & Ň & § & @ & Æ & ß   & £   \\[1.5mm]
Some non-T1 characters: & ‡  & ÿ  & ‰  & … & ¶ & ½ & ĩ & µ & | | & | | \\

Notes: luainputenc is forwarding to inputenc if called from pdfTeX. The output is the same when using an \ifluatex and loading inputenc and luainputenc separetely. The spaces between the bars in the last two columns of the second row are supposed to be a Unicode NO-BREAK SPACE (U+00A0) and a THIN SPACE (U+2009).

pdfTeX output: (everything fine here)

LuaTeX output: (notice the last five columns of the second row)

This problem does most certainly exist for other characters as well, these are just some I encountered.

Is there a way to get the pdfTeX output from LuaTeX? The problem is the same with different T1 encoded fonts, try mathpazo if you want. Using a different font encoding is not an option here as the task at hand is to migrate TeX engines, not fonts or their encodings. Am I perhaps missing a pdfTeX package that has to be replaced for LuaTeX usage?

The lutf8x (notice the leading L) luainputenc package option does not help here either. (Which is no suprise as its purpose is different.) What I'm trying to use here, according to its manual, is the "UTF-8 legacy mode" of luainputenc, i.e. mimicing the behavior of inputenc in pdfTeX by making non-ASCII characters active to determine the correct bit length of characters and so on.

Maybe the problem is not the input side (afaik luainputenc's job of translating input bytes into LICR) but the output side (afaik translating LICR into glyph positions of the font used)? Maybe LuaTeX is just translating to Unicode positions, not regarding glyph positions of the font, like the EU2 encoding?

Anyway, where does the problem come from? Can it be helped and if so, how?

Btw: It works fine with LuaTeX when using an OpenType-Font with fontspec and EU2 encoding, but that's not the primary goal here.

And here's the second problem. It's closely tied to the first one and can be reproduced using the MWE above. When using --output-format=dvi pdfTeX is producing a DVI file that dvips has no problem with. When using a LuaTeX DVI file dvips stops with something like

This is dvips(k) 5.992 Copyright 2012 Radical Eye Software (www.radicaleye.com)
' LuaTeX output 2012.08.01:1700' -> ohnexml-luatex.ps
dvips: ! invalid char 297 from font ec-lmr10

This is how I actually came up with the suspicion that for the first problem Unicode positions and not font encoding specific glyph positions are written to the output file as 297 is the decimal Unicode point of ĩ (second row, fourth last column in the MWE above).

If a solution to the first problem does not solve this problem transitively, how can this one be helped?

Thank you for your thoughts.

  • 2
    if i remember correctly, taco hoekwater, at a tug meeting sometime during the last few years, said that exact output equivalence to dek-tex was not a requirement for luatex. by this he was referring to line and page breaks, not to character presentation (where identical results seem to me to be highly desirable). but if absolutely identical results are a requirement of your project, you may want to take this into consideration. (i can't find a written record; this may have been in reply to a personal question.) – barbara beeton Aug 1 '12 at 18:06

At first you should imho better use utf8 instead of utf8x. utf8x is unmaintained and has problems e.g. with biblatex. (You will have to set up the some missing definitions for pdflatex). You will also have to add some definitions for lualatex as it will map - as you already found out - undeclared chars simply to their unicode position. Here e.g. two definitions for ½ & µ:

Some T1 characters:     & \# & \$ & \% & Ă & Ň & § & @ & Æ & ß   & £   \\[1.5mm]
Some non-T1 characters: & ‡  & ÿ  & ‰  & … & ¶ & ½ & µ %ĩ &  & | | & | | \\    \end{tabular}
  • Thank you for your answer Ulrike. I must admit I wasn't aware of \DeclareUnicodeCharacter. Can you by chance tell me where the different behavior of pdfTeX/LuaTeX comes from? Is it just that utf8x.def is unmaintained and thus behaves so much different in inputenc and luainputenc? I found the ucharacters.sty from the PassiveTeX project to be quite a comprehensive list of such character declarations, it can be used by defining e.g. \newcommand{\DefineCharacter}[3]{\DeclareUnicodeCharacter{#2}{#3}}. Your answer solves the DVI mode problem as well. – Patrick Bergner Aug 1 '12 at 16:36
  • 3
    Well pdftex/luatex are simply different. pdftex reads an utf8 char in pieces (each 8 bit long), luatex reads it as one entity. So if a char is not declared pdftex will give an error at the first octet, but luatex will simply pass it through. Beside this: as far as I can see luainputenc predeclares less chars then is done by normal inputenc. This could probably been improved but on the other side luainputenc is not much used. Most people that use lualatex uses also fontspec. – Ulrike Fischer Aug 1 '12 at 16:54

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.