9

TeX Live 2015 ships with LuaTeX 0.80. As LuaTeX 0.85 became available last month and I am supposed to do be doing other things I found this to be a wonderful opportunity to procrastinate by giving the 0.85 version a try.

Building the executable was unproblematic. However to avoid a

(Fatal format file error; I'm stymied)

error, the format files had to be regenerated.

fmtutils-sys --all

however failed to regenrate the LuaTeX related format files. Doing it manually via:

luatex -ini   -jobname=luatex -progname=luaex luatex.ini

revealed that the very first line fails:

! Undefined control sequence.
l.4 \pdfoutput
            =1
? 
! Emergency stop.

Which is quite bad, because \pdfoutput is primitive control sequence of pdfTeX, it is always there, out of the box. But not any more in LuaTeX 0.85 it seems. In fact section 8.1.7 of the LuaTeX manual explains that all but three of the \pdf* primitives are gone.

In the same section the manual provides a list of commands to get the traditional pdfTeX \pdf*-primitives back. However the use of \protected is only possible if the eTeX primitives in LuaTeX have been enabled. That happens in

/usr/local/texlive/2015/texmf-dist/tex/generic/config/luatexiniconfig.tex

which is the first thing that

/usr/local/texlive/2015/texmf-dist/tex/plain/config/luatex.ini

loads. Extending luatexiniconfig.tex with list of commands almost succeeds. The last commands on the list use \newdimen which is not a primitive. It is defined in the plain-TeX-Format.

As luatex.ini loads pdfetex.ini which in turn loads etex.src which loads plain.src the \newdimen command sequence will not be available until after all the files have been processed. However pdfetex.ini of course ends with \dump end \endinput Adding something after pdfetex.ini has been processed would be pointless.

Thus, my modified luatex.ini therefore looks like this:

% $Id: luatex.ini 26647 2012-05-24 23:54:05Z karl $
% Karl Berry, originally written 2008. Public domain.
% PDF output by default.

% must be done first (activation of primitives)
\input luatexiniconfig.tex
\input luatex-unicode-letters.tex
\input pdftexconfig.tex
\input etex.src
\input pdftexmagfix.tex
\newdimen\pdfeachlineheight
\newdimen\pdfeachlinedepth
\newdimen\pdflastlinedepth
\newdimen\pdffirstlineheight
\newdimen\pdfignoreddimen
\dump
\endinput

Now fmtutils-sys successfully generats the luatex.fmt format file. The lualatex.fmt still fails.

The lualatex.ini can be extended with the list of \pdf* commands just as well. However do something like

\protected\def\pdfliteral{\pdfextension literal}

one needs at least

\catcode`\{=1
\catcode`\}=2

as we are in iniTeX-mode. But there very first thing that latex.ltx checks is

\ifnum\catcode`\{=1
  \errmessage
  {LaTeX must be made using an initex with no format preloaded}
\fi

and this is triggered now. Thus the lualatex.ini needs and additional

\catcode`\{=2

It now looks like this:

% $Id: lualatex.ini 38624 2015-10-12 23:38:27Z karl $
% Originally written 2008 by Karl Berry. Public domain.

\begingroup
  \catcode`\{=1 %
  \catcode`\}=2 %
% JAW: Enable 'extra' pdfTeX primitives only in .ini file
% latex.ltx enables the extra primitives bu a subset of those from
% pdfTeX are needed to do the configuration below (pdftexconfig.tex).
% Rather than list them all, just enable those classed as 'pdftex'.
    \directlua{tex.enableprimitives('',tex.extraprimitives('pdftex', 'luatex', 'etex'))}
% JAW: Set up job name quoting before latex.ltx
% Web2c pdfTeX/XeTeX quote job names containing spaces, but LuaTeX does
% not do this at the engine level. The behaviour can be changed using
% a callback. Originally this code was loaded via lualatexquotejobname.tex
% but that required a hack around latex.ltx: the behaviour has been altered
% to allow the callback route to be used directly.
  \global\everyjob{\directlua{require("lualatexquotejobname.lua")}}
\endgroup

\catcode`\{=1
\catcode`\}=2

% recreate the traditional pdfTeX primitives here REMOVED FOR BREVITY

\catcode`\{=2

% settings inherited from pdftex
\input pdftexconfig.tex
% pdf output by default

\input latex.ltx

Again latex.ltx ends with \dump and \endinput so the necessary \newdimen commands:

\newdimen\pdfeachlineheight
\newdimen\pdfeachlinedepth
\newdimen\pdflastlinedepth
\newdimen\pdffirstlineheight
\newdimen\pdfignoreddimen

cannot be inserted after the \input latex.ltx line, that would be to late. Removing the last to lines from latex.ltx

\dump
\endinput

and placing them in lualatex.ini as follows

% settings inherited from pdftex
\input pdftexconfig.tex
% pdf output by default

\input latex.ltx

\newdimen\pdfeachlineheight
\newdimen\pdfeachlinedepth
\newdimen\pdflastlinedepth
\newdimen\pdffirstlineheight
\newdimen\pdfignoreddimen

\dump
\endinput

allows fmtutil-sys --all to regenerate the lualatex.fmt format file. But now that the LaTeX source has been changed all other formats of course are broken.

There must be a better way to do this! Any ideas?

  • 3
    'Wait for TL'16' :-) The LaTeX team, TeX Live maintainers, Christian Schenk and others are working on various issues to do with LuaTeX v0.85 and some related matters. We are on it but need a bit longer to get it all going. (Indeed, I think I've found quite a problematic issue in LuaTeX v0.85 for getting a 'nice' common set up in the last hour.) – Joseph Wright Dec 14 '15 at 17:07
  • 2
    As far as I know, the LaTeX team is actively working on making LuaTeX 0.85 usable with no hassle. The changes have been too many for hoping the previous setup still works. – egreg Dec 14 '15 at 17:07
  • 1
    @Uli BTW, if you are really interested I can add you onto my CC list for the work going on here! – Joseph Wright Dec 14 '15 at 17:08
  • 2
    @JosephWright yes please, support my procrastination even further ;) – uli Dec 14 '15 at 17:16
9

LuaTeX from v0.85 onward removes essentially all of the pdfTeX 'back end' primitives in favour of three new interfaces:

  • \pdfextension
  • \pdffeedback
  • \pdfvariable

At the same time, there are several other renames and changes to the Lua side (the original token library is removed, and the newtoken library replaces it meaning there is no newtoken any more). These changes require significant work in the format-building process (many of the primitives impacted are not used by end users), in parts of the LaTeX kernel (where the change to the token library impacts) and in packages (most obviously graphics drivers and ifpdf).

'Fixing' all of this in a transparent way is non-trivial. (There's not just LaTeX to think about: plain LuaTeX is also affected.) The LaTeX team have been working on this with the TeX Live maintainer for some time, deciding on how best to handle the required adaptation. At the time of writing, we have several parts of the 'jigsaw' decided but are still implementing them. There are particular issues with the package support: it's very likely that a 'compatibility' package will be needed at least in the short term. (The core team do not control all packages using the PDF back end primitives!) There is also a need to co-ordinate with MiKTeX, TeX Live management utilities and so on.

All of this means that official (CTAN release) support for LuaTeX v0.85+ will not happen 'just yet'. Currently the plan is to get this in place by early in 2016 with a definite requirement for it all to work for TL'16. Anyone testing in the meantime should get in contact with the team/me to get onto the CC list for discussions.

To be clear: unless you are entirely comfortable hand-rebuilding formats and messing with your TeX tree, do not try LuaTeX v0.85 but wait for official support from TeX Live.

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.