5

My original problem statement started like this:

I'm having trouble compiling a large document (.tex file is 4.4M) with lualatex. This is just an experiment, not a document I need to produce. There are two problems: lualatex is exceedingly slow (more than 100x pdflatex), and in the end it fails.

I found a plain-text copy of the bible online, and wrote a python script to wrap the chapter and verse numbers with a little macro call. [...]

Compiling with lualatex ended with the following "fatal error" after running for 20m (1200s, or 109 times the running time of pdflatex). [...]

But I can now re-create the problem with a script-generated .tex file:

Below is a script which produces a .tex file that compiles very slowly with lualatex and which eventually fails with this error:

! TeX capacity exceeded, sorry [number of strings=493861].
\__fontspec_primitive_font_set:Nnn ...\font #1=#2 
                                                  at \dim_eval:n {#3}\scan_s...

l.271101  \myvn{26751}c
                     ategorical
 39802 words of node memory still in use:
   46 hlist, 1 rule, 395 disc, 2 local_par, 1 dir, 413 glue, 717 kern, 24 penal
ty, 3981 glyph, 73 attribute, 48 glue_spec, 73 attribute_list, 1 temp, 1 if_sta
ck nodes
   avail lists: 2:105,3:27,4:7,5:1611,6:2,7:2165,9:39,10:6,11:716
!  ==> Fatal error occurred, no output PDF file produced!
Transcript written on big.log.

real    19m13.003s
user    18m54.195s
sys     0m7.209s

Here is the python (v2) script, I ran it on a Mac and expect it would work on Linux, but it uses /usr/share/dict/words as a source of words and I don't think that there is an equivalent list on Windows. (Perhaps it would be sufficient to repeat the same word 800000 times.)

MWE generator:

#!/usr/bin/python

with open("/usr/share/dict/words") as fp:
    wbank = fp.readlines()
    wcnt = len(wbank)

preamble8 = """\\documentclass{article}
\\RequirePackage{fontspec}
\\RequirePackage{realscripts}
\\defaultfontfeatures[mysspD]{
  Numbers        = {Proportional,OldStyle},
  UprightFont    = SourceSerifPro-Regular ,
  ItalicFont     = SourceSerifPro-RegularIt ,
  SlantedFont    = SourceSerifPro-Regular,
  SlantedFeatures= {FakeSlant=0.2},
  BoldFont       = SourceSerifPro-Semibold ,
  BoldItalicFont = SourceSerifPro-SemiboldIt ,
  Ligatures      = TeX ,
  Scale          = 1 ,
  Extension      = .otf
}
\\setmainfont{mysspD}

\\makeatletter
\\newcommand\\myvn[1]{%
  \\@ifnextchar A%
  {\\textsuperscript{\\liningnums{\\itshape #1\\/}}\\kern -0.08em}%
  {\\textsuperscript{\\liningnums{\\itshape #1\\/}}\\kern 0.14em}}
\\makeatother
\\begin{document}
"""

vn = 1
with open("big.tex", "w") as fp:
    fp.write(preamble8)
    for i in range(800000):
        fp.write(wbank[i % wcnt])
        if i % 10 == 0:
            fp.write("\\myvn{%d}" % (vn,))
            vn += 1
        if i % 150 == 0:
            fp.write("\n\n")
    fp.write("\\end{document}\n")
13
  • 2
    Questions seeking debugging help (“why isn't this code working?”) must include the desired behavior, a specific problem or error and the shortest code necessary to reproduce it in the question itself. Questions without a clear problem statement are not useful to other readers. See minimal working example (MWE). – Henri Menke Nov 22 '19 at 0:38
  • 1
    @HenriMenke I finally succeeded in creating a script that writes a .tex file that reproduces the problem. Thanks for your patience. – dedded Nov 22 '19 at 3:32
  • 2
    @dedded I just checked, it is definitely \liningnums. – Henri Menke Nov 22 '19 at 3:53
  • 3
    @dedded \addfontfeature{Numbers=Lining} reloads the entire font from disk and defines a humongous amount of temporary macros (thanks expl3). – Henri Menke Nov 22 '19 at 4:06
  • 1
    @dedded There is a related problem in unicode-math where you just can't load a math font inside a group because of this insane spew of macros. – Henri Menke Nov 22 '19 at 4:08
9

The culprit is in your repeated use of \liningnums. What this does under the hood is to load the font anew from the file with Numbers=Lining. That means that on each invocation of \myvn you are reloading the font, just to forget it again immediately. Font loading is the single most expensive thing to do in TeX.

I also removed realscripts because it suffers from the same problem. It redefines \textsuperscript to use \addfontfeature{VerticalPosition=Superior}, so it will also reload the whole font from disk on each invocation.

My suspicion is that fontspec allocates a temporary \font for that. The name of this font is stored in TeX's string pool and even though it is never used again after leaving the group, TeX will not forget it, see this question: Release space in the string pool.

You should actually be able to observe the same problem with XeLaTeX, although I have to admit I wasn't patient enough to wait for the error to trigger.

The solution to your problem is obviously to reduce font loading. Therefore you can define a new family like so

\newfontfamily\LnumItMainfont{SourceSerifPro-RegularIt.otf}[Numbers=Lining]

and then instead of

\liningnums{\itshape #1\/}

you can use

\LnumItMainfont #1\/

That saves you not only reloading the font but also reloading the family.

On my machine I now get:

$ time lualatex --interaction=batchmode big.tex
This is LuaTeX, Version 1.10.1 (TeX Live 2019) 
 restricted system commands enabled.

luaotfload | main : initialization completed in 0.079 seconds

real    0m39.016s
user    0m38.864s
sys 0m0.152s
5
  • This solves the "fatal error" problem for me, but not the slowness. It's faster, but still takes 18 minutes. I wonder why it's so much faster for you. Even if you have a much faster machine, it shouldn't be that much faster. – dedded Nov 22 '19 at 14:19
  • @dedded I should have mentioned that I also removed realscripts because it also uses \addfontfeature. – Henri Menke Nov 22 '19 at 22:22
  • @dedded I think this is actually worth a bug report: github.com/wspr/fontspec/issues/391 – Henri Menke Nov 22 '19 at 22:55
  • Yep, that was it: removing realscripts and liningnums made for a significant improvement in performance. I appreciate the bug report, too. I don't personally need to typeset a bible, but obviously bibles are regularly typeset, superscripts seem to be the most common way to set verse numbers, and it feels like this is something that the luatex system ought to be able to handle well. – dedded Nov 23 '19 at 1:05
  • @dedded It's not really LuaTeX's fault that fontspec does not cache fonts. Therefore, XeTeX will suffer from the same problem. What's also disturbing is that strings are exceeded. That means that there is an insane spew of macros from fontspec each time you call \addfontfeature. – Henri Menke Nov 23 '19 at 2:39

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.