2

I am putting together a collection of songs from around the world. I am using fontspec in lualatex with Tex Live on Windows 11 with 16GB of RAM. I am compiling the document variously by using a latex-workshop recipe in Visual Basic Code or by directly using command prompt (via a .cmd script).

I have recently noticed that the document, currently under 400 pages long (including many empty pages for placeholder sections), no longer compiles. I am receiving an error along the following lines:

c:/Program Files/FontBase/Fonts reserve/Edwin.otf><c:/Windows/Fonts/cambriaz.ttf><c:/Program Files/FontBase/Fonts reserve/NotoSans-BoldItalic.otf><c:/Program Files/FontBase/Fonts reserve/NotoSerif-BoldItalic.otf><c:/Program Files/FontBase/Fonts reserve/ScheherazadeNew-Regular.ttf><c:/Windows/Fonts/cambriab.ttf><c:/Program Files/FontBase/Fonts reserve/NotoSerif-Bold.otf><c:/Windows/Fonts/cambria.ttc><c:/Program Files/FontBase/Fonts reserve/NotoSerif-SemiBold.otf><c:/Windows/Fonts/cambriai.ttf><c:/Program Files/FontBase/Fonts reserve/NotoSans-Italic.otf><c:/Program Files/FontBase/Fonts reserve/NotoSans-Bold.otf>fatal: memory exhausted (xmalloc of 59925144 bytes).

I suspect the culprit is the number of fonts in the document. The main fonts are Noto Serif and Noto Sans. I have noticed that, if I comment out a paragraph that contains 2 Japanese kanji characters (with a footnote that contains 4 more characters) — and thus completely eliminating Japanese from the document — the document can compile. Apart from Japanese (Source Han Serif), the document currently has about 20 Chinese characters (Source Han Serif) and a few Arabic letters (Sheherazade). I am also using some fonts for specialised purposes, for example Cambria to print a few symbols that do not look good in Noto, and Edwin for in-text musical symbols.

The document uses the following packages. I strongly suspect that they are secondary contributors to the problem:

  • Lyluatex — All sheet music is inserted using an environment that wraps a \lilypondfile[insert=fullpage] command. I have noticed that, by commenting out a few songs, the document can compile. [Edit: This is unlikely to be a cause. I now compile the scores directly using lilypond and import them using \includepdf. The issue persists. Accordingly, I have removed the lilypond tag from this post]
  • Biber — The document compiles if I delete all the auxiliary files and just run lualatex without running biber. In particular, by not running biber (and therefore generating footnotes and a bibliography that contain the source with the 4 kanji characters), I can compile the document without needing to comment out the text of the paragraph that contains the 2 kanji characters.

I have an inkling that the following could also be contributing to the problem, although I have been unable to set up experiments (like those described above) that clearly point to them as a cause:

  • nomencl and the index (compiled using makeindex);
  • worldflags (noting that the documentation mentions that it is heavy on memory) [Edit: I have now added some code so I can switch this package on and off. Switching it off does not make a difference];
  • cross-referencing throughout the document;
  • an illustration made using pgfplots (I understand that pgfplots is memory-intensive, but commenting out the illustration has never made the document compilable); and[Edit: On reflection, this is also not looking like the cause]
  • a custom command that uses lrbox to determine whether lyrics should be printed in one column or two columns by measuring the width of the content of the lrbox and comparing it against the width of the page.

I have searched my Tex Live directory for all documents containing ‘xmalloc’. I have only found a few PDF documents that shed no light on what it is.

I would be really grateful if anyone could enlighten me on any of the following questions, as they will help me decide whether the project is viable or destined to be a doomed cause (considering that what I have done so far is only a small fraction of what I am envisaging the end product will be):

  1. Which of the above-described issues are most likely to cause memory to be exhausted, and are there any that I can rule out as the cause?
  2. What is xmalloc? Is that a specific form of memory and, if so, can it be expanded?
  3. Will expanding lualatex’s memory generally help resolve this problem, and, if so, how can that be done (I have tried calling lualatex with ‘--max-strings=5000000’ to no avail)?
  4. Is there a way to check precisely what is using how much memory (for example, a breakdown by package or by page)?
  5. More generally, is it possible for a document to cause memory to be exhausted by sheer size, or would memory exhaustion nearly always be the result of badly written commands?

Thank you very much in advance to anyone who might be able to assist.

6
  • 1
    Switching to the 64-bit binaries might help Commented Feb 24, 2023 at 12:31
  • 1
    you could try the texlive 2023 pretest, it has 64-bit binaries by default now. tug.org/texlive/pretest.html. You can also try to create the lua-files for the kanji fonts (it is known that they can exhaust the memory) one by one in small test documents. Once they are there it should work in the full document too. Commented Feb 24, 2023 at 12:44
  • 1
    @UlrikeFischer Can you provide a link, describing what "create the lua-files for the (name) fonts" means?
    – user287367
    Commented Feb 24, 2023 at 16:49
  • I have installed the win64 folder in C:\texlive\2022\bin and then changed the environment variable path to prioritise win64 over win32. However, it is giving me this message: L3 programming layer <2023-02-07>c:/texlive/2022/texmf-dist/tex/latex/l3kernel/expl3.lua:282: bad bytecode register stack traceback: [C]: in function 'lua.getbytecode' c:/texlive/2022/texmf-dist/tex/latex/l3kernel/expl3.lua:282: in main chunk [C]: in function 'require' [\directlua]:1: in main chunk. \lua_now:e #1->__lua_now:n {#1} Was that the proper way to do it? Commented Feb 25, 2023 at 8:47
  • 1
    @Asdfasdflkjsadkfjka Please try loading the fonts in HarfBuzz mode to reduce their memory usage (if you use fontspec you can do so by adding the [Renderer=HarfBuzz] to the font loading command) Commented Feb 26, 2023 at 14:36

1 Answer 1

0

This may be a half-solution, but I have managed to compile the whole document by switching over to xelatex (which I am now free to do because I am no longer using lyluatex).

This will do for now. Thank you to Max Chernoff and Ulrike Fischer for suggesting using 64-bit binaries. I eventually got them to work (or at least I think). They do not appear to make a difference in this case, but I am sure that they will come in handy at some future point.

If anyone has any thoughts about my original questions, I would still be grateful for your insights.

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .