How much memory does LuaTeX use?

Question

I've had issues compiling documents this week using LuaTeX on my laptop, where LuaTeX seems to gobble up as much memory as it can, totally crashing the laptop—i.e. requiring a hard reset as it's unresponsive after >10 minutes. The laptop has 6 GB available, but obviously not all of that should be used by LuaTeX when I've got GNOME, Chromium, several PDFs, etc open at the same time.

The documents in question all utilise TikZ externalisation, and usually have about 3 or 4 TikZ images as well as a few other \includegraphics, and generally are only about 10 pages in length. The crash seems to occur when LuaTeX forks (?) another run to compile the TikZ images. Watching the system performance with htop, I'll often see LuaTeX consuming > 15–20% of my system memory (~1700 MiB) per process.

How do I limit how much memory LuaTeX can use?

I don't really know what other information to provide, as testing with a MWE doesn't consume such ridiculous resources, and in the past community members haven't exactly been happy with me sharing an entire malfunctioning document.

EDIT Further information:

My system is TeXLive 2017, with LuaTeX 1.04. I haven't run an update using tlmgr since updating TeXLive last year using the recommended clean install method.

The document uses the KOMA scrartcl class, and the packages fontspec, xcolor, graphicx, etoolbox, microtype, physics, xfrac, mathtools, unicode-math, siunitx, isodate, scrpage2, caption, subfig, tikz, pgfplots, pgfplotstable, enumitem, array, tabulary, multirow, multicol, booktabs, floatrow, biblatex, glossaries, hyperref, hypcap, and cleveref. Most of these are the stock set of packages I nearly always use in documents, copying preambles along and commenting out the packages I know I won't be using. TikZ itself loads the following libraries:

\usepgflibrary{luamath}
\pgfkeys{pgf/luamath=parser}
\usepgflibrary{fpu}
\usetikzlibrary{calc}
\usetikzlibrary{decorations.markings}
\usetikzlibrary{decorations.pathmorphing}
\usetikzlibrary{decorations.pathreplacing}
\usetikzlibrary{decorations.shapes}
\usetikzlibrary{intersections}
\usetikzlibrary{positioning}
\usetikzlibrary{external}
\usetikzlibrary{patterns}
\usepgflibrary{arrows.meta}
\usetikzlibrary{bending}
\usetikzlibrary{angles}

These are the images that were in one particularly troublesome document:

The most complex image uses lots of named coordinates to ensure everything is aligned correctly, and save me the bother of calculating coordinates manually. I originally didn't externalise this image.

Compiling the document on my desktop computer isn't as much of an issue, as the desktop has 12 GB of RAM so a process that uses 1700 MB doesn't bother it quite as much. I still feel though that this is an exceptional amount of memory.

Answer 1

When running on a Linux system, you can limit the memory consumption of LuaTeX using the program ulimit. In the example I set the max memory size (-m) and the virtual memory (-v) to 100000 kilobytes each.

$ ulimit -m 100000 -v 100000
$ lualatex test.tex
This is LuaTeX, Version 1.07.0 (TeX Live 2018) 

...

[1fatal: memory exhausted (xmalloc of 2802244 bytes).

As you can see this is way too little and LuaTeX will crash because it runs out of memory.

Answer 2

This appears to be solved in more recent versions of LuaTeX—I haven't had any crashes recently, or OOM errors like the one describe. I switched to using the TikZ externalize library for most images once I get their structure finalised. I'm writing a series of documents with a large number of images in them, so I may keep an eye on memory usage if I encounter issues.