TeX capacity exceeded when externalizing

Question

I have several large TikZ plots I want to externalize. Without externalization, everything works fine.

Document:

\documentclass{article}
\usepackage{tikz}
\usetikzlibrary{external}
%\tikzexternalize
\begin{document}
  \input{plot}
\end{document}

Output:

$ pdflatex article
<output omitted>
(./plot.tex) [1{/var/lib/texmf/fonts/map/pdftex/updmap/pdftex.map}]
(./article.aux) )</usr/share/texlive/texmf-dist/fonts/type1/public/amsfonts/cm/
cmr10.pfb>
Output written on article.pdf (1 page, 357026 bytes).
Transcript written on article.log.

However, when I uncomment \tikzexternalize, pdflatex exists with an error.

$ pdflatex article
<output omitted>
(./plot.tex
Runaway text?
[x=1pt,y=1pt] \definecolor {fillColor}{RGB}{255,255,255} \path [use a\ETC.
! TeX capacity exceeded, sorry [main memory size=5000000].
<argument> ... circle ( 1.16); \par \path [dr\ETC.

l.21034 \end{tikzpicture}

!  ==> Fatal error occurred, no output PDF file produced!
Transcript written on article.log.

What is going on here? How can externalization increase space consumption? After all, the whole point of externalization is to improve efficiency and space consumption during compilation.

Answer 1

What you say is correct: the point of externalisation is to reduce resource usage. However every optimisation has some overhead. To externalise a picture TikZ reads the contents of the entire tikzpicture environment and passes that to an external run of the engine to produce the picture. The externalisation library passes around the contents of the picture in macros in order to set everything up to make the picture and this ends up creating more than one copy of the same thing.

When you compile the document the error message asks Runaway text? and tells you that you filled up TeX's memory.

Your plot.tex file is 1.1 MB, and TeX's main memory size is 5000000 “words of memory”, or 2.5 MB, so it takes a little over two copies of the contents of the file to run out of space.

The solution? Use TeX's macros and TikZ's styles to reduce the size of the input while maintaining the result of the code. For example, your file has 10269 (!) instances of

\path[draw=drawColor,line width= 0.4pt,line join=round,line cap=round]

If you do

\tikzset{mydots/.style={draw=drawColor,line width= 0.4pt}}
%,line join=round,line cap=round <-- These two don't do anything here anyway

and then replace all other occurrences of that option list by mydots you'll reduce the file size to 500 kB (which is enough to make it work).

You can go further and define a macro

\def\circ(#1,#2){\path[mydots](#1,#2)circle(1.16);}

and then replace all the

\path [mydots] ( something, another) circle (1.16);

by

\circ(something,another)

which will bring the file size down to 200 kB.

It can be further optimized using TikZ's magic, but I'll leave it as an exercise for the reader ;)

Here's the 200 kB file for you: https://pastebin.com/rJfe4ctx

Answer 2

If the document is stable and you will make few modifications to tikzpicture, it is better to handle it in separate files and include them as an image (or with input).... ltximg script you can do the following:

$ ltximg --subenv --imgdir=mypics --prefix=tkz -o test-out test-in.tex

Now you will have a document with the tikzpicture environments converted into images (preserving the consistency of the fonts) and you will also have each tikzpictureenvironment in separate files (in case you need some extra modification). First check the documentation to see the options and conditions that the input file must meet.