5

Introduction

Hello, I am facing very long compile times of more than 14 minutes for a ~130 page document, including around 40 PGF-Plots/TikZ-Images. Due to LaTeX's memory limits and (in fact) for speed up reasons, I decided to use the external-package.

Then I noticed, that each image takes up to 25 seconds to render (looking into my pdflatex.log from Miktex 2.9. Edit: When compiling the same TikZ-images using TikzEdt, it takes only a split second from beginning to the end of compilation and the PDF being there.

Today, I created a MWE for demonstrating this effect, which seems to increase with the number of other packages loaded:

MWE (find it also here: Github)

\documentclass[]{article}

\usepackage{pgfplots}
\usepackage{graphicx} 

% Comment out the next three lines for faster compilation time
\usepgfplotslibrary{external}
\tikzexternalize
\tikzsetexternalprefix{figtemp/}

\begin{document}

\section{First}
    \begin{tikzpicture}
        \tikz \draw[thick,rounded corners=8pt]
        (0,0) -- (0,2) -- (1,3.25) -- (2,2) -- (2,0) -- (0,2) -- (2,2) -- (0,0) -- (2,0);
    \end{tikzpicture}

\section{Second}
    \begin{tikzpicture}
        \draw (-1.5,0) -- (1.5,0);
        \draw (0,-1.5) -- (0,1.5);
    \end{tikzpicture}

\section{Third}
    \begin{tikzpicture}
        \draw (0,0) arc (0:315:1.75cm and 1cm);
    \end{tikzpicture}

\end{document}

MWE Output

This example gives me the two respective pdflatex.log-Files, depending on whether externalization is enabled or disabled:

Externalization enabled

Total time: ~14.0 s

2019-03-05 13:21:31,240+0100 INFO  pdflatex - starting with command line: pdflatex.exe -synctex=1 -interaction=nonstopmode -enable-write18 TeX-TikZ-slow-MWE.tex
2019-03-05 13:21:31,246+0100 INFO  pdflatex - allowing known shell commands
2019-03-05 13:21:31,246+0100 INFO  pdflatex - enabling input (output) from (to) processes
2019-03-05 13:21:31,246+0100 INFO  pdflatex - allowing all shell commands
2019-03-05 13:21:33,607+0100 INFO  pdflatex - executing write18 shell command: pdflatex -enable-write18 -halt-on-error -interaction=batchmode -jobname "figtemp/TeX-TikZ-slow-MWE-figure0" "\def\tikzexternalrealjob{TeX-TikZ-slow-MWE}\input{TeX-TikZ-slow-MWE}"
2019-03-05 13:21:33,885+0100 INFO  pdflatex - starting with command line: pdflatex -enable-write18 -halt-on-error -interaction=batchmode -jobname figtemp/TeX-TikZ-slow-MWE-figure0 \def\tikzexternalrealjob{TeX-TikZ-slow-MWE}\input{TeX-TikZ-slow-MWE}
2019-03-05 13:21:33,891+0100 INFO  pdflatex - allowing known shell commands
2019-03-05 13:21:33,891+0100 INFO  pdflatex - enabling input (output) from (to) processes
2019-03-05 13:21:33,891+0100 INFO  pdflatex - allowing all shell commands
2019-03-05 13:21:36,177+0100 INFO  pdflatex - finishing with exit code 0
2019-03-05 13:21:36,326+0100 INFO  pdflatex - executing write18 shell command: pdflatex -enable-write18 -halt-on-error -interaction=batchmode -jobname "figtemp/TeX-TikZ-slow-MWE-figure1" "\def\tikzexternalrealjob{TeX-TikZ-slow-MWE}\input{TeX-TikZ-slow-MWE}"
2019-03-05 13:21:36,614+0100 INFO  pdflatex - starting with command line: pdflatex -enable-write18 -halt-on-error -interaction=batchmode -jobname figtemp/TeX-TikZ-slow-MWE-figure1 \def\tikzexternalrealjob{TeX-TikZ-slow-MWE}\input{TeX-TikZ-slow-MWE}
2019-03-05 13:21:36,619+0100 INFO  pdflatex - allowing known shell commands
2019-03-05 13:21:36,620+0100 INFO  pdflatex - enabling input (output) from (to) processes
2019-03-05 13:21:36,620+0100 INFO  pdflatex - allowing all shell commands
2019-03-05 13:21:39,167+0100 INFO  pdflatex - finishing with exit code 0
2019-03-05 13:21:39,311+0100 INFO  pdflatex - executing write18 shell command: pdflatex -enable-write18 -halt-on-error -interaction=batchmode -jobname "figtemp/TeX-TikZ-slow-MWE-figure2" "\def\tikzexternalrealjob{TeX-TikZ-slow-MWE}\input{TeX-TikZ-slow-MWE}"
2019-03-05 13:21:39,583+0100 INFO  pdflatex - starting with command line: pdflatex -enable-write18 -halt-on-error -interaction=batchmode -jobname figtemp/TeX-TikZ-slow-MWE-figure2 \def\tikzexternalrealjob{TeX-TikZ-slow-MWE}\input{TeX-TikZ-slow-MWE}
2019-03-05 13:21:39,592+0100 INFO  pdflatex - allowing known shell commands
2019-03-05 13:21:39,592+0100 INFO  pdflatex - enabling input (output) from (to) processes
2019-03-05 13:21:39,592+0100 INFO  pdflatex - allowing all shell commands
2019-03-05 13:21:41,817+0100 INFO  pdflatex - finishing with exit code 0
2019-03-05 13:21:42,029+0100 INFO  pdflatex - finishing with exit code 0
2019-03-05 13:21:42,485+0100 INFO  pdflatex - starting with command line: pdflatex.exe -synctex=1 -interaction=nonstopmode -enable-write18 TeX-TikZ-slow-MWE.tex
2019-03-05 13:21:42,489+0100 INFO  pdflatex - allowing known shell commands
2019-03-05 13:21:42,489+0100 INFO  pdflatex - enabling input (output) from (to) processes
2019-03-05 13:21:42,489+0100 INFO  pdflatex - allowing all shell commands
2019-03-05 13:21:45,223+0100 INFO  pdflatex - finishing with exit code 0

second run, using the externalized images

Total time: ~2.4 s

2019-03-05 14:15:03,628+0100 INFO  pdflatex - starting with command line: pdflatex.exe -synctex=1 -interaction=nonstopmode -enable-write18 TeX-TikZ-slow-MWE.tex
2019-03-05 14:15:03,629+0100 INFO  pdflatex - allowing known shell commands
2019-03-05 14:15:03,629+0100 INFO  pdflatex - enabling input (output) from (to) processes
2019-03-05 14:15:03,629+0100 INFO  pdflatex - allowing all shell commands
2019-03-05 14:15:06,080+0100 INFO  pdflatex - finishing with exit code 0

Externalization disabled

Total time: ~5.1 s

2019-03-05 13:23:19,734+0100 INFO  pdflatex - starting with command line: pdflatex.exe -synctex=1 -interaction=nonstopmode -enable-write18 TeX-TikZ-slow-MWE.tex
2019-03-05 13:23:19,742+0100 INFO  pdflatex - allowing known shell commands
2019-03-05 13:23:19,742+0100 INFO  pdflatex - enabling input (output) from (to) processes
2019-03-05 13:23:19,742+0100 INFO  pdflatex - allowing all shell commands
2019-03-05 13:23:22,045+0100 INFO  pdflatex - finishing with exit code 0
2019-03-05 13:23:22,476+0100 INFO  pdflatex - starting with command line: pdflatex.exe -synctex=1 -interaction=nonstopmode -enable-write18 TeX-TikZ-slow-MWE.tex
2019-03-05 13:23:22,485+0100 INFO  pdflatex - allowing known shell commands
2019-03-05 13:23:22,485+0100 INFO  pdflatex - enabling input (output) from (to) processes
2019-03-05 13:23:22,485+0100 INFO  pdflatex - allowing all shell commands
2019-03-05 13:23:24,842+0100 INFO  pdflatex - finishing with exit code 0

Conclusion

Externalizing my three (very simple) TikZ-Images blows up compilation time almost by a factor of 3 in the first run.

When using a tool like TikzEdt, all my TikZ-images compile in a split second.

What can I do, to speed this up?

Edit:

I found out that TikzEdt uses a precompiled .fmt file, what makes it so fast. As I've got no experience in precompiling, I could use some help for this and how to pass it to the tikzexternalize calls of pdflatex. In this question someone is tryin the same, but I can't seem to organize my preamble to work as desired and meanwhile speed up the externalized image compilation.

Improve this question
15
  • It actually makes sense. You are doing much more processing when externalizing the pictures, so there is some overhead. However your example leads to a wrong conclusion because the pictures are far too simple, so the overhead of externalizing compensates for the time it takes to actually draw the picture. Try with something larger and your conclusion will change.
    – Phelype Oleinik
    Mar 5, 2019 at 12:58
  • When using a tool like TikzEdt, all my TikZ-Images compile in a split second - which allows for a very fast feedback on changes I made in the code. I'd rather suspect some unnecessary initalization for the externalized pdflatex call.
    – DPF
    Mar 5, 2019 at 13:04
  • 1
    I might be off, but you call \tikzexternalize \usepgfplotslibrary{external} in the wrong order. On my PC this actually gives me an Undefined Control Sequence. Also, again. tikzexternalize is not expected to give any speed improvements the when it actually re-draws the tikz pictures. So it would be good if you could tell us how long it takes if you compile it a second time without changing the pictures. If this is the same, this would suggest that tikzexternalize fails to recognize that the image has not changed
    – sheß
    Mar 5, 2019 at 13:11
  • 1
    Can you look into what TeX distribution TikzEdt (which is so much faster) is using? Does it leave any traces in any log files? Does it also use the same MikTeX distribution? Can you update your question with exactly which version of which operating system, and which version of MikTeX you're using?
    – ShreevatsaR
    Mar 5, 2019 at 16:35
  • 1
    I think not many people would notice that your current question is how to create a preamble. Those who've seen this question already may not open it again, and even for someone who opens it, the actual question is at the very end. Your best bet may to be create a new question, focusing solely on how to create a precompiled preamble. It would have to be a very different question from the current one, going into a lot more detail on what exactly you've tried and what problems you've encountered with it. (BTW, sorry to hear about your experience; a document taking 14 minutes sounds horrible!)
    – ShreevatsaR
    Mar 6, 2019 at 22:59

1 Answer 1

Reset to default
0

\tikzexternalize will always take longer than compiling without externalization on the first run. It needs to create the figures just like it has to do w/o tikzexternalize, and on top of the usual work, it has to export them to files. This includes overhead that is added by calling new instances of pdflatex and interacting with the OS separately for each picture.

The performance gain is best illustrated with a slightly more intensive TikZ example, as suggested by @Phelype Oleinik:

\documentclass{article}
\usepackage{pgfplots}
\usepackage{graphicx} 
\usepgfplotslibrary{external}
\tikzexternalize
\tikzsetexternalprefix{figtemp/}
\begin{document}
\section{First}
        \begin{tikzpicture}
                \foreach \i in {1,...,10000} {
                    \draw[thin,rounded corners=8pt] ({\i*0.05},0) -- ({\i*0.05},2) -- ({1+\i*0.05},3.25) -- ({2+\i*0.05},2) -- ({2+\i*0.05},0) -- ({\i*0.05},2) -- ({2+\i*0.05},2) -- ({\i*0.05},0) -- ({2+\i*0.05},0);
                }
        \end{tikzpicture}
\end{document}

The following relationships for run times should be expected:

[First run with \tikzexternalize] > [Run without \tikzexternalize] >>> [Later runs with \tikzexternalize].

Edit: To get a feeling for the additional work the first tikzexternalize run has to do, compared to the plain run w/o tikzexternalized, you can study the log files in the figtemp folder.

As a very crude way of getting a benchmark for how long it should take, you can see how fast your computer is in creating an empty tikz image in a single pdflatex run:

\documentclass{article}
%your preable as in the main document
\begin{document}
\section{First}
        \begin{tikzpicture}
            \coordinate (aa) at (0,0)
        \end{tikzpicture}
\end{document}

As this is (very) roughly equivalent to the extra work tikzexternalize has to do for each picture during the first run. So the first run should overall take:

t0 + n t1 seconds, where t0 is the time it takes to compile the document without tikzexternalize and t1 is the time the TikZ picture above takes to compile.

Improve this answer
7
  • 1
    In general, I agree with your opinion about the actual use-case for tikzexternalize. However, I suspect that somethin else is hindering each shell escaped pdflatex instance unnecessarily - as the manual, separate compilation of my actual images is up x20 faster than while being externalized automatically.
    – DPF
    Mar 5, 2019 at 13:29
  • Mh, it is hard to infer this from what you post. And I do not understand against what you are comparing this. To get a feeling for what's going on you could start looking at the log files in the figtemp/ folder.
    – sheß
    Mar 5, 2019 at 13:35
  • The x20 speedup when using TikzEdt is compared with the "per image" time in my actual document (~25 s)
    – DPF
    Mar 5, 2019 at 13:38
  • Do you mean "per image" as is "compiling each image in a separate document" or as in "without tikzexternalize"? Because the latter would be the wrong comparison. The first run should be expected to take roughly t0 + n t1, where t0 is the time it takes to compile it without tikzexternalize, n is the number of images and t1 is the time it takes to compare the empty TikZ picture from my edit above
    – sheß
    Mar 5, 2019 at 13:43
  • 1
    Sorry I couldn't help you. Good luck
    – sheß
    Mar 5, 2019 at 14:03

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .