About use of \usepgfplotslibrary{external} and \tikzexternalize in Overleaf

Question

I was reading the pgfplots manual today, and I came across the external pgfplots library. It should help decrease the compilation time and avoid regenerating every pgfplots graphics.

We (me and two other colleagues) are currently writing an article, in the Overleaf online editor, with a plot consisting of about 3000 data entries read from an external .csv file. So the compilation time makes the workflow slow.

I would like to adopt this technique of "convert[ing] every tikzpicture environment automatically to an external graphics."

How can I tell Overleaf's servers to compile the tex file in the above described way?

I tried the following:

The main text is written in main.tex:

% file main.tex
\documentclass{article}
%% Font encoding and input encoding %%
\usepackage[T1]{fontenc}
\usepackage[utf8]{inputenc}
\usepackage[english]{babel}
\usepackage{multicol} % To write in 2 columns
%% To include figures %%
\usepackage{standalone,import,pgfplots,pdfpages,graphicx,float}
\pgfplotsset{compat=1.7}

%% And here is the call to the external method
%% to reduce compilation time
\usepgfplotslibrary{external}
\tikzexternalize % Activated!
\begin{document}
Article text....
\begin{figure*}
    \includestandalone[width=\textwidth]{./figures/plot1}
    \caption{Caption for the figure.}
    \label{fig:tikz:figure1}
\end{figure*}
Text text text.
\end{document}

The file plot1.tex compiles with no error, and therefore I am not listing its contents. I did add the two commands (\usepgfplotslibrary{external} and \tikzexternalize) to its preamble. It is a standalone document class.

Lastly, I created a latexmkrc file that has the following instruction:

$latex = 'pdftex -shell-escape main.tex %O %S';

(I learned about it from here). It lines up (I think) with what is written in the pgfplots manual, subsection 3.2.6 (page 23), to provide a command line switch to pdflatex.

And, the hierarchical file structure is the following:

root folder
├main.tex
└───figures
| │ plot1.tex
│ │ plot2.tex
│ │ plot3.tex
├latexmkrc

In case you don't know, Overleaf lets the user to select a compiler in the project's settings. I tried both its pdfLatex and LaTeX compilers to do this task, and both failed.

The error I get is

Package tikz Error: Sorry, the system call 'pdflatex -shell-escape -halt-on-error -interaction=batchmode -jobname "output-figure0" "\def\tikzexternalrealjob{output}\input{output}"' did NOT result in a usable output file 'output-figure0' (expected one of .pdf:.jpg:.jpeg:.png:). Please verify that you have enabled system calls. For pdflatex, this is 'pdflatex -shell-escape'. Sometimes it is also named 'write 18' or something like that. Or maybe the command simply failed? Error messages can be found in 'output-figure0.log'. If you continue now, I'll try to typeset the picture.

Answer 1

The invocation of the standalone package needs to have the option mode=buildnew:

\usepackage[mode=buildnew]{standalone}

Without that option, standalone doesn't actually accomplish the task given in the question: (a) To compile a particular .tex file once to a graphics file and then to use the graphics file in the main document, and (b) only to rebuild the graphics file when the .tex file has changed. By default, standalone doesn't compile the .tex file, but simply inputs the file and uses the result, rather than doing the separate compilation.

Worse: With the default settings, for the standalone package, the \includestandalone command interferes with the use of the \tikzexternalize command, so that when standalone tries to input the .tex file it fails, and gives the strange errors found in the question.

With hindsight one can see the symptoms in the log files. In particular, in main-figure0.log there's a message that the "command '\includegraphics [width=\textwidth ]{plottry.tex}' has been optimized away." (Plus a probably non-optimal suggestion as to what to do about it.)