# Tikzexternalize does not update with self-written plot command

## EDIT 2:

So the problem is that the code within the tikzpicture environment is not expanded before the generation of the .md5 hash. So, the content does not change with different inputs.

Now, I have the idea of creating an own .md5 hash of the inputs to my command, which is then stored in a separate file.

\newwrite\tempfile%
\immediate\openout\tempfile=./\csvPlotOutputDir/\csvPlotLabel._md5%
\immediate\write\tempfile{\pdfmdfivesum{#1}}%
\immediate\closeout\tempfile%


This seems to work fine. It changes the hash, when the options are modified. Now I am figuring out a way to evaluate changes in that file.

## EDIT 1:

When running latexmk, an .md5-file is generated for each externalized plot. However, it is not recreated after I change some options of the \csvplot[]{} command and compile.

So, the content of the .md5 file does actually not change if I change something in the options list. Why is it so? The output in the compiled document IS different, when I clean aux files and recompile with the changes.

I wrote my own package for plotting .csv files with different settings. For example, I can choose to plot certain columns of the .csv with a simple list of column names that I pass as a parameter to my command.

However, I also want to use Tikz externalize and I usually compile with latexmk. Because I want to use as many CPU cores as possible, I use externalization with the [mode=list and make] option.

So, the following is the compilation flow:

latexmk -pdf file.tex
make -j 4 -f file.makefile
latexmk -g -pdf file.tex


I have no problems with it, and up to four images are externalized at once. I also get the .pdf output without issues.

With my self-written function for plotting .csv files, I just do not get tikzexternalize to update the graphics. It detects no change. This is also true if you do not use the mode=list and make option. So, the below MWE does not include it for easier access.

Below you find a MWE, which is not really minimal, since the package has already grown. In principle, I plot within a groupplot environment, where there are two loops iterating over different columns of the .csv file. The first loop is for the columns that serve as x-axes, the second one for the y-axes. These loops are nested.

Finally, find the full code below:

The main .tex file to compile (csvtest.txt) without the mode=list and make option. Included is a manual declaration of a plot which DOES update if I change something (e.g. the columns that it should draw its data from).

\documentclass{article}
\usepackage[default]{csvtoolbox}

\usetikzlibrary{external}%
\tikzexternalize%[mode=list and make]%

\begin{document}

\csvplot[%
%output dir = fig,  % this allows you to choose the output folder for the .pdf
label = f1, % this also defines the name of the externalized figure file name -> f1.pdf
x columns = {x},%
y columns = {a, b},%
] {data.csv}

\begin{figure}
\begin{tikzpicture}
\begin{groupplot}
\nextgroupplot
\end{groupplot}
\end{tikzpicture}
\end{figure}

\end{document}


The following is the package I wrote for plotting .csv files.

\NeedsTeXFormat{LaTeX2e}%
\ProvidesPackage{csvtoolbox}[2017/05/17 CSV Plot]%

\RequirePackage{etoolbox}%
\RequirePackage{pgfplots}%
\RequirePackage{tikz}%
\RequirePackage{pgfkeys}%
\RequirePackage{float}%
\RequirePackage{hyperref}%
\RequirePackage{tikzscale}%

\usepgfplotslibrary{groupplots}%

\DeclareOption{default}{%
\pgfplotsset{csvPlotStyle/.style=%
{%
cycle list name=black white,%
grid=both,%
grid style={dashed, gray},%
}%
}%
\pgfkeys{
% Switch to /csvplot key subset
/csvplot/.is family, /csvplot,%
% User options
def/.style = {%
axis rows = 1,%
axis columns = 1,%,
legend pos = north east,%
height = \textwidth,%
width = \textwidth,%
x columns = x,%
y columns = y,%
style = csvPlotStyle,%
caption = ,%
label = ,%
align = top,%
output dir =.%
}%
}%
}%
%
\ExecuteOptions{default}%
\ProcessOptions\relax%
%
% Set up the keys
\pgfkeys{
% Switch to /csvplot key subset
/csvplot,%
%% definitions
axis rows/.estore in = \csvPlotAxisRows,%
axis columns/.estore in = \csvPlotAxisCols,%
output dir/.estore in = \csvPlotOutputDir,%
legend pos/.estore in = \csvLegPos,%
height/.estore in = \csvPlotHeight,%
width/.estore in = \csvPlotTotalWidth,%
x columns/.estore in = \csvPlotXCol,%
y columns/.estore in = \csvPlotYCol,%
style/.estore in = \csvPlotStyle,%
caption/.estore in = \csvPlotCaption,%
label/.estore in = \csvPlotLabel,%
align/.style = {alignments/#1/.get = \csvPlotAlignment},%
% dictionary for alignment of figure
alignments/.cd,%
top/.initial = t,%
here/.initial = h,%
bottom/.initial = b,%
force/.initial = H,%
page/.initial = p,%
}
%
\newcommand{\csvplot}[2][]{%
\pgfkeys{/csvplot, def, #1}%
\tikzsetnextfilename{./\csvPlotOutputDir/\csvPlotLabel}%
% Expand figure options
\def\efigure{\begin{figure}}\expandafter\efigure\expandafter[\csvPlotAlignment]%
\centering%
\begin{tikzpicture}%
\begin{groupplot} [%
\csvPlotStyle,%
height=1/\csvPlotAxisRows*\csvPlotHeight,%
width=1/\csvPlotAxisCols*\csvPlotTotalWidth,%
legend pos=\csvLegPos,%
group style = {group size = \csvPlotAxisCols\space by \csvPlotAxisRows},%
]%
% Plot loop: All x/y columns for multiple plots in one axis
\edef\XIterate{%
\noexpand\pgfplotsforeachungrouped \noexpand\x in {\csvPlotXCol}%
}
\edef\YIterate{%
\noexpand\pgfplotsforeachungrouped \noexpand\y in {\csvPlotYCol}%
}
\XIterate {%
\YIterate {%
\eappto\PlotList{%
\noexpand\nextgroupplot%
\noexpand\addplot table[x=\x, y=\y, col sep=comma] {#2};%
}%
}%
}%
\PlotList%
\end{groupplot}%
\end{tikzpicture}%
\caption{\csvPlotCaption}%
\label{\csvPlotLabel}%
\end{figure}
}%


Finally, some data to plot (data.csv)

a,b,x,y
1,2,3,4
5,6,8,1
9,2,4,5
5,5,1,9


• The hash in the .md5 file depends only on the content of the tikzpicture environment. Doesn't pgfplots have its own version, too? Anyway, you need to either handle it manually or set something up so that the pictures are dependent on other things. See the documentation of the external library for details of how to do this. – cfr May 19 '17 at 1:58
• I think you are right. It says in the documentation: Attention: the content–based strategies md5 and diff operate on the picture content – and only on the picture content. Here, “picture content” only includes the top–level tokens; no expansion is applied and no included files are part of the strategies. The expansion part is of particular interest here. – elagil May 19 '17 at 6:58

I finally found a solution. I generate a separate .md5 hash file, based on the options that I pass to my command. Within the command, the hash file is generated as follows:

\newwrite\tempfile%
\immediate\openout\tempfile=./\csvPlotOutputDir/\csvPlotLabel.md5_%
\immediate\write\tempfile{\pdfmdfivesum{#1}}%
\immediate\closeout\tempfile%


Then, I make tikzexternalize depend on the contents of this file by adding

\tikzpicturedependsonfile{./\csvPlotOutputDir/\csvPlotLabel.md5_}%


Here is the whole csvtoolbok.sty file for reference:

\NeedsTeXFormat{LaTeX2e}%
\ProvidesPackage{csvtoolbox}[2017/05/17 CSV Plot]%

\RequirePackage{etoolbox}%
\RequirePackage{pgfplots}%
\RequirePackage{tikz}%
\RequirePackage{pgfkeys}%
\RequirePackage{float}%
\RequirePackage{hyperref}%
\RequirePackage{tikzscale}%

\usepgfplotslibrary{groupplots}%

\DeclareOption{default}{%
\pgfplotsset{csvPlotStyle/.style=%
{%
cycle list name=color,%
grid=both,%
grid style={dashed, gray},%
}%
}%
\pgfkeys{
% Switch to /csvplot key subset
/csvplot/.is family, /csvplot,%
% User options
def/.style = {%
axis rows = 1,%
axis columns = 1,%,
output dir =.,%
legend pos = outer south,%
height = \textwidth,%
width = \textwidth,%
plot data x/y = x/y,%
common style = csvPlotStyle,%
caption = ,%
label = ,%
align = top,%
}%
}%
}%
%
\ExecuteOptions{default}%
\ProcessOptions\relax%
% Define outer south legend style
\pgfplotsset{
every axis x label/.append style={
alias=current axis xlabel
},
legend pos/outer south/.style={
/pgfplots/legend style={
at={%
(%
\@ifundefined{pgf@sh@ns@current axis xlabel}%
{xticklabel cs:0.5}%
{current axis xlabel.south}%
)%
},
anchor=north
}
}
}
%
% Set up the keys
\pgfkeys{
% Switch to /csvplot key subset
/csvplot,%
%% definitions
axis rows/.estore in = \csvPlotAxisRows,%
axis columns/.estore in = \csvPlotAxisCols,%
output dir/.estore in = \csvPlotOutputDir,%
legend pos/.estore in = \csvLegPos,%
height/.estore in = \csvPlotHeight,%
width/.estore in = \csvPlotTotalWidth,%
plot data x/y/.estore in = \csvPlotXYCol,%
common style/.estore in = \csvPlotStyle,%
caption/.estore in = \csvPlotCaption,%
label/.estore in = \csvPlotLabel,%
align/.style = {alignments/#1/.get = \csvPlotAlignment},%
% dictionary for alignment of figure
alignments/.cd,%
top/.initial = t,%
here/.initial = h,%
bottom/.initial = b,%
force/.initial = H,%
page/.initial = p,%
}
%
\newcommand{\csvplot}[2][]{%
\pgfkeys{/csvplot, def, #1}%
%
\newwrite\tempfile%
\immediate\openout\tempfile=./\csvPlotOutputDir/\csvPlotLabel.md5_%
\immediate\write\tempfile{\pdfmdfivesum{#1}}%
\immediate\closeout\tempfile%
%
\tikzsetnextfilename{./\csvPlotOutputDir/\csvPlotLabel}%
\tikzpicturedependsonfile{./\csvPlotOutputDir/\csvPlotLabel.md5_}%
% Expand figure options
\def\efigure{\begin{figure}}\expandafter\efigure\expandafter[\csvPlotAlignment]%
\centering%
\begin{tikzpicture}%
\begin{groupplot} [%
\csvPlotStyle,%
height=1/\csvPlotAxisRows*\csvPlotHeight,%
width=1/\csvPlotAxisCols*\csvPlotTotalWidth,%
legend pos=\csvLegPos,%
group style = {group size = \csvPlotAxisCols\space by \csvPlotAxisRows},%
]%
% Plot loop: All x/y columns for multiple plots in one axis
\edef\XYIterate{%
\noexpand\pgfplotsforeachungrouped \noexpand\x/\noexpand\y in {\csvPlotXYCol}%
}%
\XYIterate {%
\eappto\PlotList{%
\noexpand\nextgroupplot%
\noexpand\addplot table[x=\x, y=\y, col sep=comma] {#2};%