Strange problem with pgf/tikz auxlock file when invoking tikzexternalize

Question

I'm trying to generate a number of somewhat complicated TikZ based plots using gnuplot's lua/tikz terminal. Things had been working fine last month, but when I needed to update some of these plots today, things did not work. I've constructed a minimal example of a gnuplot script and a TeX file that produces the error.

The problem is, when I run lualatex --shell-escape --interaction=nonstopmode test.in I get, amidst the copious output, the following error:

===== 'mode=convert with system call': Invoking 'lualatex --halt-on-error --int
eraction=batchmode --jobname "TikZ_figs/fig1" "\def\tikzexternalrealjob{test}\i
nput{test}"' ========
This is LuaTeX, Version beta-0.50.0-2010010505
! I can't write on file `test.auxlockalt-on-error --interaction=batchmode --job
name "TikZ_figs/fig1" "\def\tikzexternalrealjob{test}\input{test}"'.
\tikzexternalauxlock@setlock ...@realjob .auxlock 
                                                  \immediate \write \w@pgf@w...
l.74 \end{tikzpicture}

Please type another output file name
! Emergency stop.

I'd like to keep the -interaction=nonstopmode flag, because I'd like to keep using TeXlipse to compile the document. Removing this flag gives a prompt at

Please type another output file name:

If I enter the name test.auxlock (which seems to be the correct name, as the contents of this file contain only a single line: \def \tikzexternallocked {0}), then I get the same error (this time, just a warning) but without the emergency stop, thrown because of -interaction=nonstopmode.

The build then goes on and successfully generates the main document and the externalized graphics. If I leave the prompt blank, on the other hand, the test.auxlock file overwrites the main tex file test.tex. Quite distressing!

It seems that the invocation line for tikzexternalize is being used as the filename for the auxlock file. I don't know why this happens, or how to prevent it.

Is this something I'm experiencing because of some buggy installation? (As I said, it used to work and now it doesn't, I may have updated something or corrupted some file.)

I'm running texlive on Ubuntu 10.10, with pgf 2.10 from Ubuntu 12.04 (since pgf from Ubuntu 10.10 doesn't have the proper externalize routines) and texlive-luatex 2009-7.

Files and directories for minimal working example:

test.plt
test.tex
TikZ_files/

Code for MWE

Gnuplot script test.plt:

#!/usr/bin/gnuplot
#test.plt - create some plots to be rendered with TikZ

set term tikz
set output 'fig1.tex'
plot sin(x)/x

set output 'fig2.tex'
plot erf(x)

Main tex file test.tex:

%test.tex - MWE for lualatex auxlock file error
\documentclass[]{report}

\usepackage{tikz}
\usetikzlibrary{external}
\usepackage{gnuplot-lua-tikz}
\tikzset{ external/system call={lualatex \tikzexternalcheckshellescape
          --halt-on-error --interaction=batchmode
          --jobname "\image" "\texsource"} }
\tikzexternalize[
  mode=convert with system call,
  prefix=TikZ_figs/
]

\begin{document}

  \begin{figure}
    \tikzsetnextfilename{fig1}
    \input{fig1.tex}
    \caption{\label{fig:fig1} A plot of sinc($x$).}
  \end{figure}

  \begin{figure}
    \tikzsetnextfilename{fig2}
    \input{fig2.tex}
    \caption{\label{fig:fig2} A plot of erf($x$).}
  \end{figure}

\end{document}

If you don't have lua/tikz support compiled into gnuplot, the gnuplot script won't work, sorry about that. The tex files generated by gnuplot (fig1.tex and fig2.tex) are longish, but here goes.

fig1.tex:

\begin{tikzpicture}[gnuplot]
%% generated with GNUPLOT 4.6p0 (Lua 5.1; terminal rev. 99, script rev. 100)
%% Tue 22 May 2012 10:34:12 PM EDT
\path (0.000,0.000) rectangle (12.500,8.750);
\gpcolor{color=gp lt color border}
\gpsetlinetype{gp lt border}
\gpsetlinewidth{1.00}
\draw[gp path] (1.196,0.616)--(1.376,0.616);
\draw[gp path] (11.947,0.616)--(11.767,0.616);
\node[gp node right] at (1.012,0.616) {-0.4};
\draw[gp path] (1.196,1.725)--(1.376,1.725);
\draw[gp path] (11.947,1.725)--(11.767,1.725);
\node[gp node right] at (1.012,1.725) {-0.2};
\draw[gp path] (1.196,2.835)--(1.376,2.835);
\draw[gp path] (11.947,2.835)--(11.767,2.835);
\node[gp node right] at (1.012,2.835) { 0};
\draw[gp path] (1.196,3.944)--(1.376,3.944);
\draw[gp path] (11.947,3.944)--(11.767,3.944);
\node[gp node right] at (1.012,3.944) { 0.2};
\draw[gp path] (1.196,5.053)--(1.376,5.053);
\draw[gp path] (11.947,5.053)--(11.767,5.053);
\node[gp node right] at (1.012,5.053) { 0.4};
\draw[gp path] (1.196,6.162)--(1.376,6.162);
\draw[gp path] (11.947,6.162)--(11.767,6.162);
\node[gp node right] at (1.012,6.162) { 0.6};
\draw[gp path] (1.196,7.272)--(1.376,7.272);
\draw[gp path] (11.947,7.272)--(11.767,7.272);
\node[gp node right] at (1.012,7.272) { 0.8};
\draw[gp path] (1.196,8.381)--(1.376,8.381);
\draw[gp path] (11.947,8.381)--(11.767,8.381);
\node[gp node right] at (1.012,8.381) { 1};
\draw[gp path] (1.196,0.616)--(1.196,0.796);
\draw[gp path] (1.196,8.381)--(1.196,8.201);
\node[gp node center] at (1.196,0.308) {-10};
\draw[gp path] (3.884,0.616)--(3.884,0.796);
\draw[gp path] (3.884,8.381)--(3.884,8.201);
\node[gp node center] at (3.884,0.308) {-5};
\draw[gp path] (6.572,0.616)--(6.572,0.796);
\draw[gp path] (6.572,8.381)--(6.572,8.201);
\node[gp node center] at (6.572,0.308) { 0};
\draw[gp path] (9.259,0.616)--(9.259,0.796);
\draw[gp path] (9.259,8.381)--(9.259,8.201);
\node[gp node center] at (9.259,0.308) { 5};
\draw[gp path] (11.947,0.616)--(11.947,0.796);
\draw[gp path] (11.947,8.381)--(11.947,8.201);
\node[gp node center] at (11.947,0.308) { 10};
\draw[gp path] (1.196,8.381)--(1.196,0.616)--(11.947,0.616)--(11.947,8.381)--cycle;
\node[gp node right] at (10.479,8.047) {sin(x)/x};
\gpcolor{color=gp lt color 0}
\gpsetlinetype{gp lt plot 0}
\draw[gp path] (10.663,8.047)--(11.579,8.047);
\draw[gp path] (1.196,2.533)--(1.305,2.628)--(1.413,2.736)--(1.522,2.853)--(1.630,2.974)%
  --(1.739,3.094)--(1.848,3.210)--(1.956,3.315)--(2.065,3.405)--(2.173,3.476)--(2.282,3.524)%
  --(2.391,3.546)--(2.499,3.539)--(2.608,3.502)--(2.716,3.435)--(2.825,3.339)--(2.934,3.216)%
  --(3.042,3.070)--(3.151,2.905)--(3.259,2.725)--(3.368,2.539)--(3.477,2.351)--(3.585,2.171)%
  --(3.694,2.004)--(3.802,1.860)--(3.911,1.745)--(4.019,1.667)--(4.128,1.631)--(4.237,1.644)%
  --(4.345,1.708)--(4.454,1.827)--(4.562,2.002)--(4.671,2.233)--(4.780,2.517)--(4.888,2.853)%
  --(4.997,3.234)--(5.105,3.653)--(5.214,4.104)--(5.323,4.577)--(5.431,5.063)--(5.540,5.551)%
  --(5.648,6.030)--(5.757,6.490)--(5.866,6.919)--(5.974,7.308)--(6.083,7.648)--(6.191,7.930)%
  --(6.300,8.148)--(6.409,8.297)--(6.517,8.372)--(6.626,8.372)--(6.734,8.297)--(6.843,8.148)%
  --(6.952,7.930)--(7.060,7.648)--(7.169,7.308)--(7.277,6.919)--(7.386,6.490)--(7.495,6.030)%
  --(7.603,5.551)--(7.712,5.063)--(7.820,4.577)--(7.929,4.104)--(8.038,3.653)--(8.146,3.234)%
  --(8.255,2.853)--(8.363,2.517)--(8.472,2.233)--(8.581,2.002)--(8.689,1.827)--(8.798,1.708)%
  --(8.906,1.644)--(9.015,1.631)--(9.124,1.667)--(9.232,1.745)--(9.341,1.860)--(9.449,2.004)%
  --(9.558,2.171)--(9.666,2.351)--(9.775,2.539)--(9.884,2.725)--(9.992,2.905)--(10.101,3.070)%
  --(10.209,3.216)--(10.318,3.339)--(10.427,3.435)--(10.535,3.502)--(10.644,3.539)--(10.752,3.546)%
  --(10.861,3.524)--(10.970,3.476)--(11.078,3.405)--(11.187,3.315)--(11.295,3.210)--(11.404,3.094)%
  --(11.513,2.974)--(11.621,2.853)--(11.730,2.736)--(11.838,2.628)--(11.947,2.533);
\gpcolor{color=gp lt color border}
\gpsetlinetype{gp lt border}
\draw[gp path] (1.196,8.381)--(1.196,0.616)--(11.947,0.616)--(11.947,8.381)--cycle;
%% coordinates of the plot area
\gpdefrectangularnode{gp plot 1}{\pgfpoint{1.196cm}{0.616cm}}{\pgfpoint{11.947cm}{8.381cm}}
\end{tikzpicture}
%% gnuplot variables

fig2.tex:

\begin{tikzpicture}[gnuplot]
%% generated with GNUPLOT 4.6p0 (Lua 5.1; terminal rev. 99, script rev. 100)
%% Tue 22 May 2012 10:34:12 PM EDT
\path (0.000,0.000) rectangle (12.500,8.750);
\gpcolor{color=gp lt color border}
\gpsetlinetype{gp lt border}
\gpsetlinewidth{1.00}
\draw[gp path] (1.196,0.616)--(1.376,0.616);
\draw[gp path] (11.947,0.616)--(11.767,0.616);
\node[gp node right] at (1.012,0.616) {-1};
\draw[gp path] (1.196,2.557)--(1.376,2.557);
\draw[gp path] (11.947,2.557)--(11.767,2.557);
\node[gp node right] at (1.012,2.557) {-0.5};
\draw[gp path] (1.196,4.499)--(1.376,4.499);
\draw[gp path] (11.947,4.499)--(11.767,4.499);
\node[gp node right] at (1.012,4.499) { 0};
\draw[gp path] (1.196,6.440)--(1.376,6.440);
\draw[gp path] (11.947,6.440)--(11.767,6.440);
\node[gp node right] at (1.012,6.440) { 0.5};
\draw[gp path] (1.196,8.381)--(1.376,8.381);
\draw[gp path] (11.947,8.381)--(11.767,8.381);
\node[gp node right] at (1.012,8.381) { 1};
\draw[gp path] (1.196,0.616)--(1.196,0.796);
\draw[gp path] (1.196,8.381)--(1.196,8.201);
\node[gp node center] at (1.196,0.308) {-10};
\draw[gp path] (3.884,0.616)--(3.884,0.796);
\draw[gp path] (3.884,8.381)--(3.884,8.201);
\node[gp node center] at (3.884,0.308) {-5};
\draw[gp path] (6.572,0.616)--(6.572,0.796);
\draw[gp path] (6.572,8.381)--(6.572,8.201);
\node[gp node center] at (6.572,0.308) { 0};
\draw[gp path] (9.259,0.616)--(9.259,0.796);
\draw[gp path] (9.259,8.381)--(9.259,8.201);
\node[gp node center] at (9.259,0.308) { 5};
\draw[gp path] (11.947,0.616)--(11.947,0.796);
\draw[gp path] (11.947,8.381)--(11.947,8.201);
\node[gp node center] at (11.947,0.308) { 10};
\draw[gp path] (1.196,8.381)--(1.196,0.616)--(11.947,0.616)--(11.947,8.381)--cycle;
\node[gp node right] at (10.479,8.047) {erf(x)};
\gpcolor{color=gp lt color 0}
\gpsetlinetype{gp lt plot 0}
\draw[gp path] (10.663,8.047)--(11.579,8.047);
\draw[gp path] (1.196,0.616)--(1.305,0.616)--(1.413,0.616)--(1.522,0.616)--(1.630,0.616)%
  --(1.739,0.616)--(1.848,0.616)--(1.956,0.616)--(2.065,0.616)--(2.173,0.616)--(2.282,0.616)%
  --(2.391,0.616)--(2.499,0.616)--(2.608,0.616)--(2.716,0.616)--(2.825,0.616)--(2.934,0.616)%
  --(3.042,0.616)--(3.151,0.616)--(3.259,0.616)--(3.368,0.616)--(3.477,0.616)--(3.585,0.616)%
  --(3.694,0.616)--(3.802,0.616)--(3.911,0.616)--(4.019,0.616)--(4.128,0.616)--(4.237,0.616)%
  --(4.345,0.616)--(4.454,0.616)--(4.562,0.616)--(4.671,0.616)--(4.780,0.616)--(4.888,0.616)%
  --(4.997,0.616)--(5.105,0.616)--(5.214,0.617)--(5.323,0.620)--(5.431,0.626)--(5.540,0.642)%
  --(5.648,0.675)--(5.757,0.741)--(5.866,0.862)--(5.974,1.067)--(6.083,1.387)--(6.191,1.848)%
  --(6.300,2.460)--(6.409,3.210)--(6.517,4.057)--(6.626,4.940)--(6.734,5.787)--(6.843,6.537)%
  --(6.952,7.149)--(7.060,7.610)--(7.169,7.930)--(7.277,8.135)--(7.386,8.256)--(7.495,8.322)%
  --(7.603,8.355)--(7.712,8.371)--(7.820,8.377)--(7.929,8.380)--(8.038,8.381)--(8.146,8.381)%
  --(8.255,8.381)--(8.363,8.381)--(8.472,8.381)--(8.581,8.381)--(8.689,8.381)--(8.798,8.381)%
  --(8.906,8.381)--(9.015,8.381)--(9.124,8.381)--(9.232,8.381)--(9.341,8.381)--(9.449,8.381)%
  --(9.558,8.381)--(9.666,8.381)--(9.775,8.381)--(9.884,8.381)--(9.992,8.381)--(10.101,8.381)%
  --(10.209,8.381)--(10.318,8.381)--(10.427,8.381)--(10.535,8.381)--(10.644,8.381)--(10.752,8.381)%
  --(10.861,8.381)--(10.970,8.381)--(11.078,8.381)--(11.187,8.381)--(11.295,8.381)--(11.404,8.381)%
  --(11.513,8.381)--(11.621,8.381)--(11.730,8.381)--(11.838,8.381)--(11.947,8.381);
\gpcolor{color=gp lt color border}
\gpsetlinetype{gp lt border}
\draw[gp path] (1.196,8.381)--(1.196,0.616)--(11.947,0.616)--(11.947,8.381)--cycle;
%% coordinates of the plot area
\gpdefrectangularnode{gp plot 1}{\pgfpoint{1.196cm}{0.616cm}}{\pgfpoint{11.947cm}{8.381cm}}
\end{tikzpicture}
%% gnuplot variables

Answer 1

This isn't a very good answer, but it's the best I've found so far. There seems to be some problem with tikzexternalize pushing strings into variables. I altered the code found in tikzexternalshared.code.tex (in texlive on Ubuntu, located in /usr/share/texmf/tex/generic/pgf/frontendlayer/tikz/libraries) to include an additional line.

Code before (lines 415-419):

\def\tikzexternalauxlock@setlock#1{%
        \immediate\openout\w@pgf@writea=\tikzexternal@realjob.auxlock
        \immediate\write\w@pgf@writea{\noexpand\def\noexpand\tikzexternallocked{#1}}%
        \immediate\closeout\w@pgf@writea
}

and code after:

\def\tikzexternalauxlock@setlock#1{%
        \immediate\write16{Value of \noexpand\w@pgf@writea: '\the\w@pgf@writea'}
        \immediate\openout\w@pgf@writea=\tikzexternal@realjob.auxlock
        \immediate\write\w@pgf@writea{\noexpand\def\noexpand\tikzexternallocked{#1}}%
        \immediate\closeout\w@pgf@writea

}

Just a little debugging statement that prints out the value of the file handle (in my case, '3'), caused the lualatex compiler to run without a hitch, at least with respect to the auxlock filename. It seems that without the \the call, the \w@pgf@writea variable doesn't properly purge itself (it had previously been used to hold the conversion system call), and ends up writing the name of the auxlock file over the beginning of the variable, without "zeroing it out" first, thus giving trailing characters that cause a file-open failure.

I still don't know what changed on my system that this behavior arose. Can anyone confirm this behavior, or explain the correct way to patch the tikzexternalize routines?