Crashes when using \tikzsetnextfilename?

Question

I just stumbled on this, so I'll post about it in Q&A format:

I decided to try tikz-external, and all I have is this in preamble:

\usetikzlibrary{external}
\tikzexternalize[prefix=img/] %activate

... and the initial tests I ran were fine. But, I didn't like the test-figure0 default names given by the package; and I wanted to use something like this for the images:

\tikzsetnextfilename{\jobname-mypic}

However, as soon as I put this in; latex would start generating the file, and then would crash at the end? It even seemed to me - as if that second latex run, ends prematurely -- however, Timing response of externally called commands via write18: blocking or not? confirms that the shell escape call should be blocking (and so it should wait until the spawned "externalization" call to latex completes, succesful or not).

So I debugged a bit, by changing some stuff in the shell command (increasing verbosity by removing batchmode, and adding some sleeps):

\tikzset{external/system call={pdflatex \tikzexternalcheckshellescape -halt-on-error
% -interaction=batchmode -jobname "\image" "\texsource"} % default; batchmode=quiet!
-jobname "\image" "\texsource" ; echo "After pdflatex!" ; sleep 2 ; echo "After sleep!"},
external/optimize=false, % because of https://tex.stackexchange.com/questions/170291/tikz-wont-let-me-externalize-and-halts-on-error/172017#172017
}

... eventually this produces a log like this:

...
(./img/mypic.tikz
% ... runs here...
% (then this - from img/test-mypic.log):
! Package tikz Error: Sorry, image externalization failed: the resulting image
was EMPTY. I tried to externalize 'img/test-mypic', but it seems there is no such image in the document!?
l.526 \end{document}
!  ==> Fatal error occurred, no output PDF file produced!
Transcript written on img/test-mypic.log.
After pdflatex!
After sleep!
% (then this from test.log):
! Package tikz Error: Sorry, the system call ... did NOT result in a usable output file ...
...

... meaning it is spefically the "inner" externalization call to pdflatex that fails - but there are no apparent errors in that file at all? (and the same one externalizes well without a custom name).

So what could be the problem?

Answer 1

The \jobname command always contain the current compilation steps job-name. Hence your code will recurse down and change \jobname which is not what you want.

The external library has provided the command:

\tikzexternalrealjob

which does exactly what you want (the main TeX files jobname). So what you want is:

\tikzsetnextfilename{\tikzexternalrealjob-mypic}

and it should work just fine.

Answer 2

The problem is, indeed, this:

\tikzsetnextfilename{\jobname-mypic}

... or rather, the use of \jobname there... By putting this:

  \edef\tmsg{IZCK tikzexternal@getnextfilename: pgf@tempa '\pgf@tempa', }%
  \typeout{\tmsg} %

in the end of \def\tikzexternal@getnextfilename in texlive/2011/texmf-dist/tex/generic/pgf/frontendlayer/tikz/libraries/tikzexternalshared.code.tex; I could see this in the logs:

$ grep -r IZCK .
...
./test.log:IZCK tikzexternal@getnextfilename: pgf@tempa 'img/test-mypic',
./img/test-mypic.log:IZCK tikzexternal@getnextfilename: pgf@tempa 'img/img/test-mypic-mypic', 

Clearly, the problem is that in the second, externalization call, the \jobname is modified by the externalize library, and so that change recurses into \tikzsetnextfilename.

So, I thought maybe I could use the main file name instead of the \jobname - that turns out not to be straightforward: Macro to retrieve the real name of the source file if pdflatex is invoked with -jobname

Finally, I settled on creating an auxiliary file mjob.id, where upon first run I'd store the main \jobname as \mjobname. The subsequent, recursive calls would then read this \mjobname name from the file; and apply it to the \tikzsetnextfilename:

\IfFileExists{mjob.id}{%true-branch
}{%false-branch
  % NB: \immediate goes everywhere!
  \newwrite\mjobfile
  \immediate\openout\mjobfile=mjob.id
  \immediate\write\mjobfile{\noexpand\gdef\noexpand\mjobname{\jobname}}
  \immediate\closeout\mjobfile
}
\input{mjob.id}

...

\tikzsetnextfilename{\mjobname-mypic} % 

And finally, this seems to stop the recursion of names, and the crash at the end of the externalizing call - and creates all files as requested.