Luatex os.execute not working

Question

1. I've hooked luatex's stop_run callback so I can do some processing on the output pdf after it is generated.
2. To process the pdf I need to call os.execute. When doing so I an error saying "All command executes disabled" but otherwise everything works normal execute os.execute doesn't execute the command I want.
3. I use -shell-escape to try and get os.execute to work

4. I then get an error that that the

   Failed to open PDF file: Main.pdf Errors encountered. No output created. Done. Input errors, so no output created. Error: Failed to open PDF file: Main.pdf Errors encountered. No output created. Done. Input errors, so no output created.

yet Main.pdf is clearly created but os.execute still does not execute(note that adding -shell-execute to the command line is the only change that creates this error). I get a different error from my os.execute too. It is no 1 instead.

ALSO! If I simply use 'copy ...' to try and copy the main.pdf file to a tmp location(thinking maybe it was locked) I get a new pdf but it is not complete as if the buffers were not fully flushed. (if that were the case then I would just get that annoying error message but otherwise everything else would be ok)

To recap:

1. Trying to get os.excute to work. I use shell-escape but this cases other problems
2. My callback is suppose to be called after the pdf is generated BUT when I simply try to copy it, it seems to not copy all of the file. Either the pdf is not being flushed or something is adding additional stuff to the end of the pdf(although I wouldn't expect this and at the very least would expect the original pdf to be valid).

I've tried writing several minutes and the copy still doesn't get the whole file(it's missing about 1kb at the end after some hyperref stuff).

Anyone have any ideas whats going on?

(if I can get this to work I should have the ability to make \include precompilable(so it will not compile the included file unless the tex file changed but will import a precompiled version))

Answer 1

You need to write a wrapper script around your LuaTeX run. The stop_run callback is run before the PDF file is closed / finished, no way to have a correct PDF file at that point (or at any other point, except for after the LuaTeX run.

Answer 2

LuaTeX has a callback that can be used for this.

%! TEX program = lualatex
\documentclass{article}
\usepackage{luacode}
\begin{document}

123

\begin{luacode*}

luatexbase.add_to_callback("wrapup_run", function()
    -- do something with the output PDF here, for example:
    -- io.open(tex.jobname..".pdf", "rb"):read("a")
    -- the line above reads the complete content of the PDF
    -- or os.execute(...)
end, "final callback to rename pdf file")

\end{luacode*}

\end{document}

Note that the documentation (texdoc luatex) states...

Use it at your own risk.

So, if you use this solution be prepared to debug any issue that might happen later in new compiler versions.

---

Looking around, there's an alternative, although it's more ugly.

I notice that the standalone package can execute some shell command to convert the resulting PDF to another file format. https://tex.stackexchange.com/a/51766/250119

Reading its source code, it roughly does the following (assuming main file name is a.tex):

if (flag convert is given) and not (internal run):
    shell escape run("latex 'internal run = true; input{a.tex}'")
    shell escape run("pdf2svg a.pdf a.svg")
    stop program

As such, when the compilation of the main file starts, it spawns another LaTeX process to actually compile the file, then in the parent LaTeX process it does the shell escape to convert, and stop the main process.