# Accessing to .log messages from LuaTeX. Is it possible?

Reading the question How to visualize the underfull box in final ouput PDF files I thought that it could be possible to use LuaTeX to intercept the "Underfull vbox" message and draw some kind of overlayed mark (using tikz or whatever) at that point. Probably not very useful, but fun to try as a proof of concept.

Before dealing with the appropriate point at which introduce the mark, or the pattern expression required to capture the relevant info, I tried first a simpler goal, but I was unsuccessful.

My goal was, from lualatex, write a document which produces a copy of the .log file into another .lbk file (or any other extension, that's not relevant). I tried the following approaches:

1. I wrote a lua function copylog() which simply opens the .log, read all its contents, and write those back in other file:

function copylog()
local f = io.open(tex.jobname .. ".log", "r")
local fo = io.open(tex.jobname .. ".lbk", "w")
f:close()
fo:write(s)
fo:close()
end

2. In the document I wrote \AtBeginDocument{\directlua{copylog()}} (I wanted to read the log from the previous compilation).

This was partially successful. The resulting file contained part of the log, but not complete, it stopped a few lines before than the \begin{document} was processed.

I realized that I was not reading the log from previous compilation, but instead the log of the current compilation. As soon as lualatex was run, a new log file was opened, a lot of contents from the preamble was written and then my lua code read all that stuff (but not what will come later).

3. So I changed the point at which my function was executed and wrote: \AtEndDocument{\directlua{copylog()}}, trying to process the log from the current compilation, when it was done.

This failed again. This time I got a few lines more from the log file, but not the complete log file. Somehow I expected that, because at the point where my lua code reads the log it is still not complete (extra info and statistics are dumped even after the end of the document), but at least I expected it to contain part of the warning messages generated by my document. No luck, the copy ended again before \begin{document} was run.

I realized that this was due to the buffered output. Luatex already wrote the information I wanted in the file, but this info was still in some file buffers, not actually written in the disk, so not accessible to my lua function.

4. Since \end{document} was still to early to hook my function, I tried also using the end_run callback, but got the same results. At end_run the log file is still open, so I cannot get its full contents (because of the buffered output again).

From these experiments I concluded two questions:

1. Is it possible to read the .log from the previous compilation before lualatex opens it (and cleans it) again? I guess the answer is no. Apparently the .log file is the first output which is open, even before the document is loaded.
2. It is possible to force, from lua or from tex, a flush() of the log file from lua, to be able to read all the messages generated up to \end{document}?

If the two question above have negative answer, then new questions arrive:

1. There is some callback I could use to get the log messages or terminal output as they are written? I tried the obvious ones: process_output_buffer and show_error_hook. The first one didn't get any result. The second one was only activated by error messages, not by warnings.
2. Any other idea? I only want to capture the warning messages. Should not be that difficult...

Any expert in luatex internals can help me?

-
The normal way to pass information from one compilation to the next is through the .aux file. Maybe you could hook into the macro that warns about underfull vboxes and write something to the aux file. But if you want to mark them...can't this be done with an output routine or something? –  Matthew Leingang Mar 13 '13 at 12:22
@MatthewLeingang As far as I know, the "Undefull" message is not written by any macro, but by TeX "kernel" itself (see chunk 674 of "TeX: The Program" book, or the web source) –  JLDiaz Mar 13 '13 at 12:30
@MatthewLeingang There is no macro that warns about underfull boxes: the messages come from deep in TeX; however, the idea seems good if there is a callback to the function that emits such messages. –  egreg Mar 13 '13 at 12:36
@Herbert I searched (before posting the question) through the LuaTeX doc, but to no avail. Could you be more specific? The only places in that document where the log file is mentiones is in relation with find_write_file (apparently to change the file which will receive the log messages, but not to capture the log message), and process_output_buffer callback, but only to say that this callback does not receive log messages :-( –  JLDiaz Mar 13 '13 at 14:20
Maybe you are right, I didn't tried: However, file a feature request to the LuaTeX developer for show_warning_hook –  Herbert Mar 13 '13 at 15:41

From the comments received so far, and my own conclusions after reading the LuaTeX manual, I'm now convinced that the log file cannot be read reliably from lua code being run in the same document whose log file I'm trying to read.
Opening that file from lua and reading with file.read() is not reliable because the same file is open and being written in by TeX, and since file output is buffered, even if I read the file at the end of the run, I don't get its full contents.
The other approach, to use some callback to get the log messages while they are written in the log file, is unfeasible because LuaTeX does not provide any callback to process log messages. The closest thing is show_error_hook but this callback is only called at the error messages and not at the informative messages nor warning messages.
One feasible approach (but not fully robust) would be to print a very long \message at the end of the compilation, so as to exceed the buffer and cause LuaTeX to flush the contents into the log file, then read the partial log file, which contains all of the information you want. –  Bruno Le Floch Jun 12 '13 at 10:16