How can I save shell output to a variable in LaTeX? [duplicate]

Question

I need to be able to execute a shell command through LaTeX [*] and have the resulting output available for use later on in the form of a LaTeX 'variable' (e.g., via a subsequent \newcommand command). If necessary a temporary file could be used to hold the output, although I would like to avoid this. I have been able to use

\immediate\write18

to invoke a bash command before, but this was only used to write to a temporary file.

[*] I realise that the -shell-escape flag must be set for this to be allowed

Answer 1

TeX has support for file IO, and you can take advantage of this. To create a new input filehandle, you execute \newread\readFH; at this point, \readFH is a number representing a channel on which you can read or write (you've already seen one of these, the special channel 18). To open the file, you run \openin\readFH=filename.ext; now, reading from channel \readFH will read lines from filename.ext. To actually read from the file, you run \read\readFH to \nextline; this reads one line from \readFH and puts it in \nextline. (With one caveat—see below.) Closing the file is then done with \closein\readFH.

Note that this treats newlines as spaces; if you have a file containing e.g.

foo
bar

and read a line into \nextline, it will be as though you wrote \def\nextline{foo }. To avoid this, you set \endlinechar to -1.

Overall, then, your example would look like this:

\newread\myinput
% We use '\jobname.temp' to create a uniquely-named temporary file
\immediate\write18{some command > '\jobname.temp'}
\openin\myinput=\jobname.temp
% The group localizes the change to \endlinechar
\bgroup
  \endlinechar=-1
  \read\myinput to \localline
  % Since everything in the group is local, we have to explicitly make the
  % assignment global
  \global\let\myresult\localline
\egroup
\closein\myinput
% Clean up after ourselves
\immediate\write18{rm -f -- '\jobname.temp'}
\dosomething{\myresult}

You could probably abstract this into a macro, but precisely what parts ought to be parametrized probably depends on your specific use case.

The one caveat I mentioned: \read operates on balanced TeX text. This means that it will read more than one line if it has to in order to match braces, and it will finish all those lines. Thus, a single read from the file

Hello { all
you } people.

will result in the string Hello { all you } people. 

Also, just for future reference: creating filehandles to write to is done with \newwrite, you open them with \immediate\openout\writeFH=filename.ext, you write to them with \immediate\write\writeFH{some text}, and you close them with \immediate\closeout\writeFH. Note that some text is actual TeX; thus, macros are expanded and printing out e.g. backslashes or unbalanced curly brackets is non-trivial. (For backslashes, there's the LaTeX macro \@backslashchar; see the TeX Stack Exchange question "How to make a real backslash (escape) character" for more general information. The short version is "play with catcodes.")

Furthermore, note the use of \immediate for the writing commands but not the reading commands; if you don't use \immediate with these, they'll be delayed until the output routine runs (that is, when TeX decides that the current page is finished and can be shipped out to the DVI/PDF). According to chapter 21 of The TeXBook, "The reason for this delay is that \write is often used to make an index or table of contents, and the exact page on which a partcular item will appear is generally unknown when the \write instruction occurs in mid-paragraph." (Pg. 227.) For \read and friends, this clearly isn't a concern, hence the difference in behavior.

Answer 2

If you are using LuaLaTeX, you can do something like this:

\documentclass{article}
\begin{document}
\edef\out{\directlua{
  j = io.popen(" ls -l")
  tex.write(j:read("*all"))
  j:close()
}}
\out
\end{document}

Answer 3

I don't think you can have LaTeX run a shell command as part of its typesetting process, but I'm completely wrong it turns out, LaTeX does support that. You could (and I would still prefer, despite having this new information) have your build system run the shell command and store it's output in a file, and use

\input{filename}

Answer 4

You can then read the contents of the \write18d temporary file by opening the file (\openin) and then reading the file into a cs a line at a time using \read. Note that \read more-or-less asummes its input takes Tex syntax: in particular it will read from an open brace to the matching close brace. If this is a problem, you might fix it by messing about with catcodes.

Answer 5

I'm not very happy with the special mechanism of write18 (perhaps I haven't explored it enough). Instead I write my tex as noweb files, which can contain code chunks and documentation chunks. Then I have relatively simple Makefile (gnu) which interact the following way:

1. All code chunks get split out by notangle (into a special 'code' directory), and get their exec bit chmod'ed.

2. With a different target in the Makefile the required programs get compiled (if it is C) or just run if it is shell or awk...and produce tex output.

3. Then this tex output, tables, calculations, figures, whatever gets included back to the latex document.

4. pdflatex is run a couple of times (on the noweave'd document) to resolve forwards refs and whatever.

The end result is that when I work with some data set, producing statistical calculations, pictures, I can just say make and I get all the data, all the calculations, all the pictures refreshed if necessary, sometimes after hours(!).

The approach is not perfect yet, I don't know a good way to communicate from the latex file to the Makefile and back things like where in the filesystem does you data reside, and certain data must be duplicated which is error prone, but I am pretty happy that I can stuff in absolutely all code which processes my data into the document, expliain the necessary part in sufficient detail, refer to the document where I describe why the issue arose (separate chapter), with all parameters for the runs, and just type make. Hope this helps.