How can other programming languages and tools be used to create content for TeX documents?

Question

When writing about programming languages or tools one often wants to include output from them in the document. Then it is useful if such output can be created as the document is compiled. For instance, if one describes a Lua function, then one may want the returned value of that function to be shown in the document. In this case it would be useful if the Lua function could be evaluated as the document is compiled so that the returned value is automatically included in the text.

Another situation of when one wants to use functionality external to TeX and friends is when they are not fit to create or manipulate the content one wants in the document. For example, TeX and friends might not be ideal for analyzing data.

These are two examples of when content for TeX documents are created by other programming languages and tools. How can such approaches be effectively implemented?

Answer 1

Org-mode

In Org-mode, which is a mode available for Emacs, you can mix text with output from code blocks in many different languages and export it all to LaTeX. It is possible to use different languages in the same document and also to send the output from one code block to another. This makes possible for interaction between different languages.

Here is a very simple example in which the first code block is shown as well as its output and the second code block uses the output from the first code block. In this example the code blocks are written in Emacs Lisp. The following is the content of an Org-mode buffer:

* Variable passing

** Example 1

In the following code the variable =foo= is set to the value bar. Since the function =setq= returns the last value it will return bar as shown below.
#+NAME: example-one
#+BEGIN_SRC emacs-lisp :exports both
(setq foo "bar")
#+END_SRC

** Example 2

In the following code the value of =foo= is set to the concatenation of the value of =x= and =x=. The value of =x= is set to the value returned in the example above (this happens in Org-mode).
#+NAME: example-two
#+BEGIN_SRC emacs-lisp :var x=example-one :exports both
(setq foo (concat x x))
#+END_SRC

When pressing C-c C-e d Org-mode will export to LaTeX, compile it and show the resulting document. Here is part of the LaTeX output, note that the Lisp code is followed by its returned value:

\section{Variable passing}
\label{sec-1}
\subsection{Example 1}
\label{sec-1-1}


In the following code the variable \texttt{foo} is set to the value bar. Since the function \texttt{setq} returns the last value it will return bar as shown below.

\begin{verbatim}
(setq foo "bar")
\end{verbatim}




\begin{verbatim}
 bar
\end{verbatim}
\subsection{Example 2}
\label{sec-1-2}


In the following code the value of \texttt{foo} is set to the concatenation of the value of \texttt{x} and \texttt{x}. The value of \texttt{x} is set to the value returned in the example above (this happens in Org-mode).

\begin{verbatim}
(setq foo (concat x x))
\end{verbatim}




\begin{verbatim}
 barbar
\end{verbatim}

And here is part of the resulting document:

This example is very simple and it does not show the full potential of this approach. It is possible to make much more complex interactions between languages and tools. Both of the following articles are written in Org-mode and exported to LaTeX and they show complex examples of how it can be used to analyze data and such:

• A Multi-Language Computing Environment for Literate Programming and Reproducible Research. Contains three examples. The first example gets the values of Pascal's triangle using Python, draws it using dot and tests if it is correct using Emacs Lisp. The second is an example of literate programming for C. The third example collects temperatures using shell commands, creates a database using sqlite, analyzes the data using R and draws a graph of it.

• Active Documents with Org-Mode. Contains an example that collects baseball data using shell commands, analyzes it using Python, awk and R and draws a diagram. The source for this article is available at https://raw.github.com/eschulte/CiSE/master/org-mode-active-doc.org.

Answer 2

Bashful

The bashful package is specifically designed for including command line session output in documents, so can be used to display the output of any other program that produces command line output.

Quoting from the user manual:

...bashful provides a convenient interface to TEX’s primitive \write18—the execution of shell commands from within your input files, also known as shell escape. Text between \bash and \END is executed by bash, a popular Unix command line interpreter. Various flags control whether the executed commands and their output show up in the printed document, and whether they are saved to files.

Although provisions are made for using shells other than bash, this package may not operate without modifications on Microsoft’s operating systems.

Note that this requires -shell-escape option to be specified.

Example:

Here is a demonstration using basic bash commands. The documentation provides more elaborate examples.

The first line of the bash script file begins with a %, so I started the listings form line 2, and left the first line blank. When you initiate the \bash command, you provide a file name for the .sh bash file and the output file. Then use lstinputlisting to reincorporate the contents of that file back into the .tex file.

\documentclass{standalone}
\usepackage{xcolor}
\usepackage{bashful}
\usepackage{listings}

\lstdefinestyle{BashInputStyle}{
  language=bash,
  firstline=2,% Supress the first line that begins with `%`
  basicstyle=\small\sffamily,
  numbers=left,
  numberstyle=\tiny,
  numbersep=3pt,
  frame=tb,
  columns=fullflexible,
  backgroundcolor=\color{yellow!20},
  linewidth=0.9\linewidth,
  xleftmargin=0.1\linewidth
}

\lstdefinestyle{BashOutputStyle}{
  basicstyle=\small\ttfamily,
  numbers=none,
  frame=tblr,
  columns=fullflexible,
  backgroundcolor=\color{blue!10},
  linewidth=0.9\linewidth,
  xleftmargin=0.1\linewidth
}


\begin{document}
\bash[verbose,scriptFile=hello.sh,stdoutFile=hello.tex]

echo "Hello World!"
echo "Today is" `date`
echo ""
echo "Disk usage is:"
df
\END
\par\noindent
Executing the following code in \textbf{bash}
\lstinputlisting[style=BashInputStyle]{hello.sh}
%
yields the following output:
\lstinputlisting[style=BashOutputStyle]{hello.tex}
\end{document}

Answer 3

For ConTeXt, I have written a module filter that allows you to easily call external programs and include the result back in TeX. The main feature of this module, as opposed to manual \write18 calls are:

• It provides a nice key-value driven syntax. See the README file on github for details.
• It caches the results and reruns the external program only if the input has changed.

Using this module, you can use any external program to create content to be included in TeX.

For example, you can use the filter module to show the input and output of programs side by side.

\usemodule[filter,vim]                                                                                                                 

\definevimtyping[prettyRUBY][syntax=ruby]
\defineexternalfilter
  [RUBY]
  [
    filter={ruby \externalfilterinputfile\space > \externalfilteroutputfile},
    cache=yes, % Do not rerun the program if the file has not changed
    readcommand=\TypesetAndPrint,
  ]

\def\TypesetAndPrint#1%
    % The filter module expects #1 to be the 
    % file to read. But since we want to prettyprint
    % the input as well, I ignore #1 and use
    % `\externalfilterinputfile` and `\externalfilteroutputfile`
    % instead
    %
    % A simple way to typeset the output is as follows
    % {\typeprettyRUBYfile{\externalfilterinputfile}%
    %  \blank
    %  \typefile{\externalfilteroutputfile}}
    % but we use a slightly more elaborate scheme and show the code
    % and output side by side
    {\startlinecorrection
     \startcombination[2]
        {\typeprettyRUBYfile{\externalfilterinputfile}}{Ruby program}
        {\typefile{\externalfilteroutputfile}}{Output}
     \stopcombination
     \stoplinecorrection}

which can be used as

\starttext
A ruby program
\startRUBY
puts "Hello World"
\stopRUBY
\stoptext          

and gives

Note that I am using the vim module to pretty-print the source code. The same approach can be used to, for example write content in markdown and use pandoc to convert it to TeX (see this example) or use R to generate graphics (see this example).

Answer 4

Showexpl

If you want to be able to show LaTeX code and its output, the best way I know is to use the LTXexample environment from the showexpl package:

\documentclass{article}
\usepackage{amsmath}
\usepackage{xcolor}
\usepackage{showexpl}

\lstdefinestyle{myLatexStyle}{
    language=TeX,
    basicstyle=\small\ttfamily,
    backgroundcolor=\color{yellow},
    numbers=left, numberstyle=\tiny, stepnumber=2, numbersep=5pt,
    commentstyle=\color{red},
    showstringspaces=false,
    keywordstyle=\color{blue}\bfseries,
    morekeywords={align,begin},
    pos=l
}

\begin{document}
\begin{LTXexample}[style=myLatexStyle]
\par\noindent
Inline math $e = mc^2$ is part 
of a sentence.
% This is a comment
\par\noindent
Display math:
\begin{align*}
    e &= mc^2 \\
    F &= ma
\end{align*}
\end{LTXexample}
\end{document}

Answer 5

I read the question (title) as using some other language to create a TeX document, so this answer is like that.

The language is Ruby, and the library is kramdown which is a library for parsing and converting a superset of Markdown. It includes a LaTeX converter by default, but I wanted to try my hand at making a converter for my own TeX format (which is basically plain-xetex with some additions).

If you look at the sources for the LaTeX converter, you'll see how delightfully easy to approach it is. At least I did, unlike Pandoc's, but that could well be because I'm more familiar with Ruby than I am with Haskell.

Inside the class, one could just handle the conversion in a single method, but like in the LaTeX converter, one could use that to dispatch to a conversion method for a specific element type.

Here's an example of an list element for plain-xetex:

def convert_ul(el, opts)
  opts[:level] = opts[:level].to_i.succ
  retval = "\\smallskip\n" << inner(el, opts) << '\smallskip'
  opts[:level] -= 1
  return retval
end
alias :convert_ol :convert_ul

def convert_li(el, opts)
  @ulmarks ||= %w(• ⬦ ‒)
  type = "\\#{"item" * opts[:level]}"
  case opts[:parent].type
    when :ul : "#{type}{#{@ulmarks[opts[:level]-1]}} "
    when :ol : "#{type}{#{opts[:index]+1}.} "
  end << inner(el, opts)
end

Yes it's not perfect ;-), but shows how easy it is to start working on the conversion, and start to get real, tangible results!

But the interesting thing I noticed when starting to do this, was that I started to wonder how much of the TeX code I want to create on this conversion step (think about verbatim in footnotes, for example, and other similar \catcode madness, or TOC, or refs, etc.), and how much I'd want to do as TeX macros. This isn't very clear-cut (for me, at the moment at least); on one hand, one could really go to the extremes and use nothing but the primitives (which would be pretty cool I think), or at the other extreme, build a macro for everything. This realization was really big for me, as it opens up a lot of possibilities.

A ConTeXt converter would be pretty cool to make, too.

Answer 6

knitr

knitr allows you to embed R programs into a LaTeX document as "code chunks". You can then specify whether you want the code, the result, or both to appear in the final document. You can also use chunks to generate and insert plots.

The source document is an .Rnw file. This is the usual LaTeX format, with the addition of code chunks that will be evaluated by R.

Example Rnw file

\documentclass{article}
\usepackage[margin=1.5in]{geometry}

\begin{document}

By default, the code and its result get marked up in the resulting \texttt{.tex} file.

<<default>>=
summary(aov(Sepal.Length ~ Species, data = iris))
@ 

You can suppress the code:

<<just-the-output, echo=FALSE>>=
t.test(Sepal.Length ~ Species, data = subset(iris, Species != "virginica"))
@ 

You can suppress the output:

<<just-the-code, results="hide">>=
my.result <- lm(Sepal.Length ~ Sepal.Width, data = iris)
summary(my.result)
@ 

\pagebreak{}

Figures are automatically inserted in the text. They are automatically placed in a figure
environment, which can be configured in the chunk header.

<<plot,fig.height=4, fig.width=5, fig.align='center'>>=
plot(iris[,1:2], pch = unclass(iris$Species))
@ 

\end{document}

After processing by knitr, you will get a .tex file, which can then be compiled as per usual. The example above produces:

The typical knitr work-flow uses editors that handle all the processing steps, so you work only with the .Rnw file. The conversion from .Rnw -> .tex -> .pdf is automatic. There are a number of editors with built-in support. Rstudio is a popular option with a very nice gui. Emacs with ESS is another strong alternative: video demo.