How to do literate programming in TeX?

Question

How can I do literate programming in TeX where TeX, or some variant, is both the used programming language and what is used for typesetting the document? Is there more than one way to do it or different approaches for different TeX variants? I would appreciate some examples.

Answer 1

Given that the question is on how to do literate programming for TeX code then one answer as already mentioned in the comment by David is using doc or xdoc package for documentation and docstrip.tex for generating for generating runable code.

The doc/docstrip combination is not the best possible way to do things (there are better approaches that could be imagined), but it offers one immense advantage, which probably explains why a large proportion of packages these days are documented in this manner:

• It doesn't require any additional program at the user end and therefore works whereever a TeX system of any flavor is implemented.
• It also provides the possibility to generate variants of your code from the same sources, e.g., a production version and a version with additional tracing code.

On the downside

• It requires the code to be structured sequencially (at least unless you want to work artificially with a number of output files).
• It has some somewhat strange conventions (as I see it these days, even if I'm the one put them in). One is that the documentation is hidden behind % signs. The initial reasoning was that you could use such a file directly as code without any processing (the docstrip program was added later.

• Another one is that the code has to be surrounded by

  %    \begin{macrocode}
   ... code ...
  %    \end{macrocode}
  

  where the last line has to be %followed by exactly 4 spaces followed by \end{macrocode}.

However, it served fairly well as it seems. Documentation is found as part of the LaTeX distribution.

For documentation look at doc.pdf and docstrip.pdf in the LaTeX distribution and for examples of use take a look at any .dtx file in a TeX installation.

Answer 2

ConTeXt uses different form of comments to document macro code. For example, lines beginning with %D (followed by space) denote documentation, lines starting frmo %C denote copyright notice, and lines starting with %M denote the macro definitions that are used for compiling the documentation. All ConTeXt base files are documented this way. For example, see syst-aux.mkiv

To generate documentation, you need to run

mtxrun --script module --process <filename>

It is possible to use this feature with plainTeX/LaTeX as well. Process the file using

mtxrun --script module --convert <filename>

This will create a filename.ted file, which looks like follows:

\startmoduledocumentation[type=mkiv,file=syst-aux, version=1996.03.20,title=\CONTEXT\ System Macros,subtitle=General,author=Hans Hagen,date=\currentdate,copyright={PRAGMA ADE \& \CONTEXT\ Development Team}]
\startdocumentation
Some of the macros will move to syst-obs as they might become
obsolete once we've redone the bibliography module. Of course
the handy helpers will stay.

There are some references to \LUA\ variants here but these concern
(often old) experiments, moved from local test modules to here,
cleaned up, but not really used. After all it's not that urgent
and replacing helpers is a delicate process. Don't depend on it.
\stopdocumentation


\startdefinition
\registerctxluafile{syst-aux}{1.001}
\stopdefinition
 ....
 ....
\stopmoduledocumentation

You can then define appropriate macros to handle \startdefinition ... \stopdefinition and \startdocumentation ... \stopdocumentation.

Answer 3

I think a good alternative can be noweb. Pros are:

• it is very simple, as simple as it can be;
• it is a tool for real literate programming (doc and docstrip are not);
• it is 'language-independent', so it can be used for every language, TeX and LaTeX too;
• you can write the documentation part in TeX, LaTeX, HTML and troff; but you can use every other markup language (writing appropriate filters);
• it is easily extensible, and every language can be used to extend it: sed, awk, lua, etc. I have written some simple filters for noweb with 15-20 lines of python.

There are not cons in my knowledge.

You can find some examples of noweb code on CTAN.

@wasteofspace : as far as I know tweb is an extension of noweb.

Answer 4

There is the docmfp package that I wrote in 2005 but have not used since I retired later that year. It is an extension of the doc package to cater for the documentation of non-LaTeX code, such as Metafont or Metapost or Java or C or ... It provides for indexing variables, routine names, and so on.

For languages that support multiline comments one simple method is along the lines (where (* starts a language comment and *) ends it):

(*
LaTeX preamble
 end of some documentation
\begin{verbatim}
*)
code
code
(* 
\end{verbatim}
more documentation
\begin{verbatim}
*)
code
(*
\end{verbatim}
etc
\end{document}
*)

Run the file through the language compiler as is, or through LaTeX after commenting out the initial (*.

Answer 5

One should also notice that there is gmdoc, which is based around doc and docstrip, but removes some of its quirks (especially the macrocode environment).