# How to do literate programming in TeX?

How can I do literate programming in TeX where TeX, or some variant, is both the used programming language and what is used for typesetting the document? Is there more than one way to do it or different approaches for different TeX variants? I would appreciate some examples.

-
Do you mean using TeX to typeset the documentation form (eg the original tangle/weave)? Or do you mean a literate form of writing TeX macros (of which you might consider the latex doc/docstrip system an example)? –  David Carlisle Mar 6 '12 at 16:29
literate programming in/for what programming language? –  Frank Mittelbach Mar 6 '12 at 16:50
@DavidCarlisle I was mostly thinking of the latter form. I wrote this question in trying to formulate a general question on TeX and literate programming because there was no such question and I was curious. –  N.N. Mar 6 '12 at 16:56
@FrankMittelbach TeX and existing variants of it. –  N.N. Mar 6 '12 at 16:59
it still isn't clear to me what you want, but the author of tweb describes his thing as a web system. it wasn't clear to me at the time (i edited his paper) whether he meant more than the latex .dtx language -- it depends how you look at it. his stuff is on ctan in web/tweb; there are copies of his paper in there, too. –  wasteofspace Mar 6 '12 at 17:18

Given that the question is on how to do literate programming for TeX code then one answer as already mentioned in the comment by David is using doc or xdoc package for documentation and docstrip.tex for generating for generating runable code.

The doc/docstrip combination is not the best possible way to do things (there are better approaches that could be imagined), but it offers one immense advantage, which probably explains why a large proportion of packages these days are documented in this manner:

• It doesn't require any additional program at the user end and therefore works whereever a TeX system of any flavor is implemented.
• It also provides the possibility to generate variants of your code from the same sources, e.g., a production version and a version with additional tracing code.

On the downside

• It requires the code to be structured sequencially (at least unless you want to work artificially with a number of output files).
• It has some somewhat strange conventions (as I see it these days, even if I'm the one put them in). One is that the documentation is hidden behind % signs. The initial reasoning was that you could use such a file directly as code without any processing (the docstrip program was added later.
• Another one is that the code has to be surrounded by

%    \begin{macrocode}
... code ...
%    \end{macrocode}


where the last line has to be %followed by exactly 4 spaces followed by \end{macrocode}.

However, it served fairly well as it seems. Documentation is found as part of the LaTeX distribution.

For documentation look at doc.pdf and docstrip.pdf in the LaTeX distribution and for examples of use take a look at any .dtx file in a TeX installation.

-
Thanks! Could you please include a small example or link to one like Aditya did? –  N.N. Mar 6 '12 at 17:51
@N.N the bells and whistles are so many that it is better you simply take a look at the documentation –  Frank Mittelbach Mar 6 '12 at 21:51
@Yiannis the fact that documentation is hidden behind % is something I would do differently these days, though it was the right choice initially, and a number of smaller scale features could have been realised better (xdoc did some work in that regard) –  Frank Mittelbach Mar 6 '12 at 22:22
@FrankMittelbach Thanks. The % mark is a bit disruptive to the workflow. Another approach is to redefine some of the command creation macros and include part of the documentation in the macro itself (pretty much like it is used in python for example), but maybe this is too radical a change. Do we still need to strip %? –  Yiannis Lazarides Mar 6 '12 at 22:40
@Yiannis a good editor (like emacs) can manage .dtx format quite nicely. Stripping % for speed isn't necessary these days, but docstripdoes much more and allows you to generate different files from pieces of the source etc. –  Frank Mittelbach Mar 6 '12 at 23:06

ConTeXt uses different form of comments to document macro code. For example, lines beginning with %D (followed by space) denote documentation, lines starting frmo %C denote copyright notice, and lines starting with %M denote the macro definitions that are used for compiling the documentation. All ConTeXt base files are documented this way. For example, see syst-aux.mkiv

To generate documentation, you need to run

mtxrun --script module --process <filename>


It is possible to use this feature with plainTeX/LaTeX as well. Process the file using

mtxrun --script module --convert <filename>


This will create a filename.ted file, which looks like follows:

\startmoduledocumentation[type=mkiv,file=syst-aux, version=1996.03.20,title=\CONTEXT\ System Macros,subtitle=General,author=Hans Hagen,date=\currentdate,copyright={PRAGMA ADE \& \CONTEXT\ Development Team}]
\startdocumentation
Some of the macros will move to syst-obs as they might become
obsolete once we've redone the bibliography module. Of course
the handy helpers will stay.

There are some references to \LUA\ variants here but these concern
(often old) experiments, moved from local test modules to here,
cleaned up, but not really used. After all it's not that urgent
and replacing helpers is a delicate process. Don't depend on it.
\stopdocumentation

\startdefinition
\registerctxluafile{syst-aux}{1.001}
\stopdefinition
....
....
\stopmoduledocumentation


You can then define appropriate macros to handle \startdefinition ... \stopdefinition and \startdocumentation ... \stopdocumentation.

-
Does this approach require code to be structured sequencially? –  N.N. Mar 6 '12 at 17:46
@N.N. Yes. The \startdefinition and \startdocumentation block are placed in the same order in which they were defined in file. The main use of this feature is to pretty-print comments for TeX code. –  Aditya Mar 6 '12 at 18:25

I think a good alternative can be noweb. Pros are:

• it is very simple, as simple as it can be;
• it is a tool for real literate programming (doc and docstrip are not);
• it is 'language-independent', so it can be used for every language, TeX and LaTeX too;
• you can write the documentation part in TeX, LaTeX, HTML and troff; but you can use every other markup language (writing appropriate filters);
• it is easily extensible, and every language can be used to extend it: sed, awk, lua, etc. I have written some simple filters for noweb with 15-20 lines of python.

There are not cons in my knowledge.

You can find some examples of noweb code on CTAN.

@wasteofspace : as far as I know tweb is an extension of noweb.

-
If you don't need Noweb's flexibility you could also try my tool Lipsum. It (almost) uses the same syntax as Noweb but installs just as a single binary. Of course, this won't be a concern if you pull Noweb from a Linux distribution. –  Christian Lindig Sep 30 '12 at 18:05
The main con is: It's an extra program. LaTeX's and ConTeXt's approach need no program but TeX. –  Martin Schröder Sep 30 '12 at 18:24

There is the docmfp package that I wrote in 2005 but have not used since I retired later that year. It is an extension of the doc package to cater for the documentation of non-LaTeX code, such as Metafont or Metapost or Java or C or ... It provides for indexing variables, routine names, and so on.

For languages that support multiline comments one simple method is along the lines (where (* starts a language comment and *) ends it):

(*
LaTeX preamble
end of some documentation
\begin{verbatim}
*)
code
code
(*
\end{verbatim}
more documentation
\begin{verbatim}
*)
code
(*
\end{verbatim}
etc
\end{document}
*)


Run the file through the language compiler as is, or through LaTeX after commenting out the initial (*.

-

One should also notice that there is gmdoc, which is based around doc and docstrip, but removes some of its quirks (especially the macrocode environment).

-
Your answer would be better if you would elaborate on gmdoc and compare it to other tools. Currently it is more like comment. –  N.N. Oct 1 '12 at 5:18
@N.N.: true, but I have no experience with doc apart from reading its documentation ages ago... I'll try to expand my answer anyway later today. –  mbork Oct 1 '12 at 13:32