# Typesetting literate programs: how to enable page breaking mid-paragraph, but not mid-code?

I'm writing a book that is a literate program. A literate program is divided into "chunks" or "sections," each of which has a documentation part and a code part. (It is also permissible, and in my application very useful, to have a chunk that is only documentation, with no code.)

My problem is how to manage the typesetting so that no code part is ever split by a page break, while ensuring that any documentation part may be split in the middle of a paragraph and the final lines of a documentation part appear on the same page as the code they are documenting.

The technique I am using now is an unhappy one. It avoids splitting code parts, but the way it keeps documentation on the same page as code is to avoid splitting a documentation part at all.

The salient features of the implementation are as follows:

• Every documentation part is bracketed in \nwbegindocs and \nwenddocs. These macros do almost nothing—just a little fiddling with indentation.

• A new documentation part normally begins with \nwdocspar. This macro lets LaTeX know that a new part is starting and it is acceptable to have a page break here. At present the default definition of \nwdocspar is \filbreak; the goal is to try to keep the documentation part on the same page as the code part that follows it.

• You can think of \nwbegincode as setting up a special sort of environment for typesetting code. Of the standard environments it most closely resembles \begin{alltt}. It uses \trivlist, then asks LaTeX to respect newlines as follows:

\@@par
\def\par{\leavevmode\null \@@par \penalty\nwcodepenalty}%
\obeylines


The default \nwcodepenalty is \@highpenalty.

Provided each code chunk is small enough to fit on one page, this setup is sufficient to prevent code chunks from being split across pages. But this setup does not result in page breaks inside documentation chunks&emdash;page breaks occur only at \nwdocspar. This situation often leaves a lot of whitespace at the bottom of the page, which I am currently dealing with by sprinkling \nwdocspar more or less randomly throughout the text.

Here is a modest working example that will better illustrate the problem:

\nwbegindocs{0}
\begin{itemize}
\item
The list of integers shows a recursive type;
its~representation is immutable and uses both a record type and a sum
type.
\end{itemize}
\nwenddocs{}%
\nwbegindocs{1}%
\nwdocspar   %   <------------- this is important!

\subsubsection{Two-dimensional points: a mutable abstraction}

Two-dimensional points are a simple abstraction, but even the simplest
cluster demonstrates many of \clu's important features:
\begin{itemize}
\item
The structure of a cluster includes an \emph{export list} in which
each public operation is given a \emph{type}.
\item
Each cluster defines the {\Tt{}rep\nwendquote} type, which is the
representation of the abstract type that is introduced by the cluster.
\item
This cluster uses one of \uclu's built-in \emph{record types}.
\item
The operations of the cluster use \uclu's {\Tt{}seal\nwendquote} and
{\Tt{}unseal\nwendquote} operations, which convert a representation to
an abstraction and vice versa.
\end{itemize}
To~show you something that is not possible in \uscheme\ or \uml,
I~make the points \emph{mutable}: a point can be mutated by rotating
it 90 degrees counterclockwise, or by reflecting it through the origin.
Also, I~can interrogate a point to ask what quadrant of the plane it
is~in.

In cluster {\Tt{}2Dpoint\nwendquote}, a point is represented by a
mutable record containing its $x$ and $y$ coordinates, which are
integers.  This representation need not satisfy any invariants; all
pairs of coordinages are meaningful.
\nwenddocs{}%
%
\nwbegincode{2}%
\moddef{2dpoint.clu}\endmoddef\nwstartdeflinemarkup\nwenddeflinemarkup
(cluster 2Dpoint
[exports
[new      : (int int -> 2Dpoint)]
[get-x    : (2Dpoint -> int)]
[get-y    : (2Dpoint -> int)]
[reflect  : (2Dpoint -> )]
[rotate+  : (2Dpoint -> )]
[quadrant : (2Dpoint -> sym)]
[print    : (2Dpoint -> )]]

\LA{}definition of the \code{}rep\edoc{} of a \code{}2Dpoint\edoc{}\RA{}

\LA{}definitions of the operations of cluster \code{}2Dpoint\edoc{}\RA{}
)
\nwendcode{}%
%
\nwbegindocs{3}%
The types of operations {\Tt{}reflect\nwendquote},
{\Tt{}rotate+\nwendquote}, and {\Tt{}print\nwendquote} are types you
couldn't write in \timpcore\ or \tuscheme---they are functions that
return no results.  In~\clu, just as a function may take any number of
arguments, it may return any number of results.  A~function that
returns no results has no types between the arrow and the closing
bracket.
\nwenddocs{}


The only place that TeX is actually going to put a page break is at the \nwdocspar right before the \subsubsection command. To get decent page breaks I have to insert additional \nwdocspar, e.g., one before every \item and one at the paragraph break. Even then, I can't get a page break in the middle of a paragraph—but at least I use the page more effectively.

This approach does not work very well, and as the size of my manuscript approaches 800 pages, I am dreading the effort of trying to tune every one of those page breaks by hand.

The LaTeX code you see is generated by a preprocessor that I control, and I also control the definitions of the macros. So the question I am trying to ask is what markup can I put in the file, and how can I define the macros, so that the following three goods can be achieved with a limited amount of hand tuning?

• No code chunk is ever split across multiple pages

• When a code chunk follows a documentation chunk with no intervening \nwdocspar, the end of the documentation chunk appears on the same page as the code chunk. (I am using \clubpenalty=10000 and \widowpenalty=10000, and keeping the last two lines of the documentation on the same page as the code would be good enough.)

• Otherwise, a documentation chunk may be split across pages in mid-paragraph, as per typical defaults.

Edited to add: I wonder if something could be done with \vsplit and marks? Although for an 800-page text I worry a little about speed.

• I'm a senior and had a moment. I should have have said minipage instead of mbox. – Peter Wilson May 2 '16 at 18:41
• Isn't this rather reinventing the wheel? Have you looked at any of the existing solutions? I doubt, by the way, that you need to provide a lengthy explanation of literate programming to this particular audience. Coals to Newcastle and all that ... ;). – cfr May 7 '16 at 2:11
• And that is decidedly not a working example. – cfr May 7 '16 at 2:12
• @cfr Sorry about the non-working nature of the example. Preparing this much already took 40 minutes. Obviously I should have done more. (I looked at all the existing solutions when I first ported noweb to LaTeX about 25 years ago. The world of literate programming has not changed since then. But the world of LaTeX has changed deeply. Hence the question.) – Norman Ramsey May 9 '16 at 23:32
• Genius alert!!! – A Feldman Jul 1 '16 at 21:09

## 2 Answers

For the time being, I am going with the standard solution that prevents any code chunk from ever being broken. But to enable line breaks in the middle of paragraphs, I am sprinkling \stdbreak liberally throughout paragraphs, where \stdbreak is defined as follows:

\newcommand\vfilbreak[2][200]{%
\vskip 0pt plus #2 \penalty -#1 \vskip 0pt plus -#2 \relax}
\newcommand\stdbreak{\vadjust{\vfilbreak[2000]{0.5in}}}

• You should post a complete compilable code in the question to make the answer easier (or even possible). Right now this question (and answer) serves close to zero purpose for anyone that's not you. – Manuel Jul 1 '16 at 21:38
• @Manuel a complete and compilable code would be generated by a preprocessor and extremely difficult to read. If you feel the question has so little value, I will not be offended if you vote to close it. – Norman Ramsey Jul 8 '16 at 21:17

An old question without an accepted answer and you find a kind of a solution already. But as it is not closed yet I want to share my ideas.

The command \filbreak is not your friend.

The TeXbook states

The most interesting macro that plain \TeX provides for page make-up is called \filbreak. It means, roughly, "Break the page here and fill the bottom with blank space, unless there is room for more copy that is itself followed by \filbreak."

Therefore the text between several \filbreaks are placed on one page as you have observed in your setup. The command \vfilbreak is more or less a \filbreak in disguise; \filbreak can be used in \stdbreak directly.

I think otherwise the setup looks good to achieve the three items in the bulleted list. The second item is OK, the third can be achieved if the \filbreak is replaced by something like the \stdbreak (although a penalty of $-2000$ seems to be too "high/low"). A \goodbreak might be sufficient if there is enough stretchability between paragraphs, chunks, document and code parts, etc.

Only the first item might cause a problem as code chunks may need a break across a page. You have \nwcodepenalty between the lines and it must be 10000 to prevent page breaks but then overfull pages can occur. If the value is set to 9000, for example, TeX has a chance to break in an emergency but the break can be after the first or the penultimate line of the code part.