# Persuading (la)tex “silently to go beyond the envelope of 'good taste' ” to avoid an overfull hbox

My understanding, based on reading this website, is that (la)tex performs at most three passes when trying to typeset a paragraph: one based on \pretolerance, one based on \tolerance, and one based on \emergencystretch. (I have no knowledge of how the microtype package affects this.) If the third pass, with the least stringent requirements, still cannot typeset the paragraph, tex "gives up" and leaves an overfull hbox.

To a typical reader, who is familiar with the sort of typesetting that shows up in narrow-columned magazines and newspapers (and sometimes textbooks, and...), an overfull hbox is much more obvious than a relaxed requirement on \tolerance or \emergencystretch. Given the speed of modern computers, it seems it would be quite reasonable for tex to continue making passes with progressively relaxed requirements until it either is able to typeset the paragraph, or gets to values so bad that the result actually "sticks out" even more than an overfull hbox. If TeX were to follow this approach, then a warning should probably still be given any time extra passes are required. But if the user is expecting to switch to a different document class later, or prefers to ignore the warning for any number of other reasons, a sloppily typeset paragraph may well feel less like a slap in the face than an overfull hbox.

Thus, the question:
Would it be even remotely feasible to modify the behavior of (La)TeX, preferably by means of a package, so that it performs additional passes, with automatically (and progressively) relaxed parameters, before "giving up" and allowing an overfull hbox?

-
You might prefer to leave tolerance as it is but increase \emergencystretch instead. Unlike tolerance you can increase it and it does not have any affect on good paragraphs, only adds space to paragraphs that are already problematic. –  David Carlisle Nov 26 '12 at 14:29
@DavidCarlisle: I'm familiar with that approach. I've found that when small increases in \emergencystretch do not suffice, the method above gives a better result than large increases. Additionally, when I use the tolerant environment, it only affects a single paragraph. –  Charles Staats Nov 26 '12 at 14:30
OK, can't win 'em all:-) a plan b would be to ask if you require a particular engine luatex gives you an explicit hook into the paragraph breaker so you can break with different paramters and test which one is best, –  David Carlisle Nov 26 '12 at 14:32
A MWE with a problematic paragraph and what you think is an acceptable result would be helpful. –  David Carlisle Nov 26 '12 at 14:36
@DavidCarlisle: I have, with reservations, added a MWE. –  Charles Staats Nov 26 '12 at 20:05

Here is a definition for an environment autotolerantpar which will automatically enlarge \tolerance and \emergencystretch until there is no more overflow.

Note:

• I don't really know whether it'll produce better results than just using \sloppy. Please test.
• It's far from efficient. Only use on single paragraphs.
• It'll break on any fancy stuff like rules or color switches in vertical mode, but should be OK on pure text.
• Adjust with \tolstarttolerance, \tolmaxtolerance, \tolstartemergencystretch, \tolmaxemergencystretch. See below.
• The tolerance and emergencystretch will start at \tolstarttolerance and \tolstartemergencystretch and then simply be doubled until either there is no more overflow or the max values are reached.

Here's the code:

\documentclass{article}

\usepackage{lipsum}

\usepackage{environ}

\makeatletter
\newcount\tolstarttolerance
\tolstarttolerance200
\newcount\tolmaxtolerance
\tolmaxtolerance9999
\newdimen\tolstartemergencystretch
\tolstartemergencystretch\p@
\newdimen\@tolmaxemergencystretch
\newcommand\tolmaxemergencystretch{3em}
\newif\if@toloverflow
\newif\if@tolabort

\NewEnviron{autotolerantpar}
{%
\par
\@tolmaxemergencystretch\tolmaxemergencystretch
\tolerance\tolstarttolerance
\emergencystretch\tolstartemergencystretch
\@tolmakebox
\loop
\if@toloverflow
\@tolaborttrue
\multiply\tolerance\tw@
\ifnum\tolerance>\tolmaxtolerance
\tolerance\tolmaxtolerance
\else
\@tolabortfalse
\fi
\multiply\emergencystretch\tw@
\ifdim\emergencystretch>\@tolmaxemergencystretch
\emergencystretch\@tolmaxemergencystretch
\else
\@tolabortfalse
\fi
\if@tolabort
\global\@toloverflowfalse
\else
\@tolmakebox
\fi
\repeat
\BODY
\par
\ignorespaces
}

\newcommand\@tolmakebox
{%
\setbox\@tempboxa=%
\vbox
{%
\hfuzz=\maxdimen
\hsize\linewidth
\BODY
\par
\global\@toloverflowfalse
\loop
\setbox\@tempboxa=\lastbox
\unless\ifvoid\@tempboxa
\setbox\@tempboxa=\hbox to\linewidth{\unhbox\@tempboxa}%
\global\@toloverflowtrue
\fi
\unskip
\unskip
\unskip
\unpenalty
\repeat
}%
}
\makeatother

\begin{document}

\lipsum[1]
\begin{autotolerantpar}
\mbox{somelongtextwhichcantbebroken}
\mbox{somelongtextwhichcantbebroken}
\mbox{somelongtextwhichcantbebroken}
\mbox{somelongtextwhichcantbebroken}
\mbox{somelongtextwhichcantbebroken}
\mbox{somelongtextwhichcantbebroken}
\mbox{somelongtextwhichcantbebroken}
\mbox{somelongtextwhichcantbebroken}
\mbox{somelongtextwhichcantbebroken}
\mbox{somelongtextwhichcantbebroken}
\mbox{somelongtextwhichcantbebroken}
\mbox{somelongtextwhichcantbebroken}
\mbox{somelongtextwhichcantbebroken}
\mbox{somelongtextwhichcantbebroken}
\mbox{somelongtextwhichcantbebroken}
\mbox{somelongtextwhichcantbebroken}
\mbox{somelongtextwhichcantbebroken}
\mbox{somelongtextwhichcantbebroken}
\mbox{somelongtextwhichcantbebroken}
\mbox{somelongtextwhichcantbebroken}
\mbox{somelongtextwhichcantbebroken}
\mbox{somelongtextwhichcantbebroken}
\mbox{somelongtextwhichcantbebroken}
\mbox{somelongtextwhichcantbebroken}
\mbox{somelongtextwhichcantbebroken}
\mbox{somelongtextwhichcantbebroken}
\mbox{somelongtextwhichcantbebroken}
\mbox{somelongtextwhichcantbebroken}
\mbox{somelongtextwhichcantbebroken}
\end{autotolerantpar}
\lipsum[1]

\end{document}

-
It's quite strange that if you try with \lipsum*[1] in the environment, the trial is performed up to the maximum. –  egreg Nov 26 '12 at 15:57
Probably \setbox\@tempboxa=\hbox to\linewidth{\unhbox\@tempboxa} and testing \ifnum\badness>10000 is better: using \hbox{\unhbox\@tempboxa} sets the box at its natural width and doesn't shrink the glue inside. –  egreg Nov 26 '12 at 16:07
Indeed! Thanks a lot! I'll edit my answer. –  Stephan Lehmke Nov 26 '12 at 16:11
You could add also \hbadness=\@M and \hfuzz=\maxdimen before starting the tests in order to suppress tons of "Overfull" and "Underfull" messages. –  egreg Nov 26 '12 at 16:25
@egreg Good point. Edited again. –  Stephan Lehmke Nov 26 '12 at 16:41

As noted in the comments I usually find that \emergencystretch gives better results than \tolerance. In the example below, overfull lines are produced even for tolerance of 9999 tolerance of 10000 allows the paragraph to be justified but I think gives less pleasing result than \emergencystretch (although both are pretty bad).

It would be interesting to have a test case where tolerance produces a better result.

\documentclass{article}

\def\a{OneLongWord and AnotherLongWord CamelCasedWordsLookUgly. }
\def\b{VeryVeryVeryLongUnbreakable WordsInThisParagraph. }
\def\c{\par \noindent X\dotfill X\par\typeout{===}\hyphenchar\the\font=-1 \a\a\a\a\b\a\b\b\a\a\par}

\begin{document}

\c

{\tolerance 9999 \c}

{\tolerance 10000 \c}

{\emergencystretch\hsize \c}

\end{document}


Using the supplied example, again I think that \emergencystretch has advantages. Paragraphs that use \emergencystrech still produce warnings in the log (unlike increasing \tolerance as paragraphs set with the increased tolerance are 9by default) silently accepted) In both cases though the setting is not ideal and usually there is some way of getting better spacing. In this case you can adjust the url package settings and then set the paragraph with the default settings. I know that you wanted to avoid looking at the specifics of this particular paragraph but I think it is a general technique that using a more flexible hyphenation regime (which is essentially what is happening here in the url) is an alternative to fiddling with the white space.

\def\d{\par \noindent X\dotfill X\par
Moreover, this sort of differential equation, in which some sort of force
goes in the opposite direction from what it controls, shows up all over.
Certainly it occurs in many other physics situations (e.g., springs and pendulums,
as well as the motion of air molecules in sound waves), but also in
areas like economics and ecology.  I have even seen it applied to
romance: see \url{http://opinionator.blogs.nytimes.com/2009/05/26/guest-column-loves-me-loves-me-not-do-the-math/}\par}

\begin{document}

\d

{\expandafter\def\expandafter\UrlBreaks\expandafter{\UrlBreaks\do\-}
\Urlmuskip = 0mu plus 2mu
\d}

{\emergencystretch=2em \tolerance=300 \d}

{\emergencystretch=2em \d}


-
I am, very specifically, not looking for a general discussion of ways to cope with overfull hboxes. I really don't care so much whether \tolerance or \emergencystretch is used. The essence of my question is how to persuade latex, without special per-paragraph user intervention, to do extra passes with progressively relaxed parameters on problematic paragraphs. The goal is that when warnings get ignored, the result should stick out less (both literally and figuratively) to the the typical reader. In fixing the warnings, hyphenation would obviously be an important consideration. –  Charles Staats Nov 27 '12 at 1:28
By the way, why are there only three paragraphs in your picture? –  Charles Staats Nov 27 '12 at 1:30
but my point is I haven't seen an example where gradually relaxing tolerance actually produces a result that is better than simply setting \emergencystretch=\hsize. (the image was manually cropped and I just wanted to show the paragraph with the different layout) –  David Carlisle Nov 27 '12 at 1:49
No I don't think gradually increasing emergencystretch makes much (if any) difference the point of emergencystretch is basically to bring all needed stretch into finite range so tex's least cost algorithm spreads the pain equally rather than getting a chance to dump all the stretch in one infinitely bad line. That is why I would be interested to see any example paragraph where an iterative procedure produces a better result than setting emergencystretch to a fraction less than textwidth. –  David Carlisle Nov 27 '12 at 9:36
He means re-write the text to avoid bad breaks (which is the technique suggested in the TeXBook) which is still the best technique if you are hand polishing output for a book, but less fun if you are typesetting a database dump of thousands of pages. –  David Carlisle Nov 27 '12 at 16:04