Why does plain TeX set \tolerance larger than \pretolerance?

Question

When TeX breaks a paragraph into lines, it makes three passes. In the first pass, TeX tries to break the paragraph without hyphenation, during which it makes sure that no line has a badness (from glue setting) exceeding \pretolerance. If the first pass fails (i.e., no sequence of break points is found), then TeX tries the second pass with hyphenation enabled, but this time it makes sure that no line has badness exceeding \tolerance.

The plain TeX format sets \pretolerance=100 and \tolerance=200, which are inherited by LaTeX. But why a much larger \tolerance than \pretolerance?

Under this setting, TeX will allow “very loose” lines where glues are stretched to about 126% of their total stretchability (because the cubic root of 200/100 is 1.2599…) when hyphenation is enabled (in the second pass), while TeX only allows glues to stretch to 100% of the total stretchability in the first pass.

This doesn’t make sense: Hyphenation is supposed to help breaking the paragraph better. When hyphenation is enabled, there are much more feasible break points for TeX to choose from. So why allow the possibility of “looser” lines, when the more sensible setting seems to be keeping the same \tolerance=100? (i.e., keeping the same limit on badness of each line, but shouldn’t be a problem because now TeX has more break points to choose from)

---

The following code (LaTeX-friendly) illustrates my question and it partly addresses @UlrikeFischer’s comment.

% There are Unicode characters in the following source.
% Use XeLaTeX or LuaLaTeX to compile.
\documentclass{article}
\begin{document}
\noindent
\vtop{\hsize=250pt \parindent=20pt
\tracingparagraphs=1
When \TeX\ breaks a paragraph into lines, it makes three passes.
In the first pass, \TeX\ tries to break the paragraph without
hyphenation, during which it makes sure that no line has a
badness (from glue setting) exceeding \verb|\pretolerance|.
If the first pass fails (i.e., no sequence of break points is
found), then \TeX\ tries the second pass with hyphenation enabled,
but this time it makes sure that no line has badness exceeding
\verb|\tolerance|.

The plain \TeX\ format sets \verb|\pretolerance=100| and
\verb|\tolerance=200|, which are inherited by \LaTeX. But
why a much larger \verb|\tolerance| than \verb|\pretolerance|?

Under this setting, \TeX\ will allow “very loose” lines where
glues are stretched to about 126\% of their total stretchability
(because the cubic root of 200/100 is 1.2599…) \emph{when
hyphenation is enabled} (in the second pass), while \TeX\ only
allows glues to stretch to 100\%\strut\vadjust{\setbox0\hbox{%
% The baseline of the top-most line in the following margin note
% coincides with the bottom edge of the \strut in the main text.
% Raising \dp\strutbox makes two baselines agree. We then add an
% offset ``-(math-axis ht of 8pt CM)+(math-axis ht of 10pt CM)'',
% so the margin note will share the same axis with the main text
% (i.e., the bars of arrows will be at the same place).
\raise\dimexpr\dp\strutbox-2pt+2.5pt\relax
\vtop{\hsize=90pt \footnotesize\noindent
$\leftarrow$ This line is very loose. The badness is $b=124$
(stretching to $\approx$107.4\%). The paragraph itself has been
hyphenated.}}\ht0=0pt \dp0=0pt \moveright255pt \box0 }
of the total stretchability in the first pass.

This doesn’t make sense: Hyphenation is supposed to \emph{help}
breaking the paragraph \emph{better}. When hyphenation is
enabled, there are much more feasible break points for \TeX\ to
choose from. So why allow the possibility of “looser” lines,
when the more sensible setting seems to be keeping the same
\verb|\tolerance=100|? (i.e., keeping the same limit on badness
of each line, but shouldn’t be a problem because now \TeX\ has
more break points to choose from)
}
\end{document}

Answer 1

The question as stated can indeed be puzzling, but I find the situation clearer if you ask it the other way around: instead of asking “Why set \tolerance larger than \pretolerance?”, ask “Why set \pretolerance smaller than \tolerance?”

That is to say, the second pass, which uses \tolerance, is the “real” one: it solves the true optimization problem, considering all potential places to break lines (i.e. both without and with hyphenation), up to the limit of \tolerance for each line. Hyphenation can be good: it can lead to better line breaks: see an example in this answer, where a single hyphen on line 20 (the penultimate line) was able to improve the range of lines 11–19. Of course hyphenation is also undesirable to some extent, specified by the quantities \hyphenpenalty, \doublehyphendemerits, and \finalhyphendemerits that you can control, or even turn it off entirely with other methods. So this second pass is able to trade off the gains and losses, by considering all potential solutions.

Before this "ideal" second pass, the first (pre-hyphenation) pass is added for efficiency, because finding hyphenation points for each word is (or was) itself an expensive operation. If TeX can find a way of breaking the paragraph without hyphenation with each line having badness less than \pretolerance, then it doesn't even try to hyphenate words. Now a problem arises: if we're going to consider just a subset of solutions (those without hyphenation) and pick the best, then we may miss even better solutions (that could be achieved using hyphenation). So what value should we pick for \pretolerance?

Answer: we should pick for \pretolerance a value significantly lower than \tolerance, so that even when we're going to accept the best solution (if one exists) from the no-hyphenation subset (so it may not be the absolute best), we are guaranteed that it's going to be a reasonably good solution.

---

To summarize, another way of looking at the two quantities is:

• \tolerance: I will only accept solutions where each line has badness below this.

• \pretolerance: I will accept the best no-hyphenation solution, if each line has badness below this.

---

To complete the answer, here is an example where, with \tolerance=200 (the default), setting \pretolerance=100 (the default) rather than \pretolerance=200 (or more) helps matters:

\documentclass{article}
\usepackage{lipsum}
\begin{document}
\tracingonline=1
\tracingparagraphs=2
\message{^^J}
\message{1: global best (at \tolerance= 200):}
\pretolerance=-1
\lipsum[3]

\message{2: no-hyphenation best (at \pretolerance= 200):}
\pretolerance=200
\lipsum[3]

\message{3: no-hyphenation best (at \pretolerance= 100):}
\pretolerance=100
\lipsum[3]

\end{document}

Result (not showing the third paragraph, which is the same as the first one):

Here, the relevant lines from the log file are (I had to look up another answer of mine for a refresher on how to read the output of \tracingparagraphs):

1 global best (at \tolerance = 200):
…
@@64: line 10.2- t=7657 -> @@61

2 no-hyphenation best (at \pretolerance = 200):
@firstpass
…
@@15: line 10.2- t=45051 -> @@14

3 no-hyphenation best (at \pretolerance = 100):
@firstpass
…
@@9: line 7.3 t=4590 -> @@8
@secondpass
@@64: line 10.2- t=7657 -> @@61

— in other words, even though the global best has a total demerit of 7657, if we set \pretolerance=200 (or more), we would end up satisfied with a paragraph that has a total demerit of 45051, which is much worse. Setting \pretolerance=100 makes it so that there are no acceptable solutions without hyphenation, and therefore the global best paragraph will be found.

---

Of course, sometimes even this may not be enough: changing \lipsum[3] to \lipsum[2] in the above file would give a case where, against a global best of 7444, even with \pretolerance=200 we end up accepting a no-hyphenation solution of 16460. Still, this is not as bad as in the above case.

Overall, there are five possibilities qualitatively, and repeating this experiment for the 150 paragraphs of the lipsum package shows at least one instance of each of them:

1. The best paragraph happens to be one without hyphenation (and both \pretolerance=100 and \pretolerance=200 find it): This happens for 37 of the 150 paragraphs.

2. The best paragraph is one with hyphenation, but both \pretolerance=100 and \pretolerance=200 end up finding the same (slightly) worse paragraph without hyphenation: This happens for 38 of the 150 paragraphs.

3. \pretolerance=100 finds no without-hyphenation solution, but \pretolerance=200 finds one, worse than the best solution: This happens for 40 of the 150 paragraphs. This is (by a thin margin) the most common case.

4. \pretolerance=100 finds no without-hyphenation solution, but \pretolerance=200 finds one, the same as the global best solution: This happens for 1 of the 150 paragraphs. This is the only case other than Case 1 in which the best solution is a without-hyphenation one.

5. Neither \pretolerance=100 nor \pretolerance=200 find a without-hyphenation solution, so we end up picking the global best solution: This happens in 34 of the 150 cases.

In cases 1, 2, and 5, setting \pretolerance=100 or 200 makes no difference (to the quality of the paragraph). In the unusual case 4, setting \pretolerance=100 causes some extra computation (a thrown-away first pass) while \pretolerance=200 saves computation (of hyphenation), but quality-wise it doesn't matter. But case 3, the most common case, shows why it's a good idea to set \pretolerance smaller than \tolerance.

Answer 2

The point of pass 1 is efficiency, not to make nicer-looking output. Often we can set paragraphs without having to do the heavy task of hyphenation. If we can do that really well (with low pretolerance) then it is maybe not worthwhile to do it "for real" (with hyphenation) to see if we can get an even better result then.

But if the result without hyphenation is not that good (although not really bad), there is a bigger incentive to try to make it better by introducing hyphenation. That doesn't mean that it so bad we shouldn't tolerate it (\tolerance) if it turns out that we can't really make it that much better.

Answer 3

\tolerance is higher than \pretolerance to give a second pass a higher chance to do a successful line breaking. The value of \tolerance gives \TeX a limit: It is allowed to use lines with a badness up to this limit. It doesn't mean that the text requires this limit.

Sure the possibility to hyphenate words helps \TeX to do the line breaking and usually the badness stays much smaller than the \tolerance. But it is one of two options that \TeX has to break the paragraph. If hyphenation doesn't help, \TeX has in a second pass the second option to use looser lines.

Why is hyphenation not always successful? Hyphenation might be impossible because of a lot of short words linked via ties, or the user has decided to set \uchyph=0, or \hyphenpenalty was increased to 10000, etc. The first few lines are the most critical with respect to line breaking. For example, if \TeX cannot break the word that makes the first line overfull then it has to stretch the interword spaces. In the case that this wasn't possible in the first pass, i.e., it is the reason why \TeX must execute the second pass, then the \pretolerance is too small and \tolerance should have a higher value.

And finally, plain \TeX is a general purpose format that should handle a lot of texts even for inexperienced users who cannot change parameters. A less tolerant setting for the badness might produce too many overfull lines. But if someone does a careful proof-reading and has the willingness to rephrase text that cannot be broken with the current parameter set then the \tolerance can be lowered. On the other hand a careful proof-reading should find paragraphs that contain bad looking, spaced-out lines for the setting \tolerance=200.