What are penalties and which ones are defined?

Question

Very often you can find solutions by changing the penalty. One is widowpenalty. However what are penalties and is there a small list of every defined penalty.

_{There is a German site which does exactly this.}

Answer 1

Penalties are the main value that TeX tries to minimise when line or page breaking, They may be inserted explicitly (\penalty125 means that the penalty for breaking at that point is 125). Some penalties are built in to the TeX system and inserted automatically, LaTeX for example sets these default values for built in penalties

\linepenalty=10

the penalty for breaking a page after a line within a paragraph

\hyphenpenalty=50

the penalty for line breaking at an automatically inserted hyphen

\exhyphenpenalty=50

the penalty for line breaking at an explicit hyphen

\binoppenalty=700

the penalty for breaking a line at a binary operator

\relpenalty=500

the penalty for breaking a line at a relation

\clubpenalty=150

extra penalty for breaking after first line of a paragraph

\widowpenalty=150

extra penalty for breaking before last line of a paragraph

\displaywidowpenalty=50

extra penalty for breaking before last line before a display math

\brokenpenalty=100

extra penalty for page breaking after a hyphenated line

\predisplaypenalty=10000

penalty for breaking before a display

\postdisplaypenalty=0

penalty for breaking after a display

\floatingpenalty = 20000

penalty for splitting an insertion (can only be split footnote in standard LaTeX)

---

e-tex adds some additional penalties (that are not set by default in LaTeX) but are supported by all three of the major engines

\interlinepenalties

An extension to \interlinepenalty that allows different penalties to be added after each line. It takes a space separated list of numbers as value. If n is positive interlinepenalties n i₁... i_n specifies the penalty to be used after each of the lines in a paragraph, with the last value being repeated as often as needed if the paragraph has more then n lines.

\clubpenalties

An extension of \clubpenalty with the same syntax as \interlinepenalties.

\widowpenalties

An extension of \widowpenalty with the same syntax as \interlinepenalties (lines arecounted backwards from the last line in this case).

\displaywidowpenalties

An extension of \widowpenalty with the same syntax as \widowpenalties with lines counted backwards from each display math within a paragraph.

---

XeTeX supports one additional primitive penalty:

\XeTeXlinebreakpenalty

The full documentation of which is Inter-character linebreak penalty I believe it is the penalty inserted between characters for Asian typesetting which typically doesn't use inter-word spaces.

---

Other penalties are inserted by the format for example LaTeX defines a count

\newcount\interfootnotelinepenalty \interfootnotelinepenalty=100

but locally arranges that \linepenalty is set to this value while setting footnotes.

Similarly article class sets

\@lowpenalty   51
\@medpenalty  151
\@highpenalty 301

which are the values used by \pagebreak[1] ...

\samepage works by setting many of these penalties to "infinite" values:

\def\samepage{\interlinepenalty\@M
   \postdisplaypenalty\@M
   \interdisplaylinepenalty\@M
   \@beginparpenalty\@M
   \@endparpenalty\@M
   \@itempenalty\@M
   \@secpenalty\@M
   \interfootnotelinepenalty\@M}

Note penalties are not just numeric values they are actual nodes that are inserted into the horizontal or vertical list. they can be inspected with \lastpenalty or removed (in some contexts) with \unpenalty so for example \count0=\lastpenalty will put the value of the penalty (even if it was automatically inserted) into the count register if the penalty s the last thing in the current horizontal or vertical list.

For example this bit of LaTeX which tries to insert italic corrections without disturbing the penalties in the horizontal list

\def \fix@penalty {%
  \ifnum \lastpenalty=\z@
    \@@italiccorr
  \else
    \count@ \lastpenalty
    \unpenalty
    \@@italiccorr
    \penalty \count@
  \fi
}

This is part of all the commands such as \textit that try to insert automatic italic correction.

---

Demerits

The statement above that TeX tries to minimise penalties is in fact a simplification. For line breaking, the actual quantity that is minimised is demerits which is (more or less) the sum of the squares of the penalties associated with the linebreaks in a paragraph and the badness of each line (which is a measure of how much white space has been stretched beyond its specified limits) plus three additional demerit parameters as described below. The exact formula is in the TeXBook, but isn't usually relevant.

The three demerit parameters and the default values given to them by LaTeX are:

\doublehyphendemerits=10000

Additional demerit added to the paragraph for each pair of consecutive lines ending in a discretionary (typically a hyphen).

\finalhyphendemerits=5000

Additional demerit added if the penultimate line ends with a discretionary.

\adjdemerits=10000

Additional demerit added for pairs of "incompatible" lines (ie a tight line in which white space is squeezed next to a loose line in which white space is stretched.)

Note, the values of demerit parameters are typically large (they combine with the square of penalty and badness units) also demerits are just a value used in the line breaking calculation. Unlike penalties they do not correspond to nodes that may be manipulated in TeX lists.

Answer 2

Penalties are used by TeX for controlling the line and page break routines. Some of them are inserted implicitly, others can be added by the user (usually via macros). A penalty issued in horizontal or math mode will influence line breaking, one issued in vertical mode will influence page breaking decisions.

The list of "implicit penalties" can be found in the TeXbook (page 272):

\linepenalty amount added to badness of every line in a paragraph (v)
\hyphenpenalty penalty for line break after discretionary hyphen (h)
\exhyphenpenalty penalty for line break after explicit hyphen (h)
\binoppenalty penalty for line break after binary operation (m)
\relpenalty penalty for line break after math relation (m)
\clubpenalty penalty for creating a club line at bottom of page (v)
\widowpenalty penalty for creating a widow line at top of page (v)
\displaywidowpenalty ditto, before a display (v)
\brokenpenalty penalty for page break after a hyphenated line (v)
\predisplaypenalty penalty for page break just before a display (v)
\postdisplaypenalty penalty for page break just after a display (v)
\interlinepenalty additional penalty for page break between lines (v)
\floatingpenalty penalty for insertions that are split (v)

(I've added "h", "m" and "v" to show which ones are added in horizontal, math and vertical mode).

Their default values can be looked at by the usual \show\...penalty.

The LaTeX kernel adds to this list several other penalty parameters:

\@clubpenalty (storage bin)
\interfootnotelinepenalty (storage bin)
\interdisplaylinepenalty (storage bin, used for eqnarray)
\@beginparpenalty put at the beginning of a list (v)
\@endparpenalty put at the end of a list (v)
\@itempenalty put between items in a list (v)
\@secpenalty put before a sectional title (v)
\@floatpenalty used for the float mechanism (in a quite involved way)
\@lowpenalty put when the optional argument to \pagebreak or \linebreak is [1]
\@medpenalty put when the optional argument to \pagebreak or \linebreak is [2]
\@highpenalty put when the optional argument to \pagebreak or \linebreak is [3]

They are used either to insert penalties in appropriate spots (\@itempenalty, \@beginparpenalty and \@endparpenalty in list based environments; \@secpenalty for sectional titles) or as a storage bin for temporarily changing the implicit penalty values.

It's the case, for instance, of \@clubpenalty: LaTeX keeps changing the value of \clubpenalty, so one has to set this new parameter in order to establish a different value for the normally added penalty for discouraging club lines.

Also \interfootnotelinepenalty is a storage bin: the value of this parameter is used as the interline penalty in footnotes.

Of course one can directly insert a penalty manually; for example \nobreak is defined as

\penalty 10000

which inhibits a page or line break if found respectively in vertical or horizontal mode.

However one should keep in mind that penalties are discardable items, so they disappear at page or line breaks (if not used for triggering the break), but discussing this would bring too far. TeX also inserts automatically \penalty-10000 at the end of a paragraph for forcing the final break.