Background: The quality of hyphenation points within a word can vary by a large amount. Especially in languages that allow rather arbitrary combinations of words, such as German. As an example, consider the English word precondition. Hyphenation after prefix pre should be slightly preferred over any other hyphenation within the word stem condition.

The penalty associated with all potential hyphenation points is determined by \hyphenpenalty. How can \hyphenpenalty be changed (or augmented) locally so that TeX associates different penalties with hyphenations during line breaking?

My first attempt was to put a \penalty10 into a \discretionary command, like


But LuaTeX (the target engine) refuses to compile this:

This is LuaTeX, Version beta-0.76.0-2013120414 (rev 4627)  (format=luatex 2013.12.11)  3 APR 2014 15:09
 restricted \write18 enabled.
! Improper discretionary list.
<recently read> }

l.1 pre\discretionary{-\penalty10}
The following discretionary sublist has been deleted:
\penalty 10

I have tried many alternative placements of \penalty or local assignments to \hyphenpenalty, such as


but none of them succeeded. Ideas?

More background: A fully manual solution is perfectly acceptable. If that can be found, the plan is to implement an automatic solution with the help of the padrinoma package, which provides support for pattern driven node list manipulations in LuaTeX. The package is still under development, but you might want to check-out the examples.

Well, the question can be restated: How would a node list look like that associates different penalties with discretionary hyphens?

  • 1
    This has come up since the beginning of TeX:( There were at one point some extension proposals, but not in luatex or xetex. What you can do in luatex (or with a bit more effort in classic tex) is have two sets of hyphenation tables, one just breaking at compound word joins, then if that fails to break the paragraph within tolerance, switch in a second hyphenation table that breaks each word compound. This isn't as good as you ask for, as it's an all-or nothing thing either a paragraph only breaks at word joins or breaks not giving preference to words. there is a TUGboat article I think.. – David Carlisle Apr 3 '14 at 20:33
  • 1
  • 1
  • I've been aware of Sojka's article(s), but not that of Haralambous. The latter contains a lot of interesting ideas! – Stephan Hennig Apr 4 '14 at 16:14
  • A direct link to Haralambous' article: tug.org/TUGboat/tb27-1/tb86haralambous-hyph.pdf – Stephan Hennig Apr 4 '14 at 16:16

Don't know if this qualifies as an answer, but it was too long for a comment. Anyway:

What I was missing, is the following paragraph from the TeXbook:

TeX looks at the parameters that affect line breaking only when it is breaking lines. For example, you shouldn’t try to change the \hyphenpenalty in the middle of a paragraph, if you want TeX to penalize the hyphens in one word more than it does in another word.

That is, \hyphenpenalty kicks-in too late for what I had in mind. \penalty inside \discretionary isn't allowed. And \penalty outside \discretionary, but within a word, seems to have side-effects, like inhibiting ligatures.

So, what are the options to teach TeX about fine-grained hyphenation preferences? Looks like rewriting TeX's line breaking routine is the only one, like Sojka and Haralambous already did. Unfortunately, their work never hit mainstream. LuaTeX as a playground has the nice advantage that parts of the engine can be replaced at run-time. No need to fork-off yet another TeX engine.


In current LuaTeX versions (aka since TeXLive 2016), you can manually overwrite the penalty for individual discretionary nodes using the penalty keyword (it comes directly after `\discretionary, before the pre/post/replace lists).

For example, to break with penalty 10 as pre-condition, 20 for precon-dition and the default for precondi-tion, you would use

pre\discretionary penalty 10{-}{}{}con\discretionary penalty 20{-}{}{}di\discretionary{-}{}{}tion

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.