Is there a way to apply different penalties to hyphenation points in a word?

Question

Background: The quality of hyphenation points within a word can vary by a large amount. Especially in languages that allow rather arbitrary combinations of words, such as German. As an example, consider the English word precondition. Hyphenation after prefix pre should be slightly preferred over any other hyphenation within the word stem condition.

---

The penalty associated with all potential hyphenation points is determined by \hyphenpenalty. How can \hyphenpenalty be changed (or augmented) locally so that TeX associates different penalties with hyphenations during line breaking?

My first attempt was to put a \penalty10 into a \discretionary command, like

pre\discretionary{-\penalty10}{}{}con\discretionary{-}{}{}di\discretionary{-}{}{}tion
\bye

But LuaTeX (the target engine) refuses to compile this:

This is LuaTeX, Version beta-0.76.0-2013120414 (rev 4627)  (format=luatex 2013.12.11)  3 APR 2014 15:09
 restricted \write18 enabled.
**penalty.tex
(./penalty.tex
! Improper discretionary list.
<recently read> }

l.1 pre\discretionary{-\penalty10}
                                {}{}con\discretionary{-}{}{}di\discretionary...
? 
The following discretionary sublist has been deleted:
\penalty 10

I have tried many alternative placements of \penalty or local assignments to \hyphenpenalty, such as

\discretionary{{\hyphenpenalty=10-}}{}{}

but none of them succeeded. Ideas?

---

More background: A fully manual solution is perfectly acceptable. If that can be found, the plan is to implement an automatic solution with the help of the padrinoma package, which provides support for pattern driven node list manipulations in LuaTeX. The package is still under development, but you might want to check-out the examples.

Well, the question can be restated: How would a node list look like that associates different penalties with discretionary hyphens?

Answer 1

Don't know if this qualifies as an answer, but it was too long for a comment. Anyway:

What I was missing, is the following paragraph from the TeXbook:

TeX looks at the parameters that affect line breaking only when it is breaking lines. For example, you shouldn’t try to change the \hyphenpenalty in the middle of a paragraph, if you want TeX to penalize the hyphens in one word more than it does in another word.

That is, \hyphenpenalty kicks-in too late for what I had in mind. \penalty inside \discretionary isn't allowed. And \penalty outside \discretionary, but within a word, seems to have side-effects, like inhibiting ligatures.

So, what are the options to teach TeX about fine-grained hyphenation preferences? Looks like rewriting TeX's line breaking routine is the only one, like Sojka and Haralambous already did. Unfortunately, their work never hit mainstream. LuaTeX as a playground has the nice advantage that parts of the engine can be replaced at run-time. No need to fork-off yet another TeX engine.

Answer 2

In current LuaTeX versions (aka since TeXLive 2016), you can manually overwrite the penalty for individual discretionary nodes using the penalty keyword (it comes directly after `\discretionary, before the pre/post/replace lists).

For example, to break with penalty 10 as pre-condition, 20 for precon-dition and the default for precondi-tion, you would use

pre\discretionary penalty 10{-}{}{}con\discretionary penalty 20{-}{}{}di\discretionary{-}{}{}tion
\bye