One of the often highlighted features in LuaTeX is the ability to replace the default TeX line-breaking algorithm by something else. I wonder if it is similarly possible to replace the page-breaking algorithm? This would be a great feature, since the page-breaking algorithm in TeX is often described as more "greedy" than the line-breaking algorithm, and as not very good at optimizing globally.
EDIT: If this turns out to be possible, can this be used to solve the following problem, highlighted by Frank Mittelbach? I assume no, since Frank does not state so explicitly.
Issue: TEX generates pages based on precompiled paragraph data
This issue describes the fundamental problem in TeX’s approach: the program builds optimized paragraph shapes without any knowledge about their final placement on a page. The result is a “galley” from which columns are cut to a specified vertical size. A consequence of this is that one can’t have the shape of a paragraph depend on its final position on the page when using TEX’s page builder algorithm.
To some extent, it is possible to program around this limitation, e.g., by measuring the remaining space on a page and explicitly changing paragraph shapes after determining where the current textual material will finally appear. However, besides being complicated to implement, it requires accounting for all kinds of special situations that normally would be automatically managed by TEX, and providing “programmed” solutions for them.
As a result, all attempts so far to provide such functionality had to impose strong limitations on the allowed input material, i.e., they worked only in restricted setups and even then, the results were often not satisfactory.