In a display, the denominator of a fraction is set in “cramped” \textstyle, which can be emulated with \cramped[\textstyle]{...} from the mathtools package. But sometimes the emulation is perfect, while sometimes it shifts the denominator down. Why?


  \rlap{{\color{red}\vrule width #1 height 0.1pt }}%


The denominators line up in this example:
The denominators don't line up in this example:
\frac{\int_a^b k^2 g(x) f(x)\,dx}%
  {\drawbaseline{145pt}\!% <- cancels \, between \hbox{} and \mathop
   \int_a^b k^2 f(x)\,dx}\quad
\frac{\int_a^b k^2 g(x) f(x)\,dx}%
  {\cramped[\textstyle]{\int_a^b k^2 f(x)\,dx}}


cramped denominators

Added tex-core tag because I suspect this has something to do with the primitive \over.

Added clarifications

The problem became clearer after @wipet’s answer and my digging through tex.web and mathtools.dtx. In the second display of my MWE above, the latter denominator has an excessive height of exactly 1.25\fontdimen8\textfont3, which comes from \radical0 {...} (which in turn is what \cramped uses to emulate cramped styles).

Now, \cramped does try to correct this excessive height (and is ususally successful), but any later unboxing will ruin such correction because the height information will be lost. This “unboxing” behaviour is, by mathtools, referred to as

a most unfortunate TeX “feature.” (The faulty reboxing procedure.)

My original question “sometimes the emulation is perfect, while sometimes it shifts the denominator down” was only partially answered by @wipet. Why does TeX’s rebox subroutine seemingly behave differently?

Since @wipet also proposed a solution (i.e., replacing \box\z@ with \hbox{\box\z@}), I wondered if this was the best solution: Note that mathtools’ existing code seems to suggest {}\box\z@, which is a construction used in \mathXlap and \crampedXlap.

  • Will be fixed in mathtools v1.25
    – daleif
    Mar 4, 2021 at 13:35

2 Answers 2


It seems to be a bug in mathtools.sty. The cramped style cannot be given as "native" in classical TeX but it must be emulated by \radical primitive with empty radical symbol. But the constructed box is

\hbox{empty root symbol\vbox{invisible rule (with zero height)
                             \kern normalrulewidth+something
                             \hbox{$given math material$}}}

so, the resulting box is higher by normalrulewidth+something. What exactly means normalrulewith+something is described in TeXbook, Appendix G, step 11.

The mathtools.sty tries to correct this box height by the line 1449 and 1450 of its code (version 2020/03/24 v1.24):

    \advance\dimen@-\ht\z@ \ht\z@=-\dimen@

The problem is that the resulting \box\z@ is reboxed later, so this correction is lost. You can correct this, if you create your own copy of mathtools.sty in your working directory and change the lines 1449 and 1450 to:

   \advance\dimen@-\ht\z@ \ht\z@=-\dimen@

Your first example $k^{2^n}$ is created by somewhat different process and the radical+corrected box is not reboxed later, so the result is correct.

The question "why the box is reboxed sometimes" is answered in Appendix G of the TeXook, step 15a: if w(numerator)<w(denominator) rebox numerator to the w(denominator), if w(numerator)>w(denominator) rebox denominator to the w(nominator). I add note missing in 15a: \hss at both sides on box contents is added during reboxing.

  • Yes, in my second example, the \cramped denominator is moved down by exactly 1.25\fontdimen8\textfont3. So your observation that the \radical box in that example is unboxed is spot on!! However, this still doesn't explain why sometimes unboxing is involved while sometimes it isn't (as in the first example). Despite your excellent detective work, your current answer is incomplete IMHO. So I will leave my question open for a while. Dec 14, 2020 at 20:58
  • Also, the constructed box starts with \kern<height of radical>\hrule height<height of radical>\kern<clearance between bar and radicand>. So you might want to change \kern rulewidth+something in your answer. In the case of \radical\z@, both \kern<height of radical> and \hrule height<height of radical> are zero, so there is only \kern<clearance between bar and radicand>, which is what mathtools tries to correct. (That code was adapted almost verbatim from DEK himself. I wish it had award for finding bugs, LOL) Dec 14, 2020 at 21:04
  • @RuixiZhang OK, I added more detail answer why sometimes a box in generalized fraction is reboxed. Of course, we know appendix G of the TeXbook:).
    – wipet
    Dec 15, 2020 at 7:37
  • The interesting part is then which fix is the correct one? Will adding \hbox cover all the cases here? (I did not write code)
    – daleif
    Dec 15, 2020 at 8:48
  • “The interesting part is then which fix is the correct one?” Indeed! I don't understand why each fix is able to prevent reboxing. More precisely, reboxing do happens (as \hss glue are added) in either fix, but somehow the content remain boxed. Deciding on which fix is correct comes down to understanding how they prevent unhbox of the content. Dec 15, 2020 at 14:51

@RuixiZhang If you believe there is a bug in The TeXbook and/or tex.web (i.e., they don't match), please send me (karl at freefriends dot org) a minimal plain TeX example and description of the discrepancy. Thanks.

  • I mean that TeXbook is OK, tex.web too. See my last comment above. The bug is only in mathtools.sty.
    – wipet
    Dec 16, 2020 at 9:02
  • @wipet “The bug is only in mathtools.sty” Not anymore. Try the following plain TeX code \fontdimen11 \textfont2 =0.5\fontdimen11 \textfont2 $${1\over km}\quad{1\over\smash{km}}\qquad{kmkm\over km}\quad{kmkm\over\smash{km}}$$ \bye The logic of “\box\z@ is wrong” is therefore applicable to plain TeX's \smash. I've sent this example along with other descriptions to Karl. Dec 16, 2020 at 19:24
  • @RuixiZhang This seems to be a bug in plain TeX, not in TeXbook or tex.web. Moreover, I discovered a different behavior of luatex on your example \fontdimen11....
    – wipet
    Dec 16, 2020 at 20:07
  • @wipet For LuaTeX I'm not surprised. I remember LuaTeX uses \UmathXXX instead of \fontdimen, and there are on-going issues reported (tex.stackexchange.com/q/572536 and github.com/latex3/latex2e/issues/432) Dec 16, 2020 at 21:09
  • @wipet “This seems to be a bug in plain TeX, not in TeXbook or tex.web” Well, it's probably too late to change tex.web anyway, so “reboxing” might as well be a “feature” not a bug (hence the mathtools' quote). It's easier to fix plain TeX, so this is likely going to be counted as a plain TeX bug. But remember, TeXbook Appendix B gives the entire plain TeX code, so it is a TeXbook bug in one way or another. We've come full circle… (BTW, \smash, \phantom, \hphantom, \vphantom are subject to the same problem. I hope I can make it to the 2020 tuneup) Dec 16, 2020 at 21:15

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .