Wrong vertical space in theorem environment closed by an equation

Question

It happens that LaTeX, to fill a page vertically, adds extra spaces around equation environments (and other vertical spaces). Now suppose I have a theorem environment (using amsthm) which ends with an equation, like:

\begin{theorem}
  It holds
  \begin{equation}
    2+2=4
  \end{equation}
\end{theorem}

\begin{proof}
  Straightforward.
\end{proof}

and suppose that this piece of code rendered inside a page which is strechted vertically by LaTeX. Then LaTeX adds the extra vertical space only BEFORE the equation, and not after, and this looks ugly. Where is the problem?

EDIT: The problem appears only if I use a customized theorem style. So the question is: how should I use the command \newtheoremstyle so that the spacing is handled properly? For example

\newtheoremstyle{myplain} {2cm}% ⟨Space above⟩
{2cm}% ⟨Space below⟩
{\itshape}% ⟨Body font⟩
{}% ⟨Indent amount⟩
{\bfseries}% ⟨Theorem head font⟩
{.}% ⟨Punctuation after theorem head⟩
{.5em}% ⟨Space after theorem head⟩2
{}% ⟨Theorem head spec (can be left empty, meaning ‘normal’)⟩

gives the problem above, while

\newtheoremstyle{myplain} {\topsep}% ⟨Space above⟩
{\topsep}% ⟨Space below⟩
{\itshape}% ⟨Body font⟩
{}% ⟨Indent amount⟩
{\bfseries}% ⟨Theorem head font⟩
{.}% ⟨Punctuation after theorem head⟩
{.5em}% ⟨Space after theorem head⟩2
{}% ⟨Theorem head spec (can be left empty, meaning ‘normal’)⟩

does not.

EDIT2: A working example:

\documentclass[a4paper]{amsart}

\usepackage{amsthm}
\newtheoremstyle{myplain} {2cm}% ⟨Space above⟩
{2cm}% ⟨Space below⟩
{\itshape}% ⟨Body font⟩
{}% ⟨Indent amount⟩
{\bfseries}% ⟨Theorem head font⟩
{.}% ⟨Punctuation after theorem head⟩
{.5em}% ⟨Space after theorem head⟩2
{}% ⟨Theorem head spec (can be left empty, meaning ‘normal’)⟩
\theoremstyle{myplain}
\newtheorem{theorem}{Theorem}

\begin{document}
\begin{theorem}
  It holds
  \begin{equation}
    2+2=4
  \end{equation}
\end{theorem}

\begin{proof}
  Straightforward.
\end{proof}

\bigskip \bigskip \bigskip \bigskip \bigskip \bigskip \bigskip \bigskip
\bigskip \bigskip \bigskip \bigskip \bigskip \bigskip \bigskip \bigskip
\bigskip \bigskip \bigskip \bigskip \bigskip \bigskip \bigskip \bigskip
\bigskip \bigskip \bigskip \bigskip \bigskip \bigskip \bigskip \bigskip
\bigskip \bigskip \bigskip \bigskip \bigskip \bigskip \bigskip \bigskip
\bigskip \bigskip \bigskip \bigskip \bigskip \bigskip \bigskip \bigskip
\bigskip \bigskip \bigskip \bigskip \bigskip \bigskip \bigskip \bigskip
\bigskip \bigskip \bigskip \bigskip \bigskip \bigskip \bigskip \bigskip

\begin{gather}
  2\\
  2\\
  2\\
\end{gather}
\end{document}

Answer 1

That's some interesting behaviour you observed! You could regard this as kind of a bug in the LaTeX kernel. So what's happening here? There's some vertical space below the display (\belowdisplayskip) and after the theorem (the \thm@postskip of 2cm that you specified). Now LaTeX is designed such that it doesn't add the sum but the larger one of those two vertical spaces (which is good).

The problem arises in how the larger of the spaces is chosen. This is done in LaTeX's \@xaddvskip, which only compares the "natural length" components of the spaces, but not the available stretch and shrink. In your case, \belowdisplayskip is 4.19998pt plus 4.19998pt and \thm@postskip is 56.9055pt. Usually the latter is larger, but you have a situation where the space is stretched about 128 times more than generally permitted (which is why you get an Underfull \vbox warning!). Therefore, \belowdisplayskip is actually the larger of the two spaces, but \@xaddvskip adds only the smaller \thm@postskip of 2cm below the display.

In my opinion, a good solution is to work with the maximum of the two vertical spaces, in the following sense: use the maximum of the natural and stretch components, and the minimum of the shrink components. I've implemented an appropriate modification of \@xaddvskip in the code below. In the left you see the output of your code, in the right the output produced with the modified \@xaddvskip:

    

You can see that in the improved (right) version there's a bit more space below the equation than above. The reason is that above the equation \abovedisplayskip is used (4.19998pt plus 4.19998pt), and below the maximum of \belowdisplayskip and \thm@postskip, which amounts to 56.9055pt plus 4.19998pt, so about 52.7pt more than above. And that's good, since you wanted additional space below the theorem!

Note that the following code is just a proof of concept, it doesn't work as it should if infinite stretch or shrink components are involved.

\documentclass[a4paper]{amsart}

\makeatletter
\def\@xaddvskip{%
  \ifdim\lastskip<\@tempskipb
    \edef\last@stretch{\the\gluestretch\lastskip}
    \edef\last@shrink{\the\glueshrink\lastskip}
    \edef\next@stretch{\the\gluestretch\@tempskipb}
    \edef\next@shrink{\the\glueshrink\@tempskipb}
    \vskip-\lastskip
    \dimen@=\@tempskipb
    \vskip\dimen@ plus
      \ifdim \last@stretch > \next@stretch
        \last@stretch
      \else
        \next@stretch
      \fi
    minus
      \ifdim \last@shrink < \next@shrink
        \last@shrink
      \else
        \next@shrink
      \fi
  \else
    \ifdim\@tempskipb<\z@
      \ifdim\lastskip<\z@
      \else
        \advance\@tempskipb\lastskip
        \vskip-\lastskip
        \vskip \@tempskipb
      \fi
    \fi
  \fi}
\makeatother

\usepackage{amsthm}
\newtheoremstyle{myplain} {2cm}% ⟨Space above⟩
{2cm}% ⟨Space below⟩
{\itshape}% ⟨Body font⟩
{}% ⟨Indent amount⟩
{\bfseries}% ⟨Theorem head font⟩
{.}% ⟨Punctuation after theorem head⟩
{.5em}% ⟨Space after theorem head⟩2
{}% ⟨Theorem head spec (can be left empty, meaning ‘normal’)⟩
\theoremstyle{myplain}
\newtheorem{theorem}{Theorem}

\begin{document}
\begin{theorem}
  It holds
  \begin{equation}
    2+2=4
  \end{equation}
\end{theorem}

\begin{proof}
  Straightforward.
\end{proof}

\bigskip \bigskip \bigskip \bigskip \bigskip \bigskip \bigskip \bigskip
\bigskip \bigskip \bigskip \bigskip \bigskip \bigskip \bigskip \bigskip
\bigskip \bigskip \bigskip \bigskip \bigskip \bigskip \bigskip \bigskip
\bigskip \bigskip \bigskip \bigskip \bigskip \bigskip \bigskip \bigskip
\bigskip \bigskip \bigskip \bigskip \bigskip \bigskip \bigskip \bigskip
\bigskip \bigskip \bigskip \bigskip \bigskip \bigskip \bigskip \bigskip
\bigskip \bigskip \bigskip \bigskip \bigskip \bigskip \bigskip \bigskip
\bigskip \bigskip \bigskip \bigskip \bigskip \bigskip \bigskip \bigskip

\begin{gather}
  2\\
  2\\
  2\\
\end{gather}
\end{document}

Just for completeness, here's my original e-TeX-free implementation of \getstretch and \getshrink:

\begingroup
  \catcode`P=12
  \catcode`L=12
  \catcode`U=12
  \catcode`S=12
  \catcode`M=12
  \catcode`I=12
  \catcode`N=12
  \catcode`T=12
  \lowercase{
    \def\x{%
      \def\getstretch##1{\expandafter\get@stretch\the##1PLUS 0PT\endmarker}
      \def\get@stretch##1PLUS##2PT##3\endmarker{##2}
      \def\getshrink##1{\expandafter\get@shrink\the##1MINUS 0PT\endmarker}
      \def\get@shrink##1MINUS##2PT##3\endmarker{##2}
    }}
  \expandafter\endgroup\x

Answer 2

The problem is that your "space above" and "space below" have no flexibility, while \topsep has. Add some flexibility:

\documentclass[a4paper]{amsart}

\usepackage{amsthm}
\newtheoremstyle{myplain}
  {2cm plus 1cm minus 0.5cm}% ⟨Space above⟩
  {2cm plus 1cm minus 0.5cm}% ⟨Space below⟩
  {\itshape}% ⟨Body font⟩
  {}% ⟨Indent amount⟩
  {\bfseries}% ⟨Theorem head font⟩
  {.}% ⟨Punctuation after theorem head⟩
  {.5em}% ⟨Space after theorem head⟩2
  {}% ⟨Theorem head spec (can be left empty, meaning ‘normal’)⟩
\theoremstyle{myplain}
\newtheorem{theorem}{Theorem}

\begin{document}
Some text above.

\begin{theorem}
  It holds
  \begin{equation}
    2+2=4
  \end{equation}
\end{theorem}

\begin{proof}
  Straightforward.
\end{proof}

some text for stretching
\pagebreak

\end{document}