# Nested enumerate (enumitem) interferes with suppressing vertical space after theorem heads.

In the post How to suppress vertical space between theorem heads and enumitem environments? David Carlisle suggested a piece of code that helps suppressing vertical space between theorem heads and an enumerate environments. The code worked well, but I recently realized there is now additional vertical space inserted between enumerate environments when they are nested. How can this be avoided (without losing the feature of having vertical space suppressed after theorem heads)?

\documentclass{article}

\usepackage[T1]{fontenc}
\usepackage[american]{babel}
\usepackage{enumitem}
\usepackage{amsthm}

\newtheoremstyle{mythmstyle}%
{0.5em}% space above
{0.5em}% space below
{}% body font
{}% indent amount
\theoremstyle{mythmstyle}
\newtheorem{theorem}{Theorem}

% list settings
\setlist{% general list settings (enumitem's itemize, enumerate, and description)
align=left,% left-aligned enumerate
labelsep=*,% align all item bodies vertically
leftmargin=*,% no left indent
topsep=1mm,% space before enumerate
itemsep=0mm% space between enumerate items
}
\setlist[enumerate,1]{label=\alph*)}% enumerate label on level 1
\setlist[enumerate,2]{label=\roman*)}% enumerate label on level 2

% vertical spacing after theorem heads (suggested by David Carlisle)
\makeatletter
\def\enumfix{%
\if@inlabel
\noindent\par\nobreak\vskip-\parskip\vskip-\baselineskip\hrule\@height\z@
\fi}
\let\oldenumerate\enumerate
\def\enumerate{\enumfix\oldenumerate}
\makeatother

\begin{document}
\begin{theorem}% perfectly fine with David's solution: no vertical space
\begin{enumerate}
\item Foo
\item Bar
\end{enumerate}
\end{theorem}
\bigskip

\begin{enumerate}
\item Foobar
\item \begin{enumerate}% vertically not correctly aligned; comment out David's code above to see how it should look like
\item Foo
\item Bar
\end{enumerate}
\end{enumerate}
\end{document}

-

I suggest you define a new environment for the first level enumerations in theorems, the only one that must be treated specially:

\documentclass{article}

\usepackage[T1]{fontenc}
\usepackage[american]{babel}
\usepackage{enumitem}
\usepackage{amsthm}

\newtheoremstyle{mythmstyle}%
{0.5em}% space above
{0.5em}% space below
{}% body font
{}% indent amount
\theoremstyle{mythmstyle}
\newtheorem{theorem}{Theorem}

% list settings
\setlist{% general list settings (enumitem's itemize, enumerate, and description)
align=left,% left-aligned enumerate
labelsep=*,% align all item bodies vertically
leftmargin=*,% no left indent
topsep=0mm,% space before enumerate
itemsep=0mm% space between enumerate items
}
\setlist[enumerate,1]{label=\alph*)}% enumerate label on level 1
\setlist[enumerate,2]{label=\roman*)}% enumerate label on level 2

% vertical spacing after theorem heads (suggested by David Carlisle)
\makeatletter
\newenvironment{tenumerate}[1][]
{\if@inlabel
\noindent\par\nobreak\vskip-\parskip\vskip-\baselineskip\hrule\@height\z@
\fi
\enumerate[topsep=3pt,#1]}
{\endenumerate}
\makeatother

\begin{document}
\begin{theorem}% perfectly fine with David's solution: no vertical space
\begin{tenumerate}
\item Foo
\item Bar
\end{tenumerate}
\end{theorem}
\bigskip

\begin{enumerate}
\item Foobar
\item \begin{enumerate}
\item Foo
\item Bar
\end{enumerate}
\end{enumerate}
\end{document}


## New way

It seems that a different strategy can work without a new environment:

\documentclass{article}

\usepackage[T1]{fontenc}
\usepackage[american]{babel}
\usepackage{enumitem}
\usepackage{amsthm}
\usepackage{etoolbox}

\newif\ifstarttheorem
\newtheoremstyle{mythmstyle}%
{0.5em}% space above
{0.5em}% space below
{}% body font
{}% indent amount
\theoremstyle{mythmstyle}
\newtheorem{theorem}{Theorem}

% list settings
\setlist{% general list settings (enumitem's itemize, enumerate, and description)
align=left,% left-aligned enumerate
labelsep=*,% align all item bodies vertically
leftmargin=*,% no left indent
topsep=0mm,% space before enumerate
itemsep=0mm% space between enumerate items
}
\setlist[enumerate,1]{label=\alph*)}% enumerate label on level 1
\setlist[enumerate,2]{label=\roman*)}% enumerate label on level 2

% vertical spacing after theorem heads (suggested by David Carlisle)
\makeatletter
\preto\enumerate{%
\if@inlabel
\ifstarttheorem
\mbox{}\par\nobreak\vskip\glueexpr-\parskip-\baselineskip+3pt\relax\hrule\@height\z@
\global\starttheoremfalse
\fi
\fi}
\preto\endenumerate{\global\starttheoremfalse}
\makeatother

\begin{document}

This is not a theorem.
\begin{enumerate}
\item Foo
\item Bar
\end{enumerate}

\begin{theorem}
This is a theorem.
\begin{enumerate}
\item Foo
\item Bar
\end{enumerate}
\end{theorem}

This is not a theorem.
\begin{enumerate}
\item Foo
\item Bar
\end{enumerate}

\begin{theorem}
\begin{enumerate}
\item Foo
\item Bar
\end{enumerate}
\end{theorem}

\begin{theorem}
\begin{enumerate}
\item Foobar
\item \begin{enumerate}
\item Foo
\item Bar
\end{enumerate}
\end{enumerate}
\end{theorem}
\bigskip
\end{document}


Every theorem environment sets a conditional to true; if this conditional is true, an enumerate environment receives David's special treatment and sets the conditional to false. In order not to get false positives, also enumerate sets the conditional to false when it ends.

-
 Thanks for helping, egreg (and mico). Your solution works, but I'm still wondering how it can be done without an additional environment. The disadvantages of an additional environment are: 1) I have to tell my coauthors to use tenumerate instead of enumerate inside theorems (one easily tends to forget that); 2) The nesting is not trivial anymore (using the next enumerate inside tenumerate creates 'a) a)' by default for example); 3) some scientific journals have good templates where this problem does not appear (how do they solve it?). Hmmm... – Marius Hofert Jun 15 '12 at 21:02 @MariusHofert It seems that we have the solution. However more testing might be needed. Adjust the 3pt in the argument to \preto, if you feel it necessary. – egreg Jun 15 '12 at 21:57 Dear egreg, many thanks for helping. That's exactly what I was looking for! I set the argument from 3pt to 0.3em which also nicely scales with font size. – Marius Hofert Jun 15 '12 at 22:45