Why does minipage have different top/bottom margins for math environments? [duplicate]

My goal is to create a framed minipage environment that will contain text and math environments in varying combinations. I noticed that the top (and bottom) margin of the minipage is quite a bit larger if the first (and last) entry is a math environment (than it is if the first thing is normal text).

I tried to extract the relevant parts into a MWE:

\documentclass{article}
\usepackage[fleqn]{amsmath}

\begin{document}
\setlength{\mathindent}{-.15em}
\setlength{\abovedisplayskip}{0pt}
\setlength{\belowdisplayskip}{0pt}
\setlength{\abovedisplayshortskip}{0pt}
\setlength{\belowdisplayshortskip}{0pt}
\fbox{
\begin{minipage}{.1\linewidth}
XXX \\
XXX
\end{minipage}
}
\fbox{
\begin{minipage}{.1\linewidth}
\begin{align*}
\textnormal{XXX}
\end{align*}
\begin{align*}
\textnormal{XXX}
\end{align*}
\end{minipage}
}
\end{document}

Which results in:

So why are there larger top/bottom margins for math environments and how do I get rid of them?