How do I correctly align this long multi-line equation?

I wish to format the following equation:

\documentclass{article}

\usepackage{amsmath}

\begin{document}
\centering
\begin{align*}
\det\left(\mathbf{A}_{\mp}\right)
&=
\det
\begin{pmatrix}
a_{2,1} \mp a_{1,1}\\
a_{2,2} \mp a_{1,2}\\
a_{2,3} \mp a_{1,3}\\
\end{pmatrix}
=
\det
\begin{pmatrix}
a_{2,1}\\
a_{2,2} \mp a_{1,2}\\
a_{2,3} \mp a_{1,3}\\
\end{pmatrix}
\mp
\det
\begin{pmatrix}
a_{1,1}\\
a_{2,2} \mp a_{1,2}\\
a_{2,3} \mp a_{1,3}\\
\end{pmatrix}\\
&=
\det
\begin{pmatrix}
a_{2,1}\\
a_{2,2}\\
a_{2,3} \mp a_{1,3}\\
\end{pmatrix}
\mp
\det
\begin{pmatrix}
a_{2,1}\\
a_{1,2}\\
a_{2,3} \mp a_{1,3}\\
\end{pmatrix}
\mp\left\{
\det
\begin{pmatrix}
a_{1,1}\\
a_{2,2}\\
a_{2,3} \mp a_{1,3}\\
\end{pmatrix}
\mp
\det
\begin{pmatrix}
a_{1,1}\\
a_{1,2}\\
a_{2,3} \mp a_{1,3}\\
\end{pmatrix}
\right\}\\
&=
\det
\begin{pmatrix}
a_{2,1}\\
a_{2,2}\\
a_{2,3}\\
\end{pmatrix}
\mp
\det\begin{pmatrix}
a_{2,1}\\
a_{2,2}\\
a_{1,3}\\
\end{pmatrix}
\mp\left[
\det
\begin{pmatrix}
a_{2,1}\\
a_{1,2}\\
a_{2,3}\\
\end{pmatrix}
\mp
\det
\begin{pmatrix}
a_{2,1}\\
a_{1,2}\\
a_{1,3}\\
\end{pmatrix} \right]\\
&\mp
\left\{
\det
\begin{pmatrix}
a_{1,1}\\
a_{2,2}\\
a_{2,3}\\
\end{pmatrix}
\mp
\det\begin{pmatrix}
a_{1,1}\\
a_{2,2}\\
a_{1,3}\\
\end{pmatrix}
\mp\left[
\det
\begin{pmatrix}
a_{1,1}\\
a_{1,2}\\
a_{2,3}\\
\end{pmatrix}
\mp
\det\begin{pmatrix}
a_{1,1}\\
a_{1,2}\\
a_{1,3}\\
\end{pmatrix}
\right]
\right\}\\
&=
\det
\begin{pmatrix}
a_{2,1}\\
a_{2,2}\\
a_{2,3}\\
\end{pmatrix}
\mp
\det\begin{pmatrix}
a_{2,1}\\
a_{2,2}\\
a_{1,3}\\
\end{pmatrix}
\mp
\det
\begin{pmatrix}
a_{2,1}\\
a_{1,2}\\
a_{2,3}\\
\end{pmatrix}
+
\det
\begin{pmatrix}
a_{2,1}\\
a_{1,2}\\
a_{1,3}\\
\end{pmatrix}\\
&\mp
\det
\begin{pmatrix}
a_{1,1}\\
a_{2,2}\\
a_{2,3}\\
\end{pmatrix}
+
\det
\begin{pmatrix}
a_{1,1}\\
a_{2,2}\\
a_{1,3}\\
\end{pmatrix}
+
\det
\begin{pmatrix}
a_{1,1}\\
a_{1,2}\\
a_{2,3}\\
\end{pmatrix}
\mp
\det
\begin{pmatrix}
a_{1,1}\\
a_{1,2}\\
a_{1,3}\\
\end{pmatrix}
\end{align*}
\end{document}

My approach would be to align all of the 'det' terms of the first two and last four lines. I've already tried to achieve this by using nested alignedat environments and vphantom/hphantom. However, there are still a lot of misalignments:

\documentclass{article}

\usepackage{amsmath}

\newcommand{\hspaceone}{\hphantom{\left\{\vphantom{\det
\begin{pmatrix}
a_{1,1}\\
a_{1,2}\\
a_{1,3}\\
\end{pmatrix}}\right.}}

\newcommand{\hspacetwo}{\hphantom{\left\{\vphantom{\det
\begin{pmatrix}
a_{1,1}\\
a_{1,2}\\
a_{1,3}\\
\end{pmatrix}}\right.}}

\newcommand{\vspaceone}{\vphantom{\det
\begin{pmatrix}
a_{1,1}\\
a_{2,2}\\
a_{2,3} \mp a_{1,3}\\
\end{pmatrix}}}

\newcommand{\vspacetwo}{\vphantom{\det
\begin{pmatrix}
a_{1,1}\\
a_{2,2} \\
a_{2,3} \mp a_{1,3}\\
\end{pmatrix}}}

\newcommand{\vspacethree}{\vphantom{\det
\begin{pmatrix}
a_{2,1}\\
a_{1,2}\\
a_{2,3}\\
\end{pmatrix}}}

\newcommand{\vspacefour}{\vphantom{\det
\begin{pmatrix}
a_{2,1}\\
a_{1,2}\\
a_{1,3}\\
\end{pmatrix}}}

\newcommand{\vspacefive}{\vphantom{\det
\begin{pmatrix}
a_{1,1}\\
a_{1,2}\\
a_{1,3}\\
\end{pmatrix}}}

\newcommand{\vspacesix}{\vphantom{\det
\begin{pmatrix}
a_{1,1}\\
a_{1,2}\\
a_{2,3}\\
\end{pmatrix}}}

\newcommand{\vspaceseven}{\vphantom{\det
\begin{pmatrix}
a_{1,1}\\
a_{1,2}\\
a_{1,3}\\
\end{pmatrix}}}

\newcommand{\vspaceeight}{\vphantom{\det
\begin{pmatrix}
a_{1,1}\\
a_{1,2}\\
a_{1,3}\\
\end{pmatrix}}}

\begin{document}
\thispagestyle{empty}

\begin{align*}
\det\left(\mathbf{A}_{\mp}\right)
&
\begin{alignedat}[t]{6}
&=\hspaceone&&
\det
\begin{pmatrix}
a_{2,1} \mp a_{1,1}\\
a_{2,2} \mp a_{1,2}\\
a_{2,3} \mp a_{1,3}\\
\end{pmatrix}
&&=&&
\det
\begin{pmatrix}
a_{2,1}\\
a_{2,2} \mp a_{1,2}\\
a_{2,3} \mp a_{1,3}\\
\end{pmatrix}
&&\mp&&
\det
\begin{pmatrix}
a_{1,1}\\
a_{2,2} \mp a_{1,2}\\
a_{2,3} \mp a_{1,3}\\
\end{pmatrix}\\
&=\hspacetwo&&
\det
\begin{pmatrix}
a_{2,1}\\
a_{2,2}\\
a_{2,3} \mp a_{1,3}\\
\end{pmatrix}
&&\mp&&
\det
\begin{pmatrix}
a_{2,1}\\
a_{1,2}\\
a_{2,3} \mp a_{1,3}\\
\end{pmatrix}
&&\mp\left\{\vspaceone\right.&&
\det
\begin{pmatrix}
a_{1,1}\\
a_{2,2}\\
a_{2,3} \mp a_{1,3}\\
\end{pmatrix}\mp
\det
\begin{pmatrix}
a_{1,1}\\
a_{1,2}\\
a_{2,3} \mp a_{1,3}\\
\end{pmatrix}
\left.\vspacetwo\right\}
\end{alignedat}\\[0.1cm]
&
\begin{alignedat}[b]{8}
&=&&
\det
\begin{pmatrix}
a_{2,1}\\
a_{2,2}\\
a_{2,3}\\
\end{pmatrix}
&&\mp&&
\det\begin{pmatrix}
a_{2,1}\\
a_{2,2}\\
a_{1,3}\\
\end{pmatrix}
&&\mp\left[\vspacethree\right.&&
\det
\begin{pmatrix}
a_{2,1}\\
a_{1,2}\\
a_{2,3}\\
\end{pmatrix}
&&\mp&&
\det
\begin{pmatrix}
a_{2,1}\\
a_{1,2}\\
a_{1,3}\\
\end{pmatrix} \left.\vspacefour\right]\\
&\mp \left\{\vspacefive\right.&&
\det
\begin{pmatrix}
a_{1,1}\\
a_{2,2}\\
a_{2,3}\\
\end{pmatrix}
&&\mp&&
\det\begin{pmatrix}
a_{1,1}\\
a_{2,2}\\
a_{1,3}\\
\end{pmatrix}
&&\mp\left[\vspacesix\right.&&
\det
\begin{pmatrix}
a_{1,1}\\
a_{1,2}\\
a_{2,3}\\
\end{pmatrix}
&&\mp&&
\det\begin{pmatrix}
a_{1,1}\\
a_{1,2}\\
a_{1,3}\\
\end{pmatrix}
\left.\vspaceseven\right]
\left.\vspaceeight\right\}\\
&=&&
\det
\begin{pmatrix}
a_{2,1}\\
a_{2,2}\\
a_{2,3}\\
\end{pmatrix}
&&\mp&&
\det\begin{pmatrix}
a_{2,1}\\
a_{2,2}\\
a_{1,3}\\
\end{pmatrix}
&&\mp&&
\det
\begin{pmatrix}
a_{2,1}\\
a_{1,2}\\
a_{2,3}\\
\end{pmatrix}
&&+&&
\det
\begin{pmatrix}
a_{2,1}\\
a_{1,2}\\
a_{1,3}\\
\end{pmatrix}\\
&\mp&&
\det
\begin{pmatrix}
a_{1,1}\\
a_{2,2}\\
a_{2,3}\\
\end{pmatrix}
&&+&&
\det
\begin{pmatrix}
a_{1,1}\\
a_{2,2}\\
a_{1,3}\\
\end{pmatrix}
&&+&&
\det
\begin{pmatrix}
a_{1,1}\\
a_{1,2}\\
a_{2,3}\\
\end{pmatrix}
&&\mp&&
\det
\begin{pmatrix}
a_{1,1}\\
a_{1,2}\\
a_{1,3}\\
\end{pmatrix}
\end{alignedat}
\end{align*}
\end{document}

How can I correctly align the det terms?

I would use a single align* environment. Then, only after the math part is entered correctly, I would insert a few judiciously chosen \mskip directives to fine-tune the horizontal alignment. That said, I don't think it's really necessary to perform the fine-tuning.

\documentclass{article}
\usepackage{amsmath,mleftright}
\mleftright

\begin{document}
\begin{align*}
\det(\mathbf{A}_{\mp})
&= \mskip32mu
\det \begin{pmatrix}
a_{2,1} \mp a_{1,1}\\ a_{2,2} \mp a_{1,2}\\ a_{2,3} \mp a_{1,3}
\end{pmatrix} \\[1.5ex]
&= \mskip32mu
\det \begin{pmatrix}
a_{2,1}\\ a_{2,2} \mp a_{1,2}\\ a_{2,3} \mp a_{1,3}
\end{pmatrix}
\mp
\det \begin{pmatrix}
a_{1,1}\\ a_{2,2} \mp a_{1,2}\\ a_{2,3} \mp a_{1,3}
\end{pmatrix} \\[1.5ex]
&= \mskip32mu
\det \begin{pmatrix}
a_{2,1}\\ a_{2,2}\\ a_{2,3} \mp a_{1,3}
\end{pmatrix}
\mp
\det \begin{pmatrix}
a_{2,1}\\ a_{1,2}\\ a_{2,3} \mp a_{1,3}
\end{pmatrix} \\
\mp
\left\{
\det \begin{pmatrix}
a_{1,1}\\ a_{2,2}\\ a_{2,3} \mp a_{1,3}
\end{pmatrix}
\mp
\det \begin{pmatrix}
a_{1,1}\\ a_{1,2}\\ a_{2,3} \mp a_{1,3}
\end{pmatrix}
\right\} \\[1.5ex]
&= \mskip32mu
\det \begin{pmatrix}
a_{2,1}\\ a_{2,2}\\ a_{2,3}
\end{pmatrix}
\mp
\det \begin{pmatrix}
a_{2,1}\\ a_{2,2}\\ a_{1,3}
\end{pmatrix}
\mp
\left[
\det \begin{pmatrix}
a_{2,1}\\ a_{1,2}\\ a_{2,3}
\end{pmatrix}
\mp
\det \begin{pmatrix}
a_{2,1}\\ a_{1,2}\\ a_{1,3}
\end{pmatrix}
\right] \\
\left\{
\det \begin{pmatrix}
a_{1,1}\\ a_{2,2}\\ a_{2,3}
\end{pmatrix}
\mp
\det \begin{pmatrix}
a_{1,1}\\ a_{2,2}\\ a_{1,3}
\end{pmatrix}
\mp
\left[
\det \begin{pmatrix}
a_{1,1}\\ a_{1,2}\\ a_{2,3}
\end{pmatrix}
\mp
\det \begin{pmatrix}
a_{1,1}\\ a_{1,2}\\ a_{1,3}
\end{pmatrix}
\right]
\right\} \\[1.5ex]
&= \mskip16mu
\det \begin{pmatrix}
a_{2,1}\\ a_{2,2}\\ a_{2,3}
\end{pmatrix}
\mp
\det \begin{pmatrix}
a_{2,1}\\ a_{2,2}\\ a_{1,3}
\end{pmatrix}
\mp
\det \begin{pmatrix}
a_{2,1}\\ a_{1,2}\\ a_{2,3}
\end{pmatrix}
+
\det \begin{pmatrix}
a_{2,1}\\ a_{1,2}\\ a_{1,3}
\end{pmatrix} \\
\det \begin{pmatrix}
a_{1,1}\\ a_{2,2}\\ a_{2,3}
\end{pmatrix}
+
\det \begin{pmatrix}
a_{1,1}\\ a_{2,2}\\ a_{1,3}
\end{pmatrix}
+
\det \begin{pmatrix}
a_{1,1}\\ a_{1,2}\\ a_{2,3}
\end{pmatrix}
\mp
\det \begin{pmatrix}
a_{1,1}\\ a_{1,2}\\ a_{1,3}
\end{pmatrix}
\end{align*}
\end{document}
• Thanks, this looks really clean! – Skydiver Jan 28 '20 at 1:31