# algorithm2e align environment takes too much vertical space

I want to use equations in an algorithm. For this I use algorithm2e and the align* environment. The result looks nicely, but sadly, the align* takes too much vertical space and thus the whole algorithm becomes larger than the page itself. MWE:

\documentclass{article}
\usepackage[utf8]{inputenc}
\usepackage[boxruled,vlined,linesnumbered,algo2e, longend]{algorithm2e}
\usepackage{amsmath}

\title{stackquestion}
\date{June 2021}

\begin{document}

\maketitle

\section{Introduction}
\DontPrintSemicolon
\begin{algorithm2e}[htb]
\SetAlgoLined
\KwIn{Number of episodes $N$, discount factor $\gamma$}
\KwOut{Deterministic policy $\pi(\cdot |\theta^\pi)$, action-value function approximation $Q(\cdot, \cdot|\theta^{Q_1}),Q(\cdot, \cdot|\theta^{Q_2}) \approx q_*$}
Initialize replay memory $\mathcal{M}$.
\\ Initialize randomly neural network parameters $\theta^{Q_1}, \theta^{Q_2}$ for the critics and $\theta^\pi$ for the actor.
\\ Initialize target networks (for actor and critics): $\hat{\theta}^{Q_1} \leftarrow \theta^{Q_1},\hat{\theta}^{Q_2} \leftarrow \theta^{Q_2},\hat{\theta}^\pi\leftarrow \theta^\pi$
randomly.
\For{ episode$=1, \hdots, N$}{
Initialize starting state of episode $S_0$\;
\For{step $t$ in episode}{
Choose action $A_t = \pi(S_t|\theta^\pi)+\epsilon$ with $\epsilon \sim \mathcal{N}(0, \sigma)$\;
Take action $A_t$, observe $R_{t+1}, S_{t+1}$\;
Store transition $(S_t, A_t, R_{t+1}, S_{t+1})$ in $\mathcal{M}$ \;
%  $S \leftarrow S'$\;
Sample random minibatch of $m$ transitions ($S_j, A_j, R_j, S_{j+1})$ from $\mathcal{M}$\;
Set\; \begin{align*}
\tilde{A}_{j+1} &\leftarrow \pi(S_{j+1}|\hat{\theta}^\pi)+\epsilon, \quad \epsilon \sim \text{clip}(\mathcal{N}(0,\tilde{\sigma}), -c, c)
\\ y_j &\leftarrow R_{j+1}+\gamma \min_{i=1,2}Q(S_{j+1}, \tilde{A}_{j+1}|\hat{\theta}^{Q_i})
\end{align*}\;
Update critics \;
\begin{align*}
\theta^{Q_i} &\leftarrow \arg \min_{\theta^{Q_i}} \frac{1}{m}\sum_{j=1}^m(y_j-Q(S_j, A_j|\theta^{Q_i}))^2
\end{align*}\;
\If{$t$ mod $d$}{
Update $\theta^\pi$ by the deterministic policy gradient:\;
\begin{align*}
\nabla_{\theta^\pi}J(\theta^\pi) &= \frac{1}{m}\sum_{j=1}^m \nabla_a Q(S_j,a|\theta^{Q_1})|_{a=\pi(S_j|\theta^\pi)}\nabla_{\theta^\pi}\pi(S_j|\theta^\pi)
\end{align*}\;
Update target networks: \;
\begin{align*}
\hat{\theta}^{Q_i} & \leftarrow \tau \theta^{Q_i}+(1-\tau)\hat{\theta}^{Q_i} \quad i=1,2
\\ \hat{\theta}^\pi &\leftarrow \tau \theta^\pi + (1-\tau)\hat{\theta}^{\pi}
\end{align*}\;
}
Until episode ended or termination was enforced
}
}
\Return{$\pi \leftarrow \pi(\cdot |\theta^\pi)$}
\caption{Twin Delayed Deep deterministic policy gradient (T3DPG)}
\label{alg:T3DPG}
\end{algorithm2e}

\end{document}


The result is the picture below. I would like to remove the unnecessary vertical spacing which I highlighted by red. If possible, I would still like to use the align* environment, especially to align multiple equations easily.

• One should note that this float is too large for anything but [p]. Jun 27, 2021 at 14:56

I would replace the align* environments with\begin{aligned} … \end{aligned}.

Note some align* environments were unnecessary, so a simple $…$ would be enough for them.

Unrelated: loading inputenc with option utf8 is not useful nowadays, since it is what latext expects by default.

\documentclass{article}
\usepackage[boxruled,vlined,linesnumbered,algo2e, longend]{algorithm2e}
\usepackage{amsmath}

\title{stackquestion}
%\author{massimo.joerin }
\date{June 2021}

\begin{document}

\maketitle

\section{Introduction}
\DontPrintSemicolon
\begin{algorithm2e}[htb]
\SetAlgoLined
\KwIn{Number of episodes $N$, discount factor $\gamma$}
\KwOut{Deterministic policy $\pi(\cdot |\theta^\pi)$, action-value function approximation $Q(\cdot, \cdot|\theta^{Q_1}),Q(\cdot, \cdot|\theta^{Q_2}) \approx q_*$}
Initialize replay memory $\mathcal{M}$.
\\ Initialize randomly neural network parameters $\theta^{Q_1}, \theta^{Q_2}$ for the critics and $\theta^\pi$ for the actor.
\\ Initialize target networks (for actor and critics): $\hat{\theta}^{Q_1} \leftarrow \theta^{Q_1},\hat{\theta}^{Q_2} \leftarrow \theta^{Q_2},\hat{\theta}^\pi\leftarrow \theta^\pi$
randomly.
\For{ episode$=1, \hdots, N$}{
Initialize starting state of episode $S_0$\;
\For{step $t$ in episode}{
Choose action $A_t = \pi(S_t|\theta^\pi)+\epsilon$ with $\epsilon \sim \mathcal{N}(0, \sigma)$\;
Take action $A_t$, observe $R_{t+1}, S_{t+1}$\;
Store transition $(S_t, A_t, R_{t+1}, S_{t+1})$ in $\mathcal{M}$ \;
% $S \leftarrow S'$\;
Sample random minibatch of $m$ transitions ($S_j, A_j, R_j, S_{j+1})$ from $\mathcal{M}$\;
Set\;\begin{aligned} \tilde{A}_{j+1} &\leftarrow \pi(S_{j+1}|\hat{\theta}^\pi)+\epsilon, \quad \epsilon \sim \text{clip}(\mathcal{N}(0,\tilde{\sigma}), -c, c) \\ y_j &\leftarrow R_{j+1}+\gamma \min_{i=1,2}Q(S_{j+1}, \tilde{A}_{j+1}|\hat{\theta}^{Q_i}) \end{aligned}\;
Update critics \;
\begin{aligned} \theta^{Q_i} &\leftarrow \arg \min_{\theta^{Q_i}} \frac{1}{m}\sum_{j=1}^m(y_j-Q(S_j, A_j|\theta^{Q_i}))^2 \end{aligned}\;
\If{$t$ mod $d$}{
Update $\theta^\pi$ by the deterministic policy gradient:\;
\begin{aligned} \nabla_{\theta^\pi}J(\theta^\pi) &= \frac{1}{m}\sum_{j=1}^m \nabla_a Q(S_j,a|\theta^{Q_1})|_{a=\pi(S_j|\theta^\pi)}\nabla_{\theta^\pi}\pi(S_j|\theta^\pi) \end{aligned}\;
Update target networks: \;
\begin{aligned} \hat{\theta}^{Q_i} & \leftarrow \tau \theta^{Q_i}+(1-\tau)\hat{\theta}^{Q_i} \quad i=1,2 \\ \hat{\theta}^\pi &\leftarrow \tau \theta^\pi + (1-\tau)\hat{\theta}^{\pi} \end{aligned}\;
}
Until episode ended or termination was enforced
}
}
\Return{$\pi \leftarrow \pi(\cdot |\theta^\pi)$}
\caption{Twin Delayed Deep deterministic policy gradient (T3DPG)}
\label{alg:T3DPG}
\end{algorithm2e}

\end{document}


• This is a good idea. However, is it possible to mimic the horizontal behaviour of the align* environment with the aligned environment? It seems to me that the align* equations are more center and all the \leftarrow and = signs are aligned such that they are at the same horizontal position... is this possible with aligned to? What I want is basically align* without the vertical extra spacing. But, if this is not possible, your solution would be okay. Thanks!
– MJ05
Jun 27, 2021 at 19:01
• @MJ05: We can only align what's inside a given aligned environment. To have such an environment centred, the only way I found that works was to nest it between two \hfill commands (couldn't make the simpler \centering command work) Jun 27, 2021 at 19:43
• @Bernard - IIRC \centering doesn't work inside lists. Jun 28, 2021 at 15:55
• @JohnKormylo: Oh! yes. I simply forgot this kind of environment is based on list. Thanks for reminding me. Jun 28, 2021 at 16:07