1

I want to use equations in an algorithm. For this I use algorithm2e and the align* environment. The result looks nicely, but sadly, the align* takes too much vertical space and thus the whole algorithm becomes larger than the page itself. MWE:

\documentclass{article}
\usepackage[utf8]{inputenc}
\usepackage[boxruled,vlined,linesnumbered,algo2e, longend]{algorithm2e}
\usepackage{amsmath}

\title{stackquestion}
\date{June 2021}

\begin{document}

\maketitle

\section{Introduction}
\DontPrintSemicolon
\begin{algorithm2e}[htb]
  \SetAlgoLined
  \KwIn{Number of episodes $N$, discount factor $\gamma$}
  \KwOut{Deterministic policy $\pi(\cdot |\theta^\pi)$, action-value function approximation $Q(\cdot, \cdot|\theta^{Q_1}),Q(\cdot, \cdot|\theta^{Q_2}) \approx q_*$}
  Initialize replay memory $\mathcal{M}$.
  \\ Initialize randomly neural network parameters $\theta^{Q_1}, \theta^{Q_2}$ for the critics and $\theta^\pi$ for the actor.
  \\ Initialize target networks (for actor and critics): $\hat{\theta}^{Q_1} \leftarrow \theta^{Q_1},\hat{\theta}^{Q_2} \leftarrow \theta^{Q_2},\hat{\theta}^\pi\leftarrow \theta^\pi$
 randomly.
  \For{ episode$=1, \hdots, N$}{
  Initialize starting state of episode $S_0$\;
  \For{step $t$ in episode}{
   Choose action $A_t = \pi(S_t|\theta^\pi)+\epsilon$ with $\epsilon \sim \mathcal{N}(0, \sigma)$\;
   Take action $A_t$, observe $R_{t+1}, S_{t+1}$\;
  Store transition $(S_t, A_t, R_{t+1}, S_{t+1})$ in $\mathcal{M}$ \;
%  $S \leftarrow S'$\;
  Sample random minibatch of $m$ transitions ($S_j, A_j, R_j, S_{j+1})$ from $\mathcal{M}$\;
   Set\; \begin{align*}
   \tilde{A}_{j+1} &\leftarrow \pi(S_{j+1}|\hat{\theta}^\pi)+\epsilon, \quad \epsilon \sim \text{clip}(\mathcal{N}(0,\tilde{\sigma}), -c, c)
   \\ y_j &\leftarrow R_{j+1}+\gamma \min_{i=1,2}Q(S_{j+1}, \tilde{A}_{j+1}|\hat{\theta}^{Q_i})
   \end{align*}\;
   Update critics \;
   \begin{align*}
   \theta^{Q_i} &\leftarrow \arg \min_{\theta^{Q_i}} \frac{1}{m}\sum_{j=1}^m(y_j-Q(S_j, A_j|\theta^{Q_i}))^2
   \end{align*}\;
   \If{$t$ mod $d$}{
        Update $\theta^\pi$ by the deterministic policy gradient:\;
        \begin{align*}
        \nabla_{\theta^\pi}J(\theta^\pi) &= \frac{1}{m}\sum_{j=1}^m \nabla_a Q(S_j,a|\theta^{Q_1})|_{a=\pi(S_j|\theta^\pi)}\nabla_{\theta^\pi}\pi(S_j|\theta^\pi)
        \end{align*}\;
        Update target networks: \;
        \begin{align*}
        \hat{\theta}^{Q_i} & \leftarrow \tau \theta^{Q_i}+(1-\tau)\hat{\theta}^{Q_i} \quad i=1,2
        \\ \hat{\theta}^\pi &\leftarrow \tau \theta^\pi + (1-\tau)\hat{\theta}^{\pi}
        \end{align*}\;
   }
  Until episode ended or termination was enforced
}
}
  \Return{$\pi \leftarrow \pi(\cdot |\theta^\pi)$}
  \caption{Twin Delayed Deep deterministic policy gradient (T3DPG)}
  \label{alg:T3DPG}
\end{algorithm2e}


\end{document}

The result is the picture below. I would like to remove the unnecessary vertical spacing which I highlighted by red. If possible, I would still like to use the align* environment, especially to align multiple equations easily.

enter image description here

1
  • One should note that this float is too large for anything but [p]. Jun 27, 2021 at 14:56

1 Answer 1

2

I would replace the align* environments with$\begin{aligned} … \end{aligned}$.

Note some align* environments were unnecessary, so a simple $ … $ would be enough for them.

Unrelated: loading inputenc with option utf8 is not useful nowadays, since it is what latext expects by default.

\documentclass{article}
\usepackage[boxruled,vlined,linesnumbered,algo2e, longend]{algorithm2e}
\usepackage{amsmath}

\title{stackquestion}
%\author{massimo.joerin }
\date{June 2021}

\begin{document}

\maketitle

\section{Introduction}
\DontPrintSemicolon
\begin{algorithm2e}[htb]
  \SetAlgoLined
  \KwIn{Number of episodes $N$, discount factor $\gamma$}
  \KwOut{Deterministic policy $\pi(\cdot |\theta^\pi)$, action-value function approximation $Q(\cdot, \cdot|\theta^{Q_1}),Q(\cdot, \cdot|\theta^{Q_2}) \approx q_*$}
  Initialize replay memory $\mathcal{M}$.
  \\ Initialize randomly neural network parameters $\theta^{Q_1}, \theta^{Q_2}$ for the critics and $\theta^\pi$ for the actor.
  \\ Initialize target networks (for actor and critics): $\hat{\theta}^{Q_1} \leftarrow \theta^{Q_1},\hat{\theta}^{Q_2} \leftarrow \theta^{Q_2},\hat{\theta}^\pi\leftarrow \theta^\pi$
 randomly.
  \For{ episode$=1, \hdots, N$}{
  Initialize starting state of episode $S_0$\;
  \For{step $t$ in episode}{
   Choose action $A_t = \pi(S_t|\theta^\pi)+\epsilon$ with $\epsilon \sim \mathcal{N}(0, \sigma)$\;
   Take action $A_t$, observe $R_{t+1}, S_{t+1}$\;
  Store transition $(S_t, A_t, R_{t+1}, S_{t+1})$ in $\mathcal{M}$ \;
% $S \leftarrow S'$\;
  Sample random minibatch of $m$ transitions ($S_j, A_j, R_j, S_{j+1})$ from $\mathcal{M}$\;
   Set\;$ \begin{aligned}
   \tilde{A}_{j+1} &\leftarrow \pi(S_{j+1}|\hat{\theta}^\pi)+\epsilon, \quad \epsilon \sim \text{clip}(\mathcal{N}(0,\tilde{\sigma}), -c, c)
   \\ y_j &\leftarrow R_{j+1}+\gamma \min_{i=1,2}Q(S_{j+1}, \tilde{A}_{j+1}|\hat{\theta}^{Q_i})
   \end{aligned}$\;
   Update critics \;
  $ \begin{aligned}
   \theta^{Q_i} &\leftarrow \arg \min_{\theta^{Q_i}} \frac{1}{m}\sum_{j=1}^m(y_j-Q(S_j, A_j|\theta^{Q_i}))^2
   \end{aligned} $\;
   \If{$t$ mod $d$}{
        Update $\theta^\pi$ by the deterministic policy gradient:\;
        $ \begin{aligned}
        \nabla_{\theta^\pi}J(\theta^\pi) &= \frac{1}{m}\sum_{j=1}^m \nabla_a Q(S_j,a|\theta^{Q_1})|_{a=\pi(S_j|\theta^\pi)}\nabla_{\theta^\pi}\pi(S_j|\theta^\pi)
        \end{aligned} $\;
        Update target networks: \;
        $ \begin{aligned}
        \hat{\theta}^{Q_i} & \leftarrow \tau \theta^{Q_i}+(1-\tau)\hat{\theta}^{Q_i} \quad i=1,2
        \\ \hat{\theta}^\pi &\leftarrow \tau \theta^\pi + (1-\tau)\hat{\theta}^{\pi}
        \end{aligned} $\;
   }
  Until episode ended or termination was enforced
}
}
  \Return{$\pi \leftarrow \pi(\cdot |\theta^\pi)$}
  \caption{Twin Delayed Deep deterministic policy gradient (T3DPG)}
  \label{alg:T3DPG}
\end{algorithm2e}

\end{document} 

enter image description here

4
  • This is a good idea. However, is it possible to mimic the horizontal behaviour of the align* environment with the aligned environment? It seems to me that the align* equations are more center and all the \leftarrow and = signs are aligned such that they are at the same horizontal position... is this possible with aligned to? What I want is basically align* without the vertical extra spacing. But, if this is not possible, your solution would be okay. Thanks!
    – MJ05
    Jun 27, 2021 at 19:01
  • @MJ05: We can only align what's inside a given aligned environment. To have such an environment centred, the only way I found that works was to nest it between two \hfill commands (couldn't make the simpler \centering command work)
    – Bernard
    Jun 27, 2021 at 19:43
  • @Bernard - IIRC \centering doesn't work inside lists. Jun 28, 2021 at 15:55
  • @JohnKormylo: Oh! yes. I simply forgot this kind of environment is based on list. Thanks for reminding me.
    – Bernard
    Jun 28, 2021 at 16:07

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .