1

I want to use equations in an algorithm. For this I use algorithm2e and the align* environment. The result looks nicely, but sadly, the align* takes too much vertical space and thus the whole algorithm becomes larger than the page itself. MWE:

\documentclass{article}
\usepackage[utf8]{inputenc}
\usepackage[boxruled,vlined,linesnumbered,algo2e, longend]{algorithm2e}
\usepackage{amsmath}

\title{stackquestion}
\date{June 2021}

\begin{document}

\maketitle

\section{Introduction}
\DontPrintSemicolon
\begin{algorithm2e}[htb]
  \SetAlgoLined
  \KwIn{Number of episodes $N$, discount factor $\gamma$}
  \KwOut{Deterministic policy $\pi(\cdot |\theta^\pi)$, action-value function approximation $Q(\cdot, \cdot|\theta^{Q_1}),Q(\cdot, \cdot|\theta^{Q_2}) \approx q_*$}
  Initialize replay memory $\mathcal{M}$.
  \\ Initialize randomly neural network parameters $\theta^{Q_1}, \theta^{Q_2}$ for the critics and $\theta^\pi$ for the actor.
  \\ Initialize target networks (for actor and critics): $\hat{\theta}^{Q_1} \leftarrow \theta^{Q_1},\hat{\theta}^{Q_2} \leftarrow \theta^{Q_2},\hat{\theta}^\pi\leftarrow \theta^\pi$
 randomly.
  \For{ episode$=1, \hdots, N$}{
  Initialize starting state of episode $S_0$\;
  \For{step $t$ in episode}{
   Choose action $A_t = \pi(S_t|\theta^\pi)+\epsilon$ with $\epsilon \sim \mathcal{N}(0, \sigma)$\;
   Take action $A_t$, observe $R_{t+1}, S_{t+1}$\;
  Store transition $(S_t, A_t, R_{t+1}, S_{t+1})$ in $\mathcal{M}$ \;
%  $S \leftarrow S'$\;
  Sample random minibatch of $m$ transitions ($S_j, A_j, R_j, S_{j+1})$ from $\mathcal{M}$\;
   Set\; \begin{align*}
   \tilde{A}_{j+1} &\leftarrow \pi(S_{j+1}|\hat{\theta}^\pi)+\epsilon, \quad \epsilon \sim \text{clip}(\mathcal{N}(0,\tilde{\sigma}), -c, c)
   \\ y_j &\leftarrow R_{j+1}+\gamma \min_{i=1,2}Q(S_{j+1}, \tilde{A}_{j+1}|\hat{\theta}^{Q_i})
   \end{align*}\;
   Update critics \;
   \begin{align*}
   \theta^{Q_i} &\leftarrow \arg \min_{\theta^{Q_i}} \frac{1}{m}\sum_{j=1}^m(y_j-Q(S_j, A_j|\theta^{Q_i}))^2
   \end{align*}\;
   \If{$t$ mod $d$}{
        Update $\theta^\pi$ by the deterministic policy gradient:\;
        \begin{align*}
        \nabla_{\theta^\pi}J(\theta^\pi) &= \frac{1}{m}\sum_{j=1}^m \nabla_a Q(S_j,a|\theta^{Q_1})|_{a=\pi(S_j|\theta^\pi)}\nabla_{\theta^\pi}\pi(S_j|\theta^\pi)
        \end{align*}\;
        Update target networks: \;
        \begin{align*}
        \hat{\theta}^{Q_i} & \leftarrow \tau \theta^{Q_i}+(1-\tau)\hat{\theta}^{Q_i} \quad i=1,2
        \\ \hat{\theta}^\pi &\leftarrow \tau \theta^\pi + (1-\tau)\hat{\theta}^{\pi}
        \end{align*}\;
   }
  Until episode ended or termination was enforced
}
}
  \Return{$\pi \leftarrow \pi(\cdot |\theta^\pi)$}
  \caption{Twin Delayed Deep deterministic policy gradient (T3DPG)}
  \label{alg:T3DPG}
\end{algorithm2e}


\end{document}

The result is the picture below. I would like to remove the unnecessary vertical spacing which I highlighted by red. If possible, I would still like to use the align* environment, especially to align multiple equations easily.

enter image description here

1
  • One should note that this float is too large for anything but [p]. Jun 27, 2021 at 14:56

1 Answer 1

2

I would replace the align* environments with$\begin{aligned} … \end{aligned}$.

Note some align* environments were unnecessary, so a simple $ … $ would be enough for them.

Unrelated: loading inputenc with option utf8 is not useful nowadays, since it is what latext expects by default.

\documentclass{article}
\usepackage[boxruled,vlined,linesnumbered,algo2e, longend]{algorithm2e}
\usepackage{amsmath}

\title{stackquestion}
%\author{massimo.joerin }
\date{June 2021}

\begin{document}

\maketitle

\section{Introduction}
\DontPrintSemicolon
\begin{algorithm2e}[htb]
  \SetAlgoLined
  \KwIn{Number of episodes $N$, discount factor $\gamma$}
  \KwOut{Deterministic policy $\pi(\cdot |\theta^\pi)$, action-value function approximation $Q(\cdot, \cdot|\theta^{Q_1}),Q(\cdot, \cdot|\theta^{Q_2}) \approx q_*$}
  Initialize replay memory $\mathcal{M}$.
  \\ Initialize randomly neural network parameters $\theta^{Q_1}, \theta^{Q_2}$ for the critics and $\theta^\pi$ for the actor.
  \\ Initialize target networks (for actor and critics): $\hat{\theta}^{Q_1} \leftarrow \theta^{Q_1},\hat{\theta}^{Q_2} \leftarrow \theta^{Q_2},\hat{\theta}^\pi\leftarrow \theta^\pi$
 randomly.
  \For{ episode$=1, \hdots, N$}{
  Initialize starting state of episode $S_0$\;
  \For{step $t$ in episode}{
   Choose action $A_t = \pi(S_t|\theta^\pi)+\epsilon$ with $\epsilon \sim \mathcal{N}(0, \sigma)$\;
   Take action $A_t$, observe $R_{t+1}, S_{t+1}$\;
  Store transition $(S_t, A_t, R_{t+1}, S_{t+1})$ in $\mathcal{M}$ \;
% $S \leftarrow S'$\;
  Sample random minibatch of $m$ transitions ($S_j, A_j, R_j, S_{j+1})$ from $\mathcal{M}$\;
   Set\;$ \begin{aligned}
   \tilde{A}_{j+1} &\leftarrow \pi(S_{j+1}|\hat{\theta}^\pi)+\epsilon, \quad \epsilon \sim \text{clip}(\mathcal{N}(0,\tilde{\sigma}), -c, c)
   \\ y_j &\leftarrow R_{j+1}+\gamma \min_{i=1,2}Q(S_{j+1}, \tilde{A}_{j+1}|\hat{\theta}^{Q_i})
   \end{aligned}$\;
   Update critics \;
  $ \begin{aligned}
   \theta^{Q_i} &\leftarrow \arg \min_{\theta^{Q_i}} \frac{1}{m}\sum_{j=1}^m(y_j-Q(S_j, A_j|\theta^{Q_i}))^2
   \end{aligned} $\;
   \If{$t$ mod $d$}{
        Update $\theta^\pi$ by the deterministic policy gradient:\;
        $ \begin{aligned}
        \nabla_{\theta^\pi}J(\theta^\pi) &= \frac{1}{m}\sum_{j=1}^m \nabla_a Q(S_j,a|\theta^{Q_1})|_{a=\pi(S_j|\theta^\pi)}\nabla_{\theta^\pi}\pi(S_j|\theta^\pi)
        \end{aligned} $\;
        Update target networks: \;
        $ \begin{aligned}
        \hat{\theta}^{Q_i} & \leftarrow \tau \theta^{Q_i}+(1-\tau)\hat{\theta}^{Q_i} \quad i=1,2
        \\ \hat{\theta}^\pi &\leftarrow \tau \theta^\pi + (1-\tau)\hat{\theta}^{\pi}
        \end{aligned} $\;
   }
  Until episode ended or termination was enforced
}
}
  \Return{$\pi \leftarrow \pi(\cdot |\theta^\pi)$}
  \caption{Twin Delayed Deep deterministic policy gradient (T3DPG)}
  \label{alg:T3DPG}
\end{algorithm2e}

\end{document} 

enter image description here

4
  • This is a good idea. However, is it possible to mimic the horizontal behaviour of the align* environment with the aligned environment? It seems to me that the align* equations are more center and all the \leftarrow and = signs are aligned such that they are at the same horizontal position... is this possible with aligned to? What I want is basically align* without the vertical extra spacing. But, if this is not possible, your solution would be okay. Thanks!
    – MJ05
    Jun 27, 2021 at 19:01
  • @MJ05: We can only align what's inside a given aligned environment. To have such an environment centred, the only way I found that works was to nest it between two \hfill commands (couldn't make the simpler \centering command work)
    – Bernard
    Jun 27, 2021 at 19:43
  • @Bernard - IIRC \centering doesn't work inside lists. Jun 28, 2021 at 15:55
  • @JohnKormylo: Oh! yes. I simply forgot this kind of environment is based on list. Thanks for reminding me.
    – Bernard
    Jun 28, 2021 at 16:07

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.