# Drawing an unfolded recurrent neural network

I am trying to create an "unfolded" version of this diagram made with tikz:

This was made with the code:

\documentclass[a4paper, 11pt]{article}

\usepackage{float}
\usepackage{tikz}

\usetikzlibrary{matrix,chains,positioning,decorations.pathreplacing,arrows}
\usetikzlibrary{decorations.text}
\usetikzlibrary{decorations.pathmorphing}
\usetikzlibrary{fit, arrows.meta}

\usepackage{pgfplots}
\pgfplotsset{compat=1.14}

\usepackage[]{algorithm2e}

\graphicspath{ {images/} }

\tikzset{%
every neuron/.style={
circle,
draw,
minimum size=1cm
},
neuron missing/.style={
draw=none,
scale=4,
text height=0.333cm,
execute at begin node=\color{black}$\vdots$
},
}

\def\layersep{2.5cm}

\begin{document}

\begin{tikzpicture}[x=1.5cm, y=1.5cm, >=stealth]

\foreach \m/\l [count=\y] in {1,2,3,4}
\node [every neuron/.try, neuron \m/.try] (input-\m) at (0,2.5-\y) {};

\foreach \m [count=\y] in {1,2}
\node [every neuron/.try, neuron \m/.try ] (hidden-\m) at (2,2-\y*1.25) {};

\foreach \m [count=\y] in {1,2}
\node [every neuron/.try, neuron \m/.try ] (output-\m) at (4,2-\y*1.25) {};

\foreach \l [count=\i] in {1,2,3,n}
\draw [<-] (input-\i) -- ++(-1,0)
node [above, midway] {};

\foreach \l [count=\i] in {1,n}
\node [above] at (hidden-\i.north) {};

\foreach \l [count=\i] in {1,n}
\draw [->] (output-\i) -- ++(1,0)
node [above, midway] {};

\foreach \i in {1,...,4}
\foreach \j in {1,...,2}
\draw [->] (input-\i) -- (hidden-\j);

\foreach \i in {1,...,2}
\foreach \j in {1,...,2}
\draw [->] (hidden-\i) -- (output-\j);

\draw[->,shorten >=1pt] (hidden-1) to [out=90,in=180,loop,looseness=8.8] (hidden-1);
\draw[->,shorten >=1pt] (hidden-2) to [out=-90,in=180,loop,looseness=8.8] (hidden-2);

\draw[->,shorten >=1pt] (hidden-1) to [out=180,in=180,loop,looseness=2.8] (hidden-2);
\draw[->,shorten >=1pt] (hidden-2) to [out=180,in=180,loop,looseness=1.8] (hidden-1);

\foreach \l [count=\x from 0] in {Input, Hidden, Ouput}
\node [align=center, above] at (\x*2,2) {\l \\ layer};

\end{tikzpicture}

\end{document}


Now I am trying to do something like this but can't really figure out how to modify my code so that I will have Three networks next to each other:

I looked at the question in the following link, but it is not quite the same as what I am trying to do: How to Draw Recurrent Neural Network

So can my code be modified for me to do this? Do you Think it would be possible to rotate the networks -90 degrees so that they fit an A4 page better?

• That code alone doesn't produce the image you show, the circles are missing. I guess the neuron stuff is responsible for that. Can you turn your snippet into a complete example? – Torbjørn T. Apr 12 '17 at 12:57
• @TorbjørnT. thank you for telling me that, I have now updated the code in the question so that it should be complete. – Stephen Johnson Apr 12 '17 at 13:12

One possible method is to add another \foreach loop, that repeats everything you have done three times. Note that the nodes are positioned at e.g. (0+\X,2.5-\y), where \X is the loop variable for the new, outer loop. There are some adjustments to node names, to accommodate three instances of the same.

Some adjustments of the newly added arrows (final \foreach loop) might be in order.

To rotate it you could just swap x and y coordinates everywhere. Below, I instead swapped the unit vectors.

I commented out some unused, and invisible, nodes.

\documentclass[a4paper,11pt]{article}
\usepackage{tikz}
\tikzset{%
every neuron/.style={
circle,
draw,
minimum size=1cm
},
neuron missing/.style={
draw=none,
scale=4,
text height=0.333cm,
execute at begin node=\color{black}$\vdots$
},
}

\begin{document}
\begin{center}
\begin{tikzpicture}[
% swap the unit vectors
x={(0,-1cm)},
y={(1.5cm,0)},
>=stealth,
]
\foreach [count=\xx] \X in {0,6.5,13}{
\foreach \m/\l [count=\y] in {1,2,3,4}
\node [every neuron/.try, neuron \m/.try] (input-\m-\xx) at (0+\X,2.5-\y) {};

\foreach \m [count=\y] in {1,2}
\node [every neuron/.try, neuron \m/.try ] (hidden-\m-\xx) at (2+\X,2-\y*1.25) {};

\foreach \m [count=\y] in {1,2}
\node [every neuron/.try, neuron \m/.try ] (output-\m-\xx) at (4+\X,2-\y*1.25) {};

\foreach \l [count=\i] in {1,2,3,n}
\draw [<-] (input-\i-\xx) -- ++(-1,0);
%    node [above, midway] {};

%\foreach \l [count=\i] in {1,n}
%  \node [above] at (hidden-\i-\xx.north) {};

\foreach \l [count=\i] in {1,n}
\draw [->] (output-\i-\xx) -- ++(1,0);
%    node [above, midway] {};

\foreach \i in {1,...,4}
\foreach \j in {1,...,2}
\draw [->] (input-\i-\xx) -- (hidden-\j-\xx);

\foreach \i in {1,...,2}
\foreach \j in {1,...,2}
\draw [->] (hidden-\i-\xx) -- (output-\j-\xx);

\draw[->,shorten >=1pt] (hidden-1-\xx) to [out=0,in=90,loop,looseness=8.8] (hidden-1-\xx);
\draw[->,shorten >=1pt] (hidden-2-\xx) to [out=180,in=90,loop,looseness=8.8] (hidden-2-\xx);

\draw[->,shorten >=1pt] (hidden-1-\xx) to [out=90,in=90,loop,looseness=1.7] (hidden-2-\xx);
\draw[->,shorten >=1pt] (hidden-2-\xx) to [out=90,in=90,loop,looseness=1] (hidden-1-\xx);

\foreach \l/\txt [count=\x from 0] in {Input/input, Hidden/hidden, Output/output}
\node [align=center] at (\txt-1-\xx-|+0,3) {\l \\ layer};
}% end of outer foreach

\foreach [evaluate=\i as \j using int(\i+1)] \i in {1,2}
{
\draw [red] (hidden-1-\i) to[bend left] (hidden-1-\j);
\draw [red] (hidden-1-\i) to[bend left] (hidden-2-\j);
\draw [red] (hidden-2-\i) to[bend right] (hidden-1-\j);
\draw [red] (hidden-2-\i) to[bend right] (hidden-2-\j);

}
\end{tikzpicture}
\end{center}
\end{document}


# Next to each other

This is still wider than the textblock though.

\documentclass[a4paper,11pt]{article}
\usepackage{tikz}
\tikzset{%
every neuron/.style={
circle,
draw,
minimum size=7mm
},
neuron missing/.style={
draw=none,
scale=4,
text height=0.333cm,
execute at begin node=\color{black}$\vdots$
},
}

\begin{document}
\begin{center}
\begin{tikzpicture}[
% swap the unit vectors
x={(0,-1cm)},
y={(1.1cm,0)},
>=stealth,
]
\foreach [count=\xx] \X in {0,4,8}{
\foreach \m/\l [count=\y] in {1,2,3,4}
\node [every neuron/.try, neuron \m/.try] (input-\m-\xx) at (0,2.5-\y+\X) {};

\foreach \m [count=\y] in {1,2}
\node [every neuron/.try, neuron \m/.try ] (hidden-\m-\xx) at (2,2-\y*1.25+\X) {};

\foreach \m [count=\y] in {1,2}
\node [every neuron/.try, neuron \m/.try ] (output-\m-\xx) at (4,2-\y*1.25+\X) {};

\foreach \l [count=\i] in {1,2,3,n}
\draw [<-] (input-\i-\xx) -- ++(-1,0);
%    node [above, midway] {};

%\foreach \l [count=\i] in {1,n}
%  \node [above] at (hidden-\i-\xx.north) {};

\foreach \l [count=\i] in {1,n}
\draw [->] (output-\i-\xx) -- ++(1,0);
%    node [above, midway] {};

\foreach \i in {1,...,4}
\foreach \j in {1,...,2}
\draw [->] (input-\i-\xx) -- (hidden-\j-\xx);

\foreach \i in {1,...,2}
\foreach \j in {1,...,2}
\draw [->] (hidden-\i-\xx) -- (output-\j-\xx);

\draw[->,shorten >=1pt] (hidden-1-\xx) to [out=0,in=90,loop,looseness=8.8] (hidden-1-\xx);
\draw[->,shorten >=1pt] (hidden-2-\xx) to [out=180,in=90,loop,looseness=8.8] (hidden-2-\xx);

\draw[->,shorten >=1pt] (hidden-1-\xx) to [out=90,in=90,loop,looseness=1.7] (hidden-2-\xx);
\draw[->,shorten >=1pt] (hidden-2-\xx) to [out=90,in=90,loop,looseness=1] (hidden-1-\xx);

}% end of outer foreach

\foreach \l/\txt [count=\x from 0] in {Input/input, Hidden/hidden, Output/output}
\node [align=center] at (\txt-1-\xx-|0,-2.5) {\l \\ layer};

\foreach [evaluate=\i as \j using int(\i+1)] \i in {1,2}
{
\draw [red,->] (hidden-1-\i) to[bend left] (hidden-1-\j);
\draw [red,->] (hidden-1-\i) to[bend left] (hidden-2-\j);
\draw [red,->] (hidden-2-\i) to[bend right] (hidden-1-\j);
\draw [red,->] (hidden-2-\i) to[bend right] (hidden-2-\j);

}
\end{tikzpicture}
\end{center}
\end{document}

• Wait, I messed up some of those arrows. You can probably fix it yourself, but I'll edit in a bit. – Torbjørn T. Apr 12 '17 at 13:21
• I can't Believe you figured that out so fast, thanks alot! I will study your example. Using the foreach like you did, would it possible to align them next to each other (still rotated separately)? – Stephen Johnson Apr 12 '17 at 13:23
• @StephenJohnson Probably just add +\X to the y-coordinate instead of the x-coordinate, will edit in a bit. – Torbjørn T. Apr 12 '17 at 13:26
• @StephenJohnson See edited answer. – Torbjørn T. Apr 12 '17 at 13:31
• It works perfectly fine! I am very thankful. The textblock width was solved by difining Another style for the neuron, which I called small neuron, with only minimum size of 0.5 cm. Thank you alot again! – Stephen Johnson Apr 12 '17 at 13:41