# How to Draw Recurrent Neural Network

I googled that tikz is a package for drawing graph like a neural network. I really like to learn how to use it. But I wonder how to draw a recurrent neural network?

• Welcome, just out of interest, do you want the whole thing drawn for you? Sep 9 '16 at 8:13
• Some people take exception to questions of the form "Please draw this for me". You will get more help if you post some code showing what you have tried and give a minimal working example.
– user30471
Sep 9 '16 at 8:21
• @Johannes_B Not exactly same like that. Just wanted to know how to draw the 3 state right in the middle one (the middle one, it's left, and it's right). Sep 9 '16 at 8:23
• @Andrew thanks for your comment, I'll edit my question and trying to add my code asap Sep 9 '16 at 8:25

This should help you get started. I created styles for each type of neuron, so you can easily change up the styling and spacing as needed without needing to touch a lot of code. These style definitions are exhaustively commented so you can see what each line does.

\documentclass[border=5pt]{standalone}
\renewcommand{\vec}[1]{\mathbf{#1}} % re-style the vector
\usepackage{tikz}
\usetikzlibrary{
arrows.meta, % for Straight Barb arrow tip
fit, % to fit the group box around the central neurons
positioning, % for relative positioning of the neurons
}

\tikzset{
neuron/.style={ % style for each neuron
circle,draw,thick, % drawn as a thick circle
inner sep=0pt, % no built-in padding between the text and the circle shape
minimum size=3.5em, % make each neuron the same size regardless of the text inside
node distance=1ex and 2em, % spacing between neurons (y and x)
},
group/.style={ % style for the groups of neurons
rectangle,draw,thick, % drawn as a thick rectangle
inner sep=0pt, % no padding between the node contents and the rectangle shape
},
io/.style={ % style for the inputs/outputs
neuron, % inherit the neuron style
fill=gray!15, % add a fill color
},
conn/.style={ % style for the connections
-{Straight Barb[angle=60:2pt 3]}, % simple barbed arrow tip
thick, % draw in a thick weight to match other drawing elements
},
}

\begin{document}
\begin{tikzpicture}
\node[neuron] (ht1) {$h_t^1$};
\node[neuron,below=of ht1] (htm) {$h_t^{n-m}$};
\node[neuron,below=of htm] (htn) {$h_t^n$};
\node[group,fit={(ht1) (htm) (htn)}] (gr1) {};
\node[neuron,left=of ht1] (ht-1) {$h_{t-1(2)}^1$};
\node[io,above=2em of ht1] (yt) {$y_t$};
\node[io,below=2em of htn] (vt) {$\vec{v}_t$};
\draw[conn] (ht1) -- (yt);
\draw[conn] (vt) -- (htn);
\foreach \destination in {ht1,htm,htn} { % the for loop idea can be expanded to draw the entire diagram quickly
\draw[conn] (ht-1.east) -- (\destination.west);
}
\end{tikzpicture}
\end{document}


And here's the result:

OK, that was a lot of stuff! Let's break it down.

Each neuron is drawn as a node with a name in parentheses and the node text in curly braces, for example:

\node[neuron] (ht1)   {$h_t^1$};
%     ^-style  ^-name  ^-node text


Once that first node is placed, all others can be positioned relative to it, with no need for calculating coordinates manually, using the positioning library:

\node[neuron,below=of ht1] (htm) {$h_t^{n-m}$};
%            ^-this node should be placed below the first node (ht1)


The spacing is controlled by the value of the node distance key, which I set in the definition of the neuron style.

The group box around the central neurons is created using the fit library to size a node to fit around a list of existing nodes, like so:

\node[group,fit={(ht1) (htm) (htn)}] (gr1) {};
%     ^-style    ^- list of nodes to fit    ^-empty node text for this one


This node (gr1) has an empty text, because we just want the node to box the list of neurons, we don't need any new text inside it.

Once all the neurons are drawn, we can connect them with \draw statements, which create stroked paths between two coordinates (here, named coordinates) in parentheses. One example:

\draw[conn] (ht1) -- (yt);
%     ^      ^-start  ^-finish
%     -apply connection style


Drawing all of these individually gets tedious quickly. Fortunately, TikZ offers the concept of for loops, which I used to create the other arrows:

\foreach \destination in {ht1,htm,htn} {
\draw[conn] (ht-1.east) -- (\destination.west);
}


I added .east and .west as anchors on the nodes we're drawing to and from. This causes the connection to be drawn to that specific point on the node edge. If you leave those off, the arrow will be drawn in a straight line from node center to node center (notice how I left the anchors off in my first example).

The \foreach loop idea can be extended to draw the many neurons in your full diagram efficiently. You can check out TeXample: Neural Network for a nice example of this.

TikZ has excellent documentation, but it might look overwhelming at first glance. If you don't see why I did something, search for the macro name or key name in the manual to see how it works and what it does. You can access the manual on your PC by typing texdoc tikz at a terminal or command prompt. Or, you can access it online at http://texdoc.net/pkg/tikz (warning: it is a large PDF file). Hope this helps you get started! :-)

Doesn't really need much code:

\documentclass[tikz,border=5]{standalone}
\usetikzlibrary{fit,positioning,arrows.meta}
\tikzset{neuron/.style={shape=circle, minimum size=1.25cm,
inner sep=0, draw, font=\small}, io/.style={neuron, fill=gray!20}}
\begin{document}
\begin{tikzpicture}[x=2cm, y=1.5cm, >=Stealth]
\foreach \jlabel [count=\j, evaluate={\k=int(mod(\j-1,3)); \jj=int(\j-1);}]
in {t-1, t-1(1), t-1(2), t, t(1), t(2), t+1}{
\foreach \ilabel [count=\i] in {1, n-m, n}
\node [neuron] at (\j, 1-\i) (h-\i-\j){$h_{\jlabel}^{\ilabel}$};
\ifcase\k
\node [fit=(h-1-\j) (h-3-\j), inner sep=0, draw] (b-\j) {};
\node [io, above=of b-\j] (y-\j) {$y_{\jlabel}$};
\node [io, below=of b-\j] (v-\j) {$v_{\jlabel}$};
\draw [->] (v-\j) -- (b-\j);
\draw [->] (b-\j) -- (y-\j);
\fi
\ifnum\j>1
\foreach\i in {1, 2, 3}
\foreach \ii in {1, 2, 3}
\draw [->] (h-\i-\jj.east) -- (h-\ii-\j.west);
\fi
}
\node [left=of h-2-1] {\ldots};
\node [right=of h-2-7] {\ldots};
\end{tikzpicture}
\end{document}
`