I googled that tikz is a package for drawing graph like a neural network. I really like to learn how to use it. But I wonder how to draw a recurrent neural network?


  • 2
    Welcome, just out of interest, do you want the whole thing drawn for you?
    – Johannes_B
    Sep 9 '16 at 8:13
  • 1
    Some people take exception to questions of the form "Please draw this for me". You will get more help if you post some code showing what you have tried and give a minimal working example.
    – user30471
    Sep 9 '16 at 8:21
  • @Johannes_B Not exactly same like that. Just wanted to know how to draw the 3 state right in the middle one (the middle one, it's left, and it's right).
    – muhrifqii
    Sep 9 '16 at 8:23
  • @Andrew thanks for your comment, I'll edit my question and trying to add my code asap
    – muhrifqii
    Sep 9 '16 at 8:25

This should help you get started. I created styles for each type of neuron, so you can easily change up the styling and spacing as needed without needing to touch a lot of code. These style definitions are exhaustively commented so you can see what each line does.

\renewcommand{\vec}[1]{\mathbf{#1}} % re-style the vector
  arrows.meta, % for Straight Barb arrow tip
  fit, % to fit the group box around the central neurons
  positioning, % for relative positioning of the neurons

  neuron/.style={ % style for each neuron
    circle,draw,thick, % drawn as a thick circle
    inner sep=0pt, % no built-in padding between the text and the circle shape
    minimum size=3.5em, % make each neuron the same size regardless of the text inside
    node distance=1ex and 2em, % spacing between neurons (y and x)
  group/.style={ % style for the groups of neurons
    rectangle,draw,thick, % drawn as a thick rectangle
    inner sep=0pt, % no padding between the node contents and the rectangle shape
  io/.style={ % style for the inputs/outputs
    neuron, % inherit the neuron style
    fill=gray!15, % add a fill color
  conn/.style={ % style for the connections
    -{Straight Barb[angle=60:2pt 3]}, % simple barbed arrow tip
    thick, % draw in a thick weight to match other drawing elements

  \node[neuron] (ht1) {$h_t^1$};
  \node[neuron,below=of ht1] (htm) {$h_t^{n-m}$};
  \node[neuron,below=of htm] (htn) {$h_t^n$};
  \node[group,fit={(ht1) (htm) (htn)}] (gr1) {};
  \node[neuron,left=of ht1] (ht-1) {$h_{t-1(2)}^1$};
  \node[io,above=2em of ht1] (yt) {$y_t$};
  \node[io,below=2em of htn] (vt) {$\vec{v}_t$};
  \draw[conn] (ht1) -- (yt);
  \draw[conn] (vt) -- (htn);
  \foreach \destination in {ht1,htm,htn} { % the for loop idea can be expanded to draw the entire diagram quickly
    \draw[conn] (ht-1.east) -- (\destination.west);

And here's the result:

enter image description here

OK, that was a lot of stuff! Let's break it down.

Each neuron is drawn as a node with a name in parentheses and the node text in curly braces, for example:

\node[neuron] (ht1)   {$h_t^1$};
%     ^-style  ^-name  ^-node text

Once that first node is placed, all others can be positioned relative to it, with no need for calculating coordinates manually, using the positioning library:

\node[neuron,below=of ht1] (htm) {$h_t^{n-m}$};
%            ^-this node should be placed below the first node (ht1)

The spacing is controlled by the value of the node distance key, which I set in the definition of the neuron style.

The group box around the central neurons is created using the fit library to size a node to fit around a list of existing nodes, like so:

\node[group,fit={(ht1) (htm) (htn)}] (gr1) {};
%     ^-style    ^- list of nodes to fit    ^-empty node text for this one

This node (gr1) has an empty text, because we just want the node to box the list of neurons, we don't need any new text inside it.

Once all the neurons are drawn, we can connect them with \draw statements, which create stroked paths between two coordinates (here, named coordinates) in parentheses. One example:

\draw[conn] (ht1) -- (yt);
%     ^      ^-start  ^-finish
%     `-apply connection style

Drawing all of these individually gets tedious quickly. Fortunately, TikZ offers the concept of for loops, which I used to create the other arrows:

\foreach \destination in {ht1,htm,htn} {
  \draw[conn] (ht-1.east) -- (\destination.west);

I added .east and .west as anchors on the nodes we're drawing to and from. This causes the connection to be drawn to that specific point on the node edge. If you leave those off, the arrow will be drawn in a straight line from node center to node center (notice how I left the anchors off in my first example).

The \foreach loop idea can be extended to draw the many neurons in your full diagram efficiently. You can check out TeXample: Neural Network for a nice example of this.

TikZ has excellent documentation, but it might look overwhelming at first glance. If you don't see why I did something, search for the macro name or key name in the manual to see how it works and what it does. You can access the manual on your PC by typing texdoc tikz at a terminal or command prompt. Or, you can access it online at http://texdoc.net/pkg/tikz (warning: it is a large PDF file). Hope this helps you get started! :-)


Doesn't really need much code:

\tikzset{neuron/.style={shape=circle, minimum size=1.25cm, 
  inner sep=0, draw, font=\small}, io/.style={neuron, fill=gray!20}}
\begin{tikzpicture}[x=2cm, y=1.5cm, >=Stealth]
\foreach \jlabel [count=\j, evaluate={\k=int(mod(\j-1,3)); \jj=int(\j-1);}]
  in {t-1, t-1(1), t-1(2), t, t(1), t(2), t+1}{
    \foreach \ilabel [count=\i] in {1, n-m, n}
        \node [neuron] at (\j, 1-\i) (h-\i-\j){$h_{\jlabel}^{\ilabel}$};
      \node [fit=(h-1-\j) (h-3-\j), inner sep=0, draw] (b-\j) {};
      \node [io, above=of b-\j] (y-\j) {$y_{\jlabel}$};
      \node [io, below=of b-\j] (v-\j) {$v_{\jlabel}$};
      \draw [->] (v-\j) -- (b-\j);
      \draw [->] (b-\j) -- (y-\j);
      \foreach\i in {1, 2, 3}
        \foreach \ii in {1, 2, 3}
           \draw [->] (h-\i-\jj.east) -- (h-\ii-\j.west);
\node [left=of h-2-1] {\ldots};
\node [right=of h-2-7] {\ldots};

enter image description here

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.