I have the following code to draw a Neural Network, which produces the image below

\usepackage{amsmath} % for aligned
%\usepackage{amssymb} % for \mathbb
%\usepackage{etoolbox} % for \ifthen
\usepackage{listofitems} % for \readlist to create arrays
\usetikzlibrary{arrows.meta} % for arrow size
\usepackage[outline]{contour} % glow around text

\tikzset{>=latex} % for LaTeX arrow head
\tikzstyle{node}=[thick,circle,draw=myblue,minimum size=22,inner sep=0.5,outer sep=0.6]
\tikzstyle{node in}=[node,green!20!black,draw=mygreen!30!black,fill=mygreen!25]
\tikzstyle{node hidden}=[node,blue!20!black,draw=myblue!30!black,fill=myblue!20]
\tikzstyle{node convol}=[node,orange!20!black,draw=myorange!30!black,fill=myorange!20]
\tikzstyle{node out}=[node,red!20!black,draw=myred!30!black,fill=myred!20]
\tikzstyle{connect}=[thick,mydarkblue] %,line cap=round
\tikzstyle{connect arrow}=[-{Latex[length=4,width=3.5]},thick,mydarkblue,shorten <=0.5,shorten >=1]
\tikzset{ % node styles, numbered for easy mapping with \nstyle
  node 1/.style={node in},
  node 2/.style={node hidden},
  node 3/.style={node out},
\def\nstyle{int(\lay<\Nnodlen?min(2,\lay):3)} % map layer number onto 1, 2, or 3


% NEURAL NETWORK with coefficients, shifted
  \message{^^JNeural network, shifted}
  \readlist\Nnod{4,5,3} % array of number of nodes per layer
  \readlist\Nstr{q,m,J} % array of string number of nodes per layer
%\readlist\Cstr{\strut Z_{\index},w_{\index}^{(\prev)},g_{n,\index}^{}} 
\readlist\Cstr{\strut \boldsymbol{Z }_{\index},w_{\index}^{(\prev)},\boldsymbol{g}_{\index}^{}} 
  \def\yshift{0.5} % shift last node for dots
  \message{^^J  Layer}
  \foreachitem \N \in \Nnod{ % loop over layers
    \def\lay{\Ncnt} % alias of index of current layer
    \pgfmathsetmacro\prev{int(\Ncnt-1)} % number of previous layer
    \foreach \i [evaluate={\c=int(\i==\N); \y=\N/2-\i-\c*\yshift;
                 \x=\lay; \n=\nstyle;}] in {1,...,\N}{ % loop over nodes
      % NODES
      \node[node \n] (N\lay-\i) at (\x,\y) {$\Cstr[\lay]$};

      \ifnum\lay>1 % connect to previous layer
        \foreach \j in {1,...,\Nnod[\prev]}{ % loop over nodes in previous layer
          \draw[connect,white,line width=1.2] (N\prev-\j) -- (N\lay-\i);
          \draw[connect] (N\prev-\j) -- (N\lay-\i);
          %\draw[connect] (N\prev-\j.0) -- (N\lay-\i.180); % connect to left
      \fi % else: nothing to connect first layer
    \path (N\lay-\N) --++ (0,1+\yshift) node[midway,scale=1.5] {$\vdots$};
  \node[above=5,align=center,mygreen!60!black] at (N1-1.90) {input\\[-0.2em]layer};
  \node[above=1,align=center,myblue!60!black] at (N2-1.90) {hidden layer};
  \node[above=10,align=center,myred!60!black] at (N\Nnodlen-1.90) {output\\[-0.2em]layer};

enter image description here

However, I am trying to make a figure that looks like the image below, but I don't know really where to start. Could someone please give me some advice on how to proceed? Thank you a lot.

enter image description here

  • 3
    You can use \begin{scope} ...\end{scope} to surround your existing code then copy paste it. and add the option [yshift=10cm] to have the same figure twice out redoing all of it. Then, you need to draw the last layer and a rectangle as a Softmax
    – anis
    Jan 12, 2023 at 11:52
  • 2
    by looking at this line: \node \n] (N\lay-\i) at (\x,\y) {$\Cstr[\lay]$};, you can see that the node is named N<layer id>-<node id>. you can use it to link all the red nodes to all the new red nodes.
    – anis
    Jan 12, 2023 at 11:54
  • 1
    Related: Q662522 Jan 12, 2023 at 13:00

1 Answer 1


Adjusting my answer to How to exclude certain elements from a built path loop in a neural network drawing with TikZ slightly we can create the following diagram.

  • Multiple \tikzMatrixNodes of the style vertical node matrix which basically create a matrix with a bunch of nodes where the matrix is used like a node (which it is).

    • With the second to last key we switch the second to last node to be ⋮.

    • The \tikzmatrixnodecounter returns a number we can use for node contents. The last node content is set manually in the mandatory argment to \tikzMatrixNodes.

  • The row of matrices is placed via the chains library (which needs a small fix to the matrix code). Unfortunately, the nodes inside the matrix don't get named according of the chain name (chain-1, chain-2, …) which is why the matrices are named explicitly.

  • The graphs library and its complete bipartite key is used to draw almost all lines between the nodes.

  • A scope environment with a hand-picked yshift makes sure that the second set of matrices are placed below the first one.

  • After both set of matrices are placed and their nodes are connected the third output set is placed to the right=of the other two output sets. Here, the ext.positioning-plus library allows me to just do

    right=of (output'top)(output'bot),

    and TikZ automatically places the new set vertically centered to the other two.

  • Again, the graph library helps to connect all the output nodes.

  • With another help of the ext.positioning-plus library we place the softmax node, here

    right=of |output'
    % or
    right=of |(output')

    not only places the new node to right of the output' matrix but also chooses a height that is the same as the output' matrix.

  • Finally, a few edges and ext.paths.ortho's only horizontal second help to draw the lines connecting softmax.


\usetikzlibrary{arrows.meta, chains, graphs, ext.positioning-plus, ext.paths.ortho}
\ExplSyntaxOn \makeatletter % allowing matrices on chains
\tl_replace_once:Nnn \tikz@do@matrix@cont
  { \tikz@node@finish } { \tikz@alias \tikz@node@finish }
\makeatother \ExplSyntaxOff
  node matrix/.style={
    row sep=y_node_dist, column sep=x_node_dist,
    every outer matrix/.append style={/pgf/inner sep=+0pt, /pgf/outer sep=+0pt, draw=none, fill=none, shape=rectangle},
    nodes/.style={node matrix/node/.append style={##1}}},
  node matrix/node/.style={
    node contents=, anchor=center, name/.expanded={\tikzmatrixname_\tikzmatrixnodecounter}},
  node matrix/place 1st node/.code args={#1,#2}{%
    \node[node matrix/node,nm \tikzmatrixnodecounter/.try,#1];},
  node matrix/place other nodes/.style args={#1,#2}{/tikz/node matrix/place oth node/.list={#2}},
  vertical   node matrix/.style={/tikz/node matrix/place oth node/.code={%
    \pgfmatrixendrow  \node[node matrix/node,nm \tikzmatrixnodecounter/.try,##1];}},
  horizontal node matrix/.style={/tikz/node matrix/place oth node/.code={%
    \pgfmatrixnextcell\node[node matrix/node,nm \tikzmatrixnodecounter/.try,##1];}}}
  \matrix[every node matrix/.try,node matrix,#1]{
    \tikzset{node matrix/place 1st node={#2},node matrix/place other nodes={#2}}\\};}
  color let/.code args={#1=#2}{\colorlet{#1}{#2}},
  color set/.style args={#1=#2/#3}{color let={#1fg=#2}, color let={#1bg=#3}},
  color use/.style={draw=#1bg, fill=#1fg}}
  >=Latex, thick, node distance=3mm and 10mm,
  layers/.style={circle, thick, draw, minimum size=+10mm, inner sep=+.1em},
  input/.style ={layers, color use=input},
  hidden/.style={layers, color use=hidden},
  output/.style={layers, color use=output},
  every node matrix/.style={vertical node matrix},
  every label/.append style={align=center},
  label distance=1mm,
  second to last/.style={
    nm #1/.style={node contents=\vdots, text height=2ex, fill=none, draw=none, minimum size=+0pt,inner sep=+0pt, shape=rectangle},
    row #1/.append style={row sep=.75*y_node_dist},
    row \pgfinteval{#1-1}/.append style={row sep=.75*y_node_dist}},
  color set/.list={input=green!60!black/green!30!black,
\begin{scope}[start chain=going right]
    on chain, second to last=4, name=input'top, % can't use chain name
    nodes={input, node contents=$Z_\tikzmatrixnodecounter$},
  ]{,,,,node contents=$Z_q$}
    label={[hiddenfg]hidden layer},
    on chain,  second to last=5, name=hidden'top,
    nodes={hidden, node contents=$w_\tikzmatrixnodecounter^{(1)}$}
  ]{,,,,,node contents=$w_m^{(1)}$}
    on chain,  second to last=3, name=output'top,
    nodes={output, node contents=$g_\tikzmatrixnodecounter$}
  ]{,,,node contents=$g_J$}
  \path[blue] graph[use existing nodes]{
                               {\foreach \x in {1,...,3,5} {input'top_\x}}
       -- [complete bipartite] {\foreach \x in {1,...,4,6} {hidden'top_\x}}
       -- [complete bipartite] {\foreach \x in {1,2,4}     {output'top_\x}}
\begin{scope}[start chain=going right, yshift=-8.5cm]
    on chain, second to last=4, name=input'bot, % can't use chain name
    nodes={input, node contents=$Z_\tikzmatrixnodecounter$},
  ]{,,,,node contents=$Z_q$}
    on chain,  second to last=5, name=hidden'bot,
    nodes={hidden, node contents=$w_\tikzmatrixnodecounter^{(1)}$}
  ]{,,,,,node contents=$w_m^{(1)}$}
    on chain,  second to last=3, name=output'bot,
    nodes={output, node contents=$g_\tikzmatrixnodecounter$}
  ]{,,,node contents=$g_J$}
  \path[yellow!50!black] graph[use existing nodes]{
                               {\foreach \x in {1,...,3,5} {input'bot_\x}}
       -- [complete bipartite] {\foreach \x in {1,...,4,6} {hidden'bot_\x}}
       -- [complete bipartite] {\foreach \x in {1,2,4}     {output'bot_\x}}

  node distance=3mm and 20mm,
  right=of (output'top)(output'bot),
  second to last=3, name=output',
  nodes={output, node contents=$g_\tikzmatrixnodecounter$}
]{,,,node contents=$g_r$}
\graph[use existing nodes]{
  {\foreach \tb in {top, bot}{\foreach \x in {1,2,4}{output'\tb_\x}}}
  -- [complete bipartite]    {\foreach \x in {1,2,4}{output'_\x}}
  draw, minimum width=1cm,
  align=center, right=of |output'] (softmax) {S\\o\\f\\t\\m\\a\\x};
\path[->] (softmax) edge + (right:1cm);
\path[<-, only horizontal second] (softmax) edge (output'_1)
                                            edge (output'_2)
                                            edge (output'_4);


enter image description here

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .