# Want to improve ANN presentation adding the activation functions

Well, there is actually two problems that I want to resolve, but I don't know if I should do two posts. Anyway...

I have this representation of my ANN and want to include the action functions, but I also don't know how to make it looks better

            \documentclass{article}
\usepackage{tikz}
\begin{document}
\pagestyle{empty}
\def\layersep{1.5cm}
\begin{tikzpicture}[
shorten >=1pt,->,
draw=black!50,
node distance=\layersep,
every pin edge/.style={<-,shorten <=1pt},
neuron/.style={circle,fill=black!25,minimum size=17pt,inner sep=0pt},
input neuron/.style={neuron, fill=green!50},
output neuron/.style={neuron, fill=red!50},
hidden neuron/.style={neuron, fill=blue!50},
annot/.style={text width=4em, text centered}
]
% Draw the input layer nodes
\foreach \name / \y in {1,...,3}
% This is the same as writing \foreach \name / \y in {1/1,2/2,3/3,4/4}
\node[input neuron, pin=left:Input \#\y] (I-\name) at (0,-\y) {};
% set number of hidden layers
\newcommand\Nhidden{2}
% Draw the hidden layer nodes
\foreach \N in {1,...,\Nhidden} {
\foreach \y in {1,...,12} {
\path[yshift=4cm]
node[hidden neuron] (H\N-\y) at (\N*\layersep,-\y cm) {};
}
\node[annot,above of=H\N-1, node distance=1cm] (hl\N) {Hidden layer \N};
}
% Draw the output layer node
\node[output neuron,pin={[pin edge={->}]right:Output}, right of=H\Nhidden-6] (O) {};

% Connect every node in the input layer with every node in the
% hidden layer.
\foreach \source in {1,...,3}
\foreach \dest in {1,...,12}
\path (I-\source) edge (H1-\dest);
% connect all hidden stuff
\foreach [remember=\N as \lastN (initially 1)] \N in {2,...,\Nhidden}
\foreach \source in {1,...,12}
\foreach \dest in {1,...,12}
\path (H\lastN-\source) edge (H\N-\dest);
% Connect every node in the hidden layer with the output layer
\foreach \source in {1,...,12}
\path (H\Nhidden-\source) edge (O);
% Annotate the layers
\node[annot,left of=hl1] {Input layer};
\node[annot,right of=hl\Nhidden] {Output layer};
\end{tikzpicture}
% End of code
\end{document}


• you can write text in the {} at the end of the \node lines. You'll have to spend some time tweaking the loops and distances though – Thorbjørn E. K. Christensen Nov 15 '17 at 8:28

You can write text in the curly brackets after the nodes (as below), however this ruins the spacing of the original drawing, and it looks really clouded.

\documentclass{standalone}
\usepackage{tikz}
\begin{document}
\pagestyle{empty}
\def\layersep{3cm}
\def\nodeinlayersep{1.5cm}
\begin{tikzpicture}[
shorten >=1pt,->,
draw=black!50,
node distance=\layersep,
every pin edge/.style={<-,shorten <=1pt},
neuron/.style={circle,fill=black!25,minimum size=17pt,inner sep=0pt},
input neuron/.style={neuron, fill=green!50},
output neuron/.style={neuron, fill=red!50},
hidden neuron/.style={neuron, fill=blue!50},
annot/.style={text width=4em, text centered}
]
% Draw the input layer nodes
\foreach \name / \y in {1,...,3}
% This is the same as writing \foreach \name / \y in {1/1,2/2,3/3,4/4}
\node[input neuron, pin=left:Input \#\y] (I-\name) at (0,-\y) {};
% set number of hidden layers
\newcommand\Nhidden{2}
% Draw the hidden layer nodes
\foreach \N in {1,...,\Nhidden} {
\foreach \y in {1,...,12} {
\path[yshift=6cm]
node[hidden neuron] (H\N-\y) at (\N*\layersep,-\y*\nodeinlayersep ) {$\frac{1}{1+e^{-x}}$};
}
\node[annot,above of=H\N-1, node distance=1cm] (hl\N) {Hidden layer \N};
}
% Draw the output layer node
\node[output neuron,pin={[pin edge={->}]right:Output}, right of=H\Nhidden-6] (O) {};
%
% Connect every node in the input layer with every node in the
% hidden layer.
\foreach \source in {1,...,3}
\foreach \dest in {1,...,12}
\path (I-\source) edge (H1-\dest);
% connect all hidden stuff
\foreach [remember=\N as \lastN (initially 1)] \N in {2,...,\Nhidden}
\foreach \source in {1,...,12}
\foreach \dest in {1,...,12}
\path (H\lastN-\source) edge (H\N-\dest);
% Connect every node in the hidden layer with the output layer
\foreach \source in {1,...,12}
\path (H\Nhidden-\source) edge (O);
% Annotate the layers
\node[annot,left of=hl1] {Input layer};
\node[annot,right of=hl\Nhidden] {Output layer};
\end{tikzpicture}
% End of code
\end{document}

• Is there a way to draw the graph of the function instead of the function? Is it too hard? @ThorbjørnEK – Caio Custódio Nov 16 '17 at 17:43
• Yes, this can quite easily be done by exchanging $\frac{1}{1+e^{-x}}$ With \tikz\draw[scale=0.05,samples=100,domain=-6:6,smooth,variable=\x,red,-] plot ({\x},{4/(1+exp(-\x))}); @CaioCustódio – Thorbjørn E. K. Christensen Nov 16 '17 at 18:18
• @ThorbjørnEK worked with an error: ' Package pgfkeys Error: I do not know the key '/tikz/variabl‌​e', to which you passed '\x ', and I am going to ignore it. Perhaps you misspelled it. }' – Caio Custódio Nov 16 '17 at 19:33
• I don't get any error, so that's wierd. – Thorbjørn E. K. Christensen Nov 16 '17 at 19:38
• I did it again and worked, maybe a case of bad copy & past. Thank you! So it works with every other kind of function? @ThorbjørnEK – Caio Custódio Nov 16 '17 at 19:41