# Making a working neuronal network

I've got the following two MWE:

\documentclass[border=5pt,tikz]{standalone}
\begin{document}

\begin{tikzpicture}

%%% %%% %%%
%>> We are building a perceptron
%%% %%% %%%

% input vectors
\xdef\firstinputarray{{1,2}}
\xdef\secondinputarray{{4,8}}

% output vector
\xdef\outputarray{{1,0}} % The 1 means that for \firstinputarray the statement is true, for \secondinputarray is wrong

% weights
\xdef\zerow{.5}
\xdef\firstw{2}
\xdef\secondw{1.75}

% bias
\xdef\bias{1}

% Heaviside

\newcommand{\heaviside}[1]{
\ifnum#1>0
\pgfmathsetmacro{\a}{1}
\fi
\ifnum#1=0
\pgfmathsetmacro{\a}{1}
\fi
\ifnum#1<0
\pgfmathsetmacro{\a}{0}
\fi
}

% Floor function
\newcommand{\floor}[1]{
\pgfmathsetmacro{\floornumber}{int(floor(#1))}
}

\newcommand{\forheavi}[1]{
\pgfmathsetmacro{\forheavinumber}{int(floor(#1))}
}

% new weights
\pgfmathsetmacro{\firstway}{\zerow*\bias+\firstinputarray[0]*\firstw+\firstinputarray[1]*\secondw}

\forheavi{\firstway}
\heaviside{\forheavinumber}

\node (a) {\texttt{firstway}: \pgfmathprintnumber{\firstway}\qquad$\mathrm{H}(\mathtt{firstway})$:\pgfmathprintnumber{\a}};

% set new weights
\ifnum\a>0
\pgfmathsetmacro{\firstw}{.5*(\firstway-0)*\firstinputarray[0]}
\fi
\forheavi{\firstway}
\heaviside{\forheavinumber}
\node[above left,yshift=1cm] at (a.north east) {new \texttt{firstw}: \pgfmathprintnumber{\firstw}\qquad$\mathrm{H}(\mathtt{new fistw})$: \pgfmathprintnumber{\a}};

%           \forheavi{0}
%           \heaviside{\forheavinumber}
%               \node[red,draw,below=.5cm] at (a) {\pgfmathprintnumber{\a}};

\end{tikzpicture}

\end{document}

and

\documentclass[border=5pt,tikz]{standalone}
\usetikzlibrary{decorations.pathreplacing}
\tikzset{
neuron/.style={
draw,circle,inner sep=.3cm,fill=white
},
brace/.style={
decorate,decoration={brace,amplitude=.3cm},thick
}
}
\begin{document}
\begin{tikzpicture}
\foreach \x in {0,1,...,8}
{
\node[neuron] (a\x) at (0,\x) {};
}
\foreach \y in {-4,-3,...,12}
{
\node[neuron] (b\y) at (8,\y) {};
}
\foreach \z in {-2,-1,...,10}
{
\node[neuron] (c\z) at (16,\z) {};
\pgfmathsetmacro{\n}{10-\z}
\draw[->] (c\z) --+ (1,0) node[right] {\pgfmathprintnumber{\n}};
}
\foreach \x in {0,1,...,8}
\foreach \y in {-4,-3,...,12}
\foreach \z in {-2,-1,...,10}
{
\draw[->] (a\x) -- (b\y);
\draw[->] (b\y) -- (c\z);
}
\draw[brace] ([xshift=-1cm]a0.south) -- ([xshift=-1cm]a8.north) node[midway,text width=2.5cm,left=.5cm] {
\begin{tabular}{c}
input layer \\
(784 neurons)
\end{tabular}
};
\node[above=1cm] at (b12) {
\begin{tabular}{c}
hidden layer \\
($n = 15$ neurons)
\end{tabular}
};
\node[above=1cm] at (c10) {
\begin{tabular}{c}
output layer
\end{tabular}
};
\end{tikzpicture}
\end{document}


with the following output:

I've got the following questions:

• How can I achieve that I can enter the input arrays and the starting weights and the first code does the calculations (more precisely: how can I “extend“ the foreach loop so that the algorithm stops when the right resulst is achieved) and
• How can I than highlight the weights visual e.g. I let TikZ draw two nodes (which represent neurons) and the more “important“ a bond is the line (which connects the two neurons) gets thicker (this all happens step for step, so that we can create a little animation)?

P.S.: The feedforward network doesn't need to have such a big amount of neurons, this is just an illustration.

• I once wrote a macro that allows a somewhat more automatic generation of these things. Do you think this is a step in the right direction? – marmot Oct 11 at 14:59