# How to draw ELU using tikzpicture

I am trying to draw ELU graph. But I can't figure out what the formula should be. I would appreciate if someone can help please.

This is what I have in code.

    \begin{figure}[H]
\centering
\begin{tikzpicture}
\begin{axis}%
[
domain=-3:5,
legend pos=north west,
style={font=\scriptsize,row sep=5cm,/tikz/every odd column/.append style={column sep=0.01cm}
},
]
[
blue,%
mark=none,
samples=100,
domain=-5:5,
]
(x,{(x>=-2)*((exp(x))-1)});
\legend{$R(z)=\left \{ \begin{tabular}{ccc} z & z \textgreater 0 \\ \alpha.(e^z–1) & z \leq 0 \end{tabular} \right \}$
}
\end{axis}

\end{tikzpicture}

\label{fig:ELU}
\caption{ELU}
\end{figure}


I'm not that familiar with this stuff, according to https://en.wikipedia.org/wiki/Rectifier_(neural_networks)#ELUs the definition of an ELU is different from the plot you show. I'm guessing you're after

       \addplot%
[
blue,%
mark=none,
samples=100,
domain=-3:2,
]
(x,{
(x>=-2) * (exp(x)-1) + % this applies when x is \geq to the threshold value, -2
(x<-2)  * (x+(exp(-2)+1) % this applies when x is less
});


where the extra term in the second part is what you need for the function to approach the same value from both sides. I'd probably parametrize it a bit though, by defining a function and making a macro for -2, for example like the code below.

\documentclass[border=5mm]{standalone}
\usepackage{pgfplots}
\usepackage{amsmath}

\begin{document}
\begin{tikzpicture}[
declare function={ % define a function for the ELU
a = 1;
ELU(\x)={a * (exp(\x) - 1)};
}
]

% a macro for the value where the function changes
\newcommand\ELUThreshold{0}

\begin{axis}%
[
legend pos=north west,
style={font=\scriptsize,
row sep=5cm,
/tikz/every odd column/.append style={column sep=0.01cm}
},
]
[
blue,%
mark=none,
samples=100,
domain=-3:2,
]
(x,{
(x>=\ELUThreshold) * (ELU(x)) + % this applies when x is \geq to the threshold value
(x<\ELUThreshold)  * (x+(ELU(\ELUThreshold)-\ELUThreshold) % this applies when x is less
});

% some fine tuning of the legend entry
\addlegendentry[minimum height=4ex,text depth=1.5ex]{$R(z)= \begin{cases} z & z < \ELUThreshold \\ \alpha (e^z–1) & z \geq \ELUThreshold \end{cases}$
}
\end{axis}

\end{tikzpicture}

\end{document}

• Many thanks, this is exactly what I was looking for. Jun 3, 2019 at 10:24

I had the same problem and simply used two plots for the function.

        \begin{tikzpicture}
\begin{axis}[
xmin=-2,
xmax=2,
ymin=-2.1,
ymax=2.1,
grid=major,
xlabel={$v$},
ylabel={$f(v)$},
legend entries={ReLU,ELU},
legend pos=south east
]
% Draws the exp part
% Draws the linear part
\end{axis}
\end{tikzpicture}


I slightly modified it and this is more like what I need. but it is based on what Torbjørn T recommended.

   \begin{figure}[H]
\centering
\begin{tikzpicture}

\begin{axis}%
[
legend pos=north west,
style={font=\scriptsize,
row sep=5cm,
/tikz/every odd column/.append style={column sep=0.01cm}
},
]
[
blue,%
mark=none,
samples=100,
domain=-4:4,
]
(x,{
(x<=0) * (exp(x)-1) +
(x>0)  * x
});

\addlegendentry[minimum height=4ex,text depth=1.5ex]{$R(z)= \begin{cases} z & z > \ELUThreshold \\ \alpha (e^z–1) & z \leq \ELUThreshold \end{cases}$
}
\end{axis}

\end{tikzpicture}

\label{fig:ELU}
\caption{ELU}
\end{figure}