0

I am trying to draw ELU graph. But I can't figure out what the formula should be. I would appreciate if someone can help please.

This is what I have in code.

    \begin{figure}[H]
    \centering
        \begin{tikzpicture}
            \begin{axis}%
                [
                    domain=-3:5,
                    legend pos=north west,  
                    style={font=\scriptsize,row sep=5cm,/tikz/every odd column/.append style={column sep=0.01cm}
                    },
                ]
                \addplot%
                [
                    blue,%
                    mark=none,
                    samples=100,
                    domain=-5:5,
                ]
                (x,{(x>=-2)*((exp(x))-1)});
                \legend{$
                            R(z)=\left \{
                            \begin{tabular}{ccc}
                            z & z \textgreater 0  \\
                            \alpha.(e^z–1) & z \leq 0  
                            \end{tabular}
                            \right \}
                            $
                        }
            \end{axis}

        \end{tikzpicture}

    \label{fig:ELU}
    \caption{ELU}
\end{figure}

enter image description here

2

I'm not that familiar with this stuff, according to https://en.wikipedia.org/wiki/Rectifier_(neural_networks)#ELUs the definition of an ELU is different from the plot you show. I'm guessing you're after

       \addplot%
            [
                blue,%
                mark=none,
                samples=100,
                domain=-3:2,
            ]
            (x,{
                (x>=-2) * (exp(x)-1) + % this applies when x is \geq to the threshold value, -2
                (x<-2)  * (x+(exp(-2)+1) % this applies when x is less
                });

where the extra term in the second part is what you need for the function to approach the same value from both sides. I'd probably parametrize it a bit though, by defining a function and making a macro for -2, for example like the code below.

\documentclass[border=5mm]{standalone}
\usepackage{pgfplots}
\usepackage{amsmath}

\begin{document}
\begin{tikzpicture}[
  declare function={ % define a function for the ELU
    a = 1;
    ELU(\x)={a * (exp(\x) - 1)};
    }
]

% a macro for the value where the function changes
\newcommand\ELUThreshold{0}

            \begin{axis}%
                [
                    legend pos=north west,  
                    style={font=\scriptsize,
                           row sep=5cm,
                           /tikz/every odd column/.append style={column sep=0.01cm}
                    },
                ]
                \addplot%
                [
                    blue,%
                    mark=none,
                    samples=100,
                    domain=-3:2,
                ]
                (x,{
                    (x>=\ELUThreshold) * (ELU(x)) + % this applies when x is \geq to the threshold value
                    (x<\ELUThreshold)  * (x+(ELU(\ELUThreshold)-\ELUThreshold) % this applies when x is less
                    });

                % some fine tuning of the legend entry
                \addlegendentry[minimum height=4ex,text depth=1.5ex]{$R(z)=
                            \begin{cases}
                            z & z < \ELUThreshold  \\
                            \alpha (e^z–1) & z \geq \ELUThreshold 
                            \end{cases}
                            $
                        }
            \end{axis}

        \end{tikzpicture}

\end{document} 
| improve this answer | |
  • Many thanks, this is exactly what I was looking for. – Zardaloop Jun 3 '19 at 10:24
0

I slightly modified it and this is more like what I need. but it is based on what Torbjørn T recommended.

   \begin{figure}[H]
    \centering
       \begin{tikzpicture}

            \begin{axis}%
                [
                    legend pos=north west,  
                    style={font=\scriptsize,
                    row sep=5cm,
                    /tikz/every odd column/.append style={column sep=0.01cm}
                    },
                ]
                \addplot%
                [
                    blue,%
                    mark=none,
                    samples=100,
                    domain=-4:4,
                ]
                (x,{
                    (x<=0) * (exp(x)-1) +
                    (x>0)  * x
                    });

                \addlegendentry[minimum height=4ex,text depth=1.5ex]{$R(z)=
                            \begin{cases}
                            z & z > \ELUThreshold  \\
                            \alpha (e^z–1) & z \leq \ELUThreshold 
                            \end{cases}
                            $
                        }
            \end{axis}

        \end{tikzpicture}

    \label{fig:ELU}
    \caption{ELU}
\end{figure}
| improve this answer | |

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.