1

I'm writing about neural networks, where coordinate vectors are transformed by matrices and then pointwise transformed by a non-linear function.

As an equation, it is something like σ(wx + b), where σ is the nonlinear function, w and b are a matrix and vector correspondingly and x is the input vector, here a coordinate in TikZ.

I want to illustrate that along basic examples using TikZ. The wx + b transformation is easy to implement using the [cm={w-entries, b-coordinate}] option and I can also transform the individual coordinates using the calc library.

However, as you can see from the MWE provided below, it is in the wrong order. I have w σ(x) + b and therefore need to nudge the coordinates quite a bit. It works fine for the simple example, but fails when I go to more complex ones.

Is there an easy way to implement non-linear transformations after the coordinate transformation via cm?

\documentclass{standalone}
\usepackage{tikz}
\usetikzlibrary{calc}
\begin{document}
\pgfmathsetseed{1}
\begin{tikzpicture}[scale=2]
\fill[red!20] (0,0) -- (2.5, 2.5) -- (2.5, 0);
\fill[blue!20] (0,0) -- (2.5, 2.5) -- (0, 2.5);
\begin{scope}[cm={0, 2, -2, 0, (2.25, 0.25)}]
    \foreach \i in {0, ..., 50} {
        \draw[red] ({1/(1+exp(-3*(rnd-1.5)))}, {1/(1+exp(-3*(rnd-1.5))}) circle (0.015);
        \draw[blue] ({1/(1+exp(-3*(rnd+.5)}, {1/(1+exp(-3*(rnd+.5))}) circle (0.015);
    };
\end{scope}
\draw (0,0) -- (2.5, 2.5);
\end{tikzpicture}

\end{document}

EDIT Progress update: I've managed to use marmot's excellent answer and create the desired transformation.

However I fail to make it use any external variables. In particular, I would like to have a scaling parameter or some way to pass the surrounding scale value into the transformation. Currently I can only increase the picture size if I change the transformation value between centimeters and points. (Is there a better way of going between the two coordinate systems? Hard-scaling by 28.4 feels clunky.)

\documentclass[border=5mm]{standalone}
\usepackage{tikz}
\usepgfmodule{nonlineartransformations}
\makeatletter
\def\sigmoidtransformation{%
\edef\oriX{\the\pgf@x}%
\edef\oriY{\the\pgf@y}%
\typeout{old\space y=\oriX\space old \space y=\oriY}
\pgfmathsetmacro{\sigmoidx}{28.4/(1+exp(min(-\oriX/28,4, 5))}
\pgfmathsetmacro{\sigmoidy}{28.4/(1+exp(min(-\oriY/28.4, 5))}
\typeout{new\space x=\sigmoidx\space new\space y=\sigmoidy}
\setlength{\pgf@x}{\sigmoidx pt}
\setlength{\pgf@y}{\sigmoidy pt}
}
\begin{document}
\pgfmathsetseed{1}
\begin{tikzpicture}
\draw[red!50] (0,0) grid[xstep=0.333cm, ystep=0.333cm] (1,1);
\draw[red!50, shift={(0.5, 0.5)}] (0,0) circle (0.5);
\pgftransformnonlinear{\sigmoidtransformation}
\draw[gray] (-3,-3) grid[xstep=15pt, ystep=15pt] (3,3);
\draw[cm={1, 1, 0, 1, (1, 1)}] (0,0) circle(1);
\end{tikzpicture}

\end{document}

enter image description here

  • I think there is a minor typo in \pgfmathsetmacro{\sigmoidx}{28.4/(1+exp(min(-\oriX/28,4, 5))}, 28,4 should be 28.4. And you can avoid the 28.4s altogether using \pgfmathsetmacro{\sigmoidx}{1cm/(1+exp(min(-\oriX/1cm, 5))} \pgfmathsetmacro{\sigmoidy}{1cm/(1+exp(min(-\oriY/1cm, 5))}. However, I do not understand the question. – marmot Feb 27 at 14:16
1

The conversion from cm to pt is as simple as adding factors like 1cm/1pt. And you could make things depend on a parameter by building them into the definition of the transformation. (I also cleaned up a bit my old code.)

\documentclass[border=5mm]{standalone}
\usepackage{tikz}
\usepgfmodule{nonlineartransformations}
\makeatletter
\def\mytrafoA{1}
\def\sigmoidtransformation{%
\pgf@xa=\pgf@x%
\pgf@ya=\pgf@y%
\typeout{old\space x=\pgf@xa\space old \space y=\pgf@ya}%
\pgfmathsetmacro{\sigmoidx}{1cm/(1+exp(min(-\mytrafoA\pgf@xa/1cm, 5))}%
\pgfmathsetmacro{\sigmoidy}{1cm/(1+exp(min(-\mytrafoA\pgf@ya/1cm, 5))}%
\typeout{new\space x=\sigmoidx\space new\space y=\sigmoidy}%
\pgf@x=\sigmoidx pt
\pgf@y=\sigmoidy pt
}
\begin{document}
\begin{tikzpicture}
\draw[red!50] (0,0) grid[xstep=0.333cm, ystep=0.333cm] (1,1);
\draw[red!50, shift={(0.5, 0.5)}] (0,0) circle (0.5);
\pgftransformnonlinear{\sigmoidtransformation}
\draw[gray] (-3,-3) grid[xstep=15pt, ystep=15pt] (3,3);
\draw[cm={1, 1, 0, 1, (1, 1)}] (0,0) circle(1);
\end{tikzpicture}
\quad
\begin{tikzpicture}
\draw[red!50] (0,0) grid[xstep=0.333cm, ystep=0.333cm] (1,1);
\draw[red!50, shift={(0.5, 0.5)}] (0,0) circle (0.5);
\def\mytrafoA{2}
\pgftransformnonlinear{\sigmoidtransformation}
\draw[gray] (-3,-3) grid[xstep=15pt, ystep=15pt] (3,3);
\draw[cm={1, 1, 0, 1, (1, 1)}] (0,0) circle(1);
\end{tikzpicture}
\end{document}

enter image description here

Please note also that a circle is constructed with some Bezier curves, so if you want to have more accurately transformed circles you may want to plot them via parametric plots.

  • 1
    As always the marmot gives a great answer. Only thing I changed is such that the scale \mytrafoA scales the outside picture, not the inside, i.e. \mytrafoA*1cm/(1+exp(min(\pgf@xa/1cm, 5)). Thanks for your help! – Huang_d Feb 27 at 14:51

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.