# Non-linear coordinate transformations

I'm writing about neural networks, where coordinate vectors are transformed by matrices and then pointwise transformed by a non-linear function.

As an equation, it is something like σ(wx + b), where σ is the nonlinear function, w and b are a matrix and vector correspondingly and x is the input vector, here a coordinate in TikZ.

I want to illustrate that along basic examples using TikZ. The wx + b transformation is easy to implement using the [cm={w-entries, b-coordinate}] option and I can also transform the individual coordinates using the calc library.

However, as you can see from the MWE provided below, it is in the wrong order. I have w σ(x) + b and therefore need to nudge the coordinates quite a bit. It works fine for the simple example, but fails when I go to more complex ones.

Is there an easy way to implement non-linear transformations after the coordinate transformation via cm?

\documentclass{standalone}
\usepackage{tikz}
\usetikzlibrary{calc}
\begin{document}
\pgfmathsetseed{1}
\begin{tikzpicture}[scale=2]
\fill[red!20] (0,0) -- (2.5, 2.5) -- (2.5, 0);
\fill[blue!20] (0,0) -- (2.5, 2.5) -- (0, 2.5);
\begin{scope}[cm={0, 2, -2, 0, (2.25, 0.25)}]
\foreach \i in {0, ..., 50} {
\draw[red] ({1/(1+exp(-3*(rnd-1.5)))}, {1/(1+exp(-3*(rnd-1.5))}) circle (0.015);
\draw[blue] ({1/(1+exp(-3*(rnd+.5)}, {1/(1+exp(-3*(rnd+.5))}) circle (0.015);
};
\end{scope}
\draw (0,0) -- (2.5, 2.5);
\end{tikzpicture}

\end{document}


EDIT Progress update: I've managed to use marmot's excellent answer and create the desired transformation.

However I fail to make it use any external variables. In particular, I would like to have a scaling parameter or some way to pass the surrounding scale value into the transformation. Currently I can only increase the picture size if I change the transformation value between centimeters and points. (Is there a better way of going between the two coordinate systems? Hard-scaling by 28.4 feels clunky.)

\documentclass[border=5mm]{standalone}
\usepackage{tikz}
\usepgfmodule{nonlineartransformations}
\makeatletter
\def\sigmoidtransformation{%
\edef\oriX{\the\pgf@x}%
\edef\oriY{\the\pgf@y}%
\typeout{old\space y=\oriX\space old \space y=\oriY}
\pgfmathsetmacro{\sigmoidx}{28.4/(1+exp(min(-\oriX/28,4, 5))}
\pgfmathsetmacro{\sigmoidy}{28.4/(1+exp(min(-\oriY/28.4, 5))}
\typeout{new\space x=\sigmoidx\space new\space y=\sigmoidy}
\setlength{\pgf@x}{\sigmoidx pt}
\setlength{\pgf@y}{\sigmoidy pt}
}
\begin{document}
\pgfmathsetseed{1}
\begin{tikzpicture}
\draw[red!50] (0,0) grid[xstep=0.333cm, ystep=0.333cm] (1,1);
\draw[red!50, shift={(0.5, 0.5)}] (0,0) circle (0.5);
\pgftransformnonlinear{\sigmoidtransformation}
\draw[gray] (-3,-3) grid[xstep=15pt, ystep=15pt] (3,3);
\draw[cm={1, 1, 0, 1, (1, 1)}] (0,0) circle(1);
\end{tikzpicture}

\end{document} • I think there is a minor typo in \pgfmathsetmacro{\sigmoidx}{28.4/(1+exp(min(-\oriX/28,4, 5))}, 28,4 should be 28.4. And you can avoid the 28.4s altogether using \pgfmathsetmacro{\sigmoidx}{1cm/(1+exp(min(-\oriX/1cm, 5))} \pgfmathsetmacro{\sigmoidy}{1cm/(1+exp(min(-\oriY/1cm, 5))}. However, I do not understand the question. – user121799 Feb 27 '19 at 14:16

The conversion from cm to pt is as simple as adding factors like 1cm/1pt. And you could make things depend on a parameter by building them into the definition of the transformation. (I also cleaned up a bit my old code.)

\documentclass[border=5mm]{standalone}
\usepackage{tikz}
\usepgfmodule{nonlineartransformations}
\makeatletter
\def\mytrafoA{1}
\def\sigmoidtransformation{%
\pgf@xa=\pgf@x%
\pgf@ya=\pgf@y%
\typeout{old\space x=\pgf@xa\space old \space y=\pgf@ya}%
\pgfmathsetmacro{\sigmoidx}{1cm/(1+exp(min(-\mytrafoA\pgf@xa/1cm, 5))}%
\pgfmathsetmacro{\sigmoidy}{1cm/(1+exp(min(-\mytrafoA\pgf@ya/1cm, 5))}%
\typeout{new\space x=\sigmoidx\space new\space y=\sigmoidy}%
\pgf@x=\sigmoidx pt
\pgf@y=\sigmoidy pt
}
\begin{document}
\begin{tikzpicture}
\draw[red!50] (0,0) grid[xstep=0.333cm, ystep=0.333cm] (1,1);
\draw[red!50, shift={(0.5, 0.5)}] (0,0) circle (0.5);
\pgftransformnonlinear{\sigmoidtransformation}
\draw[gray] (-3,-3) grid[xstep=15pt, ystep=15pt] (3,3);
\draw[cm={1, 1, 0, 1, (1, 1)}] (0,0) circle(1);
\end{tikzpicture}
\begin{tikzpicture}
\draw[red!50] (0,0) grid[xstep=0.333cm, ystep=0.333cm] (1,1);
\draw[red!50, shift={(0.5, 0.5)}] (0,0) circle (0.5);
\def\mytrafoA{2}
\pgftransformnonlinear{\sigmoidtransformation}
\draw[gray] (-3,-3) grid[xstep=15pt, ystep=15pt] (3,3);
\draw[cm={1, 1, 0, 1, (1, 1)}] (0,0) circle(1);
\end{tikzpicture}
\end{document} Please note also that a circle is constructed with some Bezier curves, so if you want to have more accurately transformed circles you may want to plot them via parametric plots.

• As always the marmot gives a great answer. Only thing I changed is such that the scale \mytrafoA scales the outside picture, not the inside, i.e. \mytrafoA*1cm/(1+exp(min(\pgf@xa/1cm, 5)). Thanks for your help! – Huang_d Feb 27 '19 at 14:51