# How to draw tikz image of the ridge regression image?

I am trying to recreate the following image:

Unfortunately, this has been drawn in Python, so I can't really use it. So I thought I should recreate it with Tikz.

My attempt at creating the left image:

\usetikzlibrary{arrows}
\usetikzlibrary{decorations.markings}
\begin{tikzpicture}
\coordinate (Origin) at (0,0);
\coordinate (OLSEstimates) at (2,8);
\coordinate (Intersect) at (2.55,4.3);
%0.5 scaled
\draw[rotate=45] (OLSEstimates) ellipse (1.275 and 3);
%0.75 scaled
\draw[rotate=45] (OLSEstimates) ellipse (1.9125 and 4.5);
%original boundary
\draw[rotate=45] (OLSEstimates) ellipse (2.56 and 6);
\draw [<->,thick] (0,10) node (yaxis) [above] {$y$}
|- (10,0) node (xaxis) [right] {$x$};
\draw (0,0) circle (5);
%draw line to intersection
\draw[draw=black,-triangle 90] (Origin) -- (Intersect);
\end{tikzpicture}


The problem now is, even though it looks like my circle is hitting the ellipse, it has been done painfully by manually adjusting the coordinates (so it's probably wrong). Second, the point intersection between the ellipse and the circle has also been done with the same method, so it's probably off by a bit.

Now I saw something crazy, that Tikz could calculate the coordinate of the intersections of two lines, so I was wondering, whether some kind of technique could be applied here.

A small sidenote: it would be perfect if it could even automatically scale the ellipse to the correct "size" as to intersect it at exactly one point, but I guess this is impossible (I've never seen someone do that).

Thanks!

• pythontex and sagetex let you use Python. See, for example, my answer here where I used matplotlib code in the LaTeX document. – DJP Mar 21 '15 at 17:37

Modified the code to place the concentric ellipses and the point. Same idea with anchoring the ellipses to the square node is possible too.

The orientation of the ellipses can be instead changed by anchoring them with respect to the origin node (but I didn't understand that part) which I think the ridge equation should point to.

\documentclass[tikz]{standalone}
\usetikzlibrary{shapes.geometric,calc}
\begin{document}
\begin{tikzpicture}[>=latex]
\clip (-1,-1) rectangle (6,6);
\draw[->,thick] (-1,0) --++(6,0);
\draw[->,thick] (0,-1) --++(0,6);
% Given
\def\mypoint{(3,2)}
% computed
\node[circle,draw,minimum height=2*\myradius] (o) at (0,0) {};
% here we take the point and compute the distance to the circle node
% and also the angle of the point wrt to origin. Then we rotate ellipses and adjust the size
\path let \p1=\mypoint,\n1 = {veclen(\x1,\y1)-\myradius},\n2={atan2(\y1,\x1)} in
\foreach \x in {1,0.75,0.5}{
node[ellipse,draw,
minimum height=2*\n1*\x,
minimum width=3.5*\n1*\x,
rotate=\n2-90] (a) at \mypoint {}
};
\draw[->] (o.center) -- (a.center)
node[above,inner sep=1pt,rounded corners,fill=white,draw] {$\theta_{\text{Normal Equation}}$};
\draw[->] (o.center) -- (o.80)
node[above,inner sep=1pt,rounded corners,fill=white,draw] {$\theta_{\text{RidgeEquation}}$};
\end{tikzpicture}
\end{document}


You can try something like this, but there should be more efficient methods out there:

\usetikzlibrary{intersections,calc}
\usetikzlibrary{arrows}
\usetikzlibrary{decorations.markings}
\begin{tikzpicture}
\draw [<->,thick] (0,10) node (yaxis) [above] {$y$}
|- (10,0) node (xaxis) [right] {$x$};

\coordinate (Origin) at (0,0);
\coordinate (OLSEstimates) at (2,8);
\coordinate (Intersect) at (0,5.1); % Initial guess, must be inside the ellipse
%original boundary
\draw[rotate=45, name path=ellipse] (OLSEstimates) ellipse (2.56 and 6);
%scaled boundaries
\foreach \scale in {0.75,0.5} {
\draw[rotate=45] (OLSEstimates) ellipse (\scale*2.56 and \scale*6);
}
%Compute intersection point iteratively
\foreach \j in {1,...,3} {
\path[name path=circle-\j, overlay] let \p1 = ($(Origin)-(Intersect)$) in (Origin) circle ({veclen(\x1,\y1)});
\draw[name intersections={of=ellipse and circle-\j,by={a,b}, total=\t}] let \p1 = ($0.5*(a)+0.5*(b)$) in (\x1,\y1) coordinate (Intersect);
}
\draw let \p1 = ($(Origin)-(Intersect)$) in (Origin) circle ({veclen(\x1,\y1)});
%draw line to intersection
\draw[draw=black,-triangle 90] (Origin) -- (Intersect);
\end{tikzpicture}


We want to find a circle such that it has only one point of intersection with the ellipse. In this code, it is computed iteratively by taking the center of the two intersection points between a circle and the ellipse, and making the next circle cross this point.

The code works, but is extremely slow!

As for drawing the ellipsis, I put the \draw command in a foreach loop, it is quite simple as you can see.