2

I would like to compare in a figure the steps of a running execution of gradient descent algorithm but taking three possible approaches: batch, mini-batch, and stochastic.

I have found an example of batch gradient descent in Plotting the gradient descent.

\documentclass[tikz, margin=3mm]{standalone}
\usepackage{physics,amsmath} %\vb{}
%
% \usefonttheme[onlymath]{serif} %\vec{}

\usetikzlibrary{arrows.meta,
                bending,
                intersections,
                quotes,
                shapes.geometric}

\begin{document}
    \begin{tikzpicture}[
every edge/.style = {draw, -{Triangle[angle=60:1pt 3,flex]},
                             bend right=11, blue,ultra thick},
every edge quotes/.style = {font=\scriptsize, inner sep=1pt, 
                            auto, sloped}
                            ]
\fill (0,0) circle[radius=3pt];
\path[name path=C] foreach \i in {4, 8, 16, 22, 28}
        {(0,0) circle[draw=red!\i, x radius=2*\i mm, y radius=\i mm, rotate=-5]};
\foreach \i in  {4, 8, 16, 22, 28}
    \draw[line width=11.2/\i, draw=white!\i!gray]
        (0,0) circle[x radius=2*\i mm, y radius=\i mm, rotate=-5];
\path[name path=V] (-4,2.4) .. controls + (0,-2) and + (-2,0) .. (0,0);
%
\draw [name intersections={of=C and V, sort by=C, name=A}]
        (A-5) edge ["${\boldsymbol{\theta}[0]}$"] (A-4)
        (A-4) edge ["${\boldsymbol{\theta}[1]}$"] (A-3)
        (A-3) edge ["${\boldsymbol{\theta}[2]}$"] (A-2);
    \end{tikzpicture}
\end{document}

enter image description here

And I would like to make a comparison like in this figure:

enter image description here

Improve this question
2
  • Are you able to program the three algorithms? At least in a simple case, say for the function x^2 +4y^2. If "yes", do it; then, either do it with math library in tikz or recuperate the descent steps. Either way, you'll have your drawing. If "no", your drawing will represent nothing.
    – Daniel N
    Commented Feb 8, 2023 at 4:40
  • @DanielN, thank you. You are right about programming the method. However, I intend to illustrate with a scheme, like the blue line in the example. I want something like a random walk between two points and somehow controlling the direction because stochastic is more random than mini-batch.
    – pablo
    Commented Feb 8, 2023 at 8:22

0

Reset to default

You must log in to answer this question.

Browse other questions tagged .