# Batch vs mini batch vs stochastic gradient descent

I would like to compare in a figure the steps of a running execution of gradient descent algorithm but taking three possible approaches: batch, mini-batch, and stochastic.

I have found an example of batch gradient descent in Plotting the gradient descent.

\documentclass[tikz, margin=3mm]{standalone}
\usepackage{physics,amsmath} %\vb{}
%
% \usefonttheme[onlymath]{serif} %\vec{}

\usetikzlibrary{arrows.meta,
bending,
intersections,
quotes,
shapes.geometric}

\begin{document}
\begin{tikzpicture}[
every edge/.style = {draw, -{Triangle[angle=60:1pt 3,flex]},
bend right=11, blue,ultra thick},
every edge quotes/.style = {font=\scriptsize, inner sep=1pt,
auto, sloped}
]
\path[name path=C] foreach \i in {4, 8, 16, 22, 28}
\foreach \i in  {4, 8, 16, 22, 28}
\draw[line width=11.2/\i, draw=white!\i!gray]
\path[name path=V] (-4,2.4) .. controls + (0,-2) and + (-2,0) .. (0,0);
%
\draw [name intersections={of=C and V, sort by=C, name=A}]
(A-5) edge ["${\boldsymbol{\theta}}$"] (A-4)
(A-4) edge ["${\boldsymbol{\theta}}$"] (A-3)
(A-3) edge ["${\boldsymbol{\theta}}$"] (A-2);
\end{tikzpicture}
\end{document} And I would like to make a comparison like in this figure: • Are you able to program the three algorithms? At least in a simple case, say for the function x^2 +4y^2. If "yes", do it; then, either do it with math library in tikz or recuperate the descent steps. Either way, you'll have your drawing. If "no", your drawing will represent nothing. Feb 8 at 4:40
• @DanielN, thank you. You are right about programming the method. However, I intend to illustrate with a scheme, like the blue line in the example. I want something like a random walk between two points and somehow controlling the direction because stochastic is more random than mini-batch. Feb 8 at 8:22