enter image description here

The equation is basically Bellman equation essential to the Q-learning algorithm. If there's any way to include the notes below (such as "New Q-Value") aswell that'd be even better!

  • 2
    welcome to tex.se! please provide small but complete document with your equation (without comments under it), that we no need type it from scratch. help us to help you! – Zarko May 21 at 12:14

My welcome to TeX.SE and you here have a my proposal done as pure LaTeX. There is a bit of difference from your image because there is much text in a specific part of the formula (Maximum predicted reward, given new state and all possible actions).

Some small suggestions:

  1. In LaTeX there is a specific command to indicate the maximum and it is \max that is not written in italics (max) like each letter is a variable. The same discussion for "New", which I wrote as text into a formula \text{New}, not in italics.
  2. An important indication: if you change something in the formula that you have inserted as an image, the spaces may change. This can be easily resolved by reducing or increasing the distance with the command \mkern-30mu (for example).

enter image description here

\[\underbrace{\text{New}Q(s,a)}_{\scriptstyle\text{New Q-Value}}=Q(s,a)+\mkern-34mu\underset{\text{New Q-Value}}{\underset{\Bigl|}{\alpha}}\mkern-30mu[\underbrace{R(s,a)}_{\scriptstyle\text{Reward}}+\mkern-30mu\underset{\text{Discount rate}}{\underset{\Biggl|}{\gamma}}\mkern-75mu\overbrace{\max Q'(s',a')}^{\scriptstyle\substack{\text{Maximum predicted reward, given} \\ \text{new state and all possible actions}}}\mkern-45mu-Q(s,a)]\]
  • 2
    Nice answer showing all the alternatives and more for the user to play with without being just another copy – KJO May 21 at 12:21
  • You're always so kind to me and I thank you so much. The first thing I thought was not to use TikZ probably because I really like fantasy and imagination and especially creativity. Thanks always. – Sebastiano May 21 at 19:48

An alternative, with use of tikzmark library:


\underbrace{\text{New }Q(s,a)}_{\substack{\text{New}\\
    = \underbrace{Q(s,a)}_{\substack{\text{Current}\\
    + \tikzmarknode{A}{\alpha}
    + \tikzmarknode{B}{\gamma}
        \overbrace{\max Q'(s',a')}^{\mathclap{%
            \substack{\text{Maximum predicted reward, given} \\
                      \text{new state and all possible actions}}
    - Q(s,a)
\begin{tikzpicture}[overlay, remember picture,shorten <=1mm, font=\footnotesize, align=center]
\draw (A.south) -- ++ (0,-.8) node (C) [below] {Learning\\ rate};
\draw (B.south) -- (B |- C.north)  node[below] {Discount\\ rate};

After two compilation the result is:

enter image description here

  • I'm too spartan :-) like Diogenes who always went around dressed in essential clothes :-) +1 – Sebastiano May 21 at 19:46

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.