8

This is what I want to achieve:

mdp

This is what I have done so far:

mdp_1

This is the code to reproduce my image. I was inspired by this answer:

\documentclass{standalone}
\usepackage[utf8]{inputenc}
\usepackage{tikz}
\usetikzlibrary{arrows,shadows,positioning}

\tikzset{
  frame/.style={
    rectangle, draw,
    text width=6em, text centered,
    minimum height=4em,drop shadow,fill=white,
    rounded corners,
  },
  line/.style={
    draw, -latex',rounded corners=3mm,
  }
}

\begin{document}
\begin{tikzpicture}[font=\small\sffamily\bfseries,very thick,node distance = 4cm]
\node [frame] (agent) {Agent};
\node [below right of=agent] (aer) {};
\node [below left of=agent] (ael) {};
\node [frame, below left of=aer] (environment) {Environment};

\path [line] (agent)
  -| node[right, pos=1, align=left] {action\\ $A_t$} (aer)
  |- (environment);
\path [line] (environment.170)
  -| node[right, pos=1, align=left] {reward\\ $R_t$} (ael)
  |- (agent.190);
\path [line] (environment.190)
  -| node[left, pos=1, align=right] {state\\ $s_t$} (ael)
  |- (agent.170);
\end{tikzpicture}
\end{document}

I don't want a perfect result. I'm interested in maintaining the reward of state and reward separated and I don't want that blank space at the center of each path. Maybe a better way to start/end paths would be appreciated: currently, it's done with an angle but I don't know which is the actual distance between lines.


This is how I modified marmot's answer to have the same distance between state and reward lines both horizontally and vertically.

\documentclass{standalone}
\usepackage[utf8]{inputenc}
\usepackage{tikz}
\usetikzlibrary{arrows.meta,shadows,positioning}

\tikzset{
  frame/.style={
    rectangle, draw,
    text width=6em, text centered,
    minimum height=4em,drop shadow,fill=white,
    rounded corners,
  },
  line/.style={
    draw, -{Latex},rounded corners=3mm,
  }
}

\begin{document}
\begin{tikzpicture}[font=\small\sffamily\bfseries,very thick,node distance = 4cm]
\node [frame] (agent) {Agent};
\node [frame, below=1.2cm of agent] (environment) {Environment};
\draw[line] (agent) -- ++ (3.5,0) |- (environment)
node[right,pos=0.25,align=left] {action\\ $A_t$};
\coordinate[left=8mm of environment] (P);
\coordinate[above=3mm of environment.west] (ENW);
\coordinate[below=3mm of environment.west] (ESW);
\coordinate[above=3mm of agent.west] (ANW);
\coordinate[below=3mm of agent.west] (ASW);
\draw[thin,dashed] (P|-environment.north) -- (P|-environment.south);
\draw[line] (ESW) -- (P |- ESW)
node[midway,above]{$S_{i+1}$};
\draw[line,thick] (ENW) -- (P |- ENW)
node[midway,above]{$R_{i+1}$};
\draw[line] (P |- ESW) -- ++ (-1.4,0) |- (ANW)
node[left, pos=0.25, align=right] {state\\ $s_t$};
\draw[line,thick] (P |- ENW) -- ++ (-0.8,0) |- (ASW)
node[right,pos=0.25,align=left] {reward\\ $R_t$};
\end{tikzpicture}
\end{document}

And this is the result.

mdp_3

This is already the visual result that I wanted to achieve. The question is if I could have avoided to define four new coordinates (ASW,ANW,ESW,ENW).

7

You were loading but not using the positioning library, that is instead of right of=, say, please use right=of or right=5mm of. And one can kick out several auxiliary nodes/coordinates like (ael) and (aer), they are not at all needed.

\documentclass{standalone}
\usepackage[utf8]{inputenc}
\usepackage{tikz}
\usetikzlibrary{arrows.meta,shadows,positioning}

\tikzset{
  frame/.style={
    rectangle, draw,
    text width=6em, text centered,
    minimum height=4em,drop shadow,fill=white,
    rounded corners,
  },
  line/.style={
    draw, -{Latex},rounded corners=3mm,
  }
}

\begin{document}
\begin{tikzpicture}[font=\small\sffamily\bfseries,very thick,node distance = 4cm]
\node [frame] (agent) {Agent};
\node [frame, below=1.2cm of agent] (environment) {Environment};
\draw[line] (environment) -- ++ (3.5,0) |- (agent) 
node[right,pos=0.25,align=left] {action\\ $A_t$};
\coordinate[left=8mm of environment] (P);
\draw[thin,dashed] (P|-environment.north) -- (P|-environment.south);
\draw[line] (environment.200) -- (P |- environment.200)
node[midway,above]{$S_{i+1}$};
\draw[line,thick] (environment.160) -- (P |- environment.160)
node[midway,above]{$R_{i+1}$};
\draw[line] (P |- environment.200) -- ++ (-1.4,0) |- (agent.160)
node[left, pos=0.25, align=right] {state\\ $s_t$};
\draw[line,thick] (P |- environment.160) -- ++ (-0.8,0) |- (agent.200)
node[right,pos=0.25,align=left] {reward\\ $R_t$};
\end{tikzpicture}
\end{document}

enter image description here

ADDENDUM: If you want to adjust the distances of the two lines, there is no need to introduce new coordinat (However, your way to achieve this with additional coordinates is very elegant, I think. Just in the original question the coordinates (ael) and (aer) were not necessarily a good choice as they interrupted paths. In principle one could avoid introducing P in the following by switching to decorations.markings, but this might be an overkill.)

\documentclass{standalone}
\usepackage[utf8]{inputenc}
\usepackage{tikz}
\usetikzlibrary{arrows.meta,shadows,positioning}

\tikzset{
  frame/.style={
    rectangle, draw,
    text width=6em, text centered,
    minimum height=4em,drop shadow,fill=white,
    rounded corners,
  },
  line/.style={
    draw, -{Latex},rounded corners=3mm,
  }
}

\begin{document}
\begin{tikzpicture}[font=\small\sffamily\bfseries,very thick,node distance = 4cm]
\node [frame] (agent) {Agent};
\node [frame, below=1.2cm of agent] (environment) {Environment};
\draw[line] (environment) -- ++ (3.5,0) |- (agent) 
node[right,pos=0.25,align=left] {action\\ $A_t$};
\coordinate[left=12mm of environment] (P);
\draw[thin,dashed] (P|-environment.north) -- (P|-environment.south);
\pgfmathsetmacro{\Ldist}{4mm}
\draw[line] ([yshift=-\Ldist]environment.west) -- 
([yshift=-\Ldist]environment.west -| P) node[midway,above]{$S_{i+1}$};
\draw[line,thick] ([yshift=\Ldist]environment.west) -- ([yshift=\Ldist]environment.west
-|P) node[midway,above]{$R_{i+1}$};
\draw[line] ([yshift=-\Ldist]environment.west -| P) -- ++ (-12mm-\Ldist,0) |- 
([yshift=\Ldist]agent.west) node[left, pos=0.25, align=right] {state\\ $S_t$};
\draw[line,thick] ([yshift=\Ldist]environment.west -| P) -- ++ (-12mm+\Ldist,0) 
|- ([yshift=-\Ldist]agent.west) node[right,pos=0.25,align=left] {reward\\ $R_t$};
\end{tikzpicture}
\end{document}

enter image description here

  • Ok, this is so good that I've almost scared to ask for more but I feel brave today. Could I set in some way the offset of the state and reward arrows with respect to environment and agent? I mean without angles but with cartesian offset. Or should I simply use more \coordinates? – gvgramazio Nov 22 '18 at 17:26
  • @gvgramazio I'm sorry, I do not understand. Do you want to move the dashed line around? If yes, just change \coordinate[left=8mm of environment] (P); to \coordinate[left=12mm of environment] (P);, say. If you want something else, could you please try to reword your request a bit? – marmot Nov 22 '18 at 17:30
  • My bad. The horizontal space between state and reward lines is 0.6 because is 1.4-0.8. The vertical space between state and reward lines is determined through an angle (200 and 160). How could I set the vertical space directly? – gvgramazio Nov 23 '18 at 9:20
  • I added an example with just a little modification to your answer achieving what I wanted. The question is if I could avoid defining 4 new coordinates. – gvgramazio Nov 23 '18 at 10:29
  • @gvgramazio I added something that does not introduce new coordinates. Introducing auxiliary coordinates is, however, IMHO not a bad thing. – marmot Nov 23 '18 at 11:12
5

You can remove the blank spaces in the middle by declaring (ael) and (aer) as \coordinate.

Currently Reward and State are same node in the question. Shift the State node left to get some separation between them. This can be done with positioning library:

\coordinate [left=1cm of ael] (aell) {}

Note that I have used left=1cm of ael not left of=. As @marmot mentioned, writing left of= is not using the loaded library positioning. This can reduce the number of nodes required also.

enter image description here

\documentclass[border=3mm]{standalone}
\usepackage[utf8]{inputenc}
\usepackage{tikz}
\usetikzlibrary{arrows,shadows,positioning}

\tikzset{
  frame/.style={
    rectangle, draw,
    text width=6em, text centered,
    minimum height=4em,drop shadow,fill=white,
    rounded corners,
  },
  line/.style={
    draw, -latex',rounded corners=3mm,
  }
}

\begin{document}
\begin{tikzpicture}[font=\small\sffamily\bfseries,very thick,node distance = 2cm]
\node [frame] (agent) {Agent};
\coordinate [below right=of agent] (aer) {};
\coordinate [below left=of agent] (ael) {};
\coordinate [left=0.9 cm of ael] (aell) {};
\node [frame, below left=of aer] (environment) {Environment};

\path [line] (agent) -| node[right, pos=1,align=left] {action\\ $A_t$} (aer) |- (environment);
\path [line] (environment.160) -| node[right,pos=1,align=left] {reward\\ $R_t$} (ael) |- (agent.200);
\path [line] (environment.200) -| node[left,pos=1,align=right]  {state\\ $S_t$} (aell) |- (agent.160);

\draw[-latex] (environment.160) -- ++(-1,0) node[above,midway]{$R_{i+1}$} coordinate(c1);
\draw[-latex] (environment.200) -- ++(-1,0) node[above,midway]{$S_{i+1}$} coordinate(c2);
\draw[dashed,shorten >=-3mm,shorten <=-3mm] (c1) -- (c2);
\end{tikzpicture}
\end{document}
  • It might be worthwhile to point out that in your update you also switched to the positioning syntax. – marmot Nov 22 '18 at 18:38

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.