Help to reproduce a diagram in TikZ

Question

This is what I want to achieve:

This is what I have done so far:

This is the code to reproduce my image. I was inspired by this answer:

\documentclass{standalone}
\usepackage[utf8]{inputenc}
\usepackage{tikz}
\usetikzlibrary{arrows,shadows,positioning}

\tikzset{
  frame/.style={
    rectangle, draw,
    text width=6em, text centered,
    minimum height=4em,drop shadow,fill=white,
    rounded corners,
  },
  line/.style={
    draw, -latex',rounded corners=3mm,
  }
}

\begin{document}
\begin{tikzpicture}[font=\small\sffamily\bfseries,very thick,node distance = 4cm]
\node [frame] (agent) {Agent};
\node [below right of=agent] (aer) {};
\node [below left of=agent] (ael) {};
\node [frame, below left of=aer] (environment) {Environment};

\path [line] (agent)
  -| node[right, pos=1, align=left] {action\\ $A_t$} (aer)
  |- (environment);
\path [line] (environment.170)
  -| node[right, pos=1, align=left] {reward\\ $R_t$} (ael)
  |- (agent.190);
\path [line] (environment.190)
  -| node[left, pos=1, align=right] {state\\ $s_t$} (ael)
  |- (agent.170);
\end{tikzpicture}
\end{document}

---

I don't want a perfect result. I'm interested in maintaining the reward of state and reward separated and I don't want that blank space at the center of each path. Maybe a better way to start/end paths would be appreciated: currently, it's done with an angle but I don't know which is the actual distance between lines.

---

This is how I modified marmot's answer to have the same distance between state and reward lines both horizontally and vertically.

\documentclass{standalone}
\usepackage[utf8]{inputenc}
\usepackage{tikz}
\usetikzlibrary{arrows.meta,shadows,positioning}

\tikzset{
  frame/.style={
    rectangle, draw,
    text width=6em, text centered,
    minimum height=4em,drop shadow,fill=white,
    rounded corners,
  },
  line/.style={
    draw, -{Latex},rounded corners=3mm,
  }
}

\begin{document}
\begin{tikzpicture}[font=\small\sffamily\bfseries,very thick,node distance = 4cm]
\node [frame] (agent) {Agent};
\node [frame, below=1.2cm of agent] (environment) {Environment};
\draw[line] (agent) -- ++ (3.5,0) |- (environment)
node[right,pos=0.25,align=left] {action\\ $A_t$};
\coordinate[left=8mm of environment] (P);
\coordinate[above=3mm of environment.west] (ENW);
\coordinate[below=3mm of environment.west] (ESW);
\coordinate[above=3mm of agent.west] (ANW);
\coordinate[below=3mm of agent.west] (ASW);
\draw[thin,dashed] (P|-environment.north) -- (P|-environment.south);
\draw[line] (ESW) -- (P |- ESW)
node[midway,above]{$S_{i+1}$};
\draw[line,thick] (ENW) -- (P |- ENW)
node[midway,above]{$R_{i+1}$};
\draw[line] (P |- ESW) -- ++ (-1.4,0) |- (ANW)
node[left, pos=0.25, align=right] {state\\ $s_t$};
\draw[line,thick] (P |- ENW) -- ++ (-0.8,0) |- (ASW)
node[right,pos=0.25,align=left] {reward\\ $R_t$};
\end{tikzpicture}
\end{document}

And this is the result.

This is already the visual result that I wanted to achieve. The question is if I could have avoided to define four new coordinates (ASW,ANW,ESW,ENW).

Answer 1

You were loading but not using the positioning library, that is instead of right of=, say, please use right=of or right=5mm of. And one can kick out several auxiliary nodes/coordinates like (ael) and (aer), they are not at all needed.

\documentclass{standalone}
\usepackage[utf8]{inputenc}
\usepackage{tikz}
\usetikzlibrary{arrows.meta,shadows,positioning}

\tikzset{
  frame/.style={
    rectangle, draw,
    text width=6em, text centered,
    minimum height=4em,drop shadow,fill=white,
    rounded corners,
  },
  line/.style={
    draw, -{Latex},rounded corners=3mm,
  }
}

\begin{document}
\begin{tikzpicture}[font=\small\sffamily\bfseries,very thick,node distance = 4cm]
\node [frame] (agent) {Agent};
\node [frame, below=1.2cm of agent] (environment) {Environment};
\draw[line] (environment) -- ++ (3.5,0) |- (agent) 
node[right,pos=0.25,align=left] {action\\ $A_t$};
\coordinate[left=8mm of environment] (P);
\draw[thin,dashed] (P|-environment.north) -- (P|-environment.south);
\draw[line] (environment.200) -- (P |- environment.200)
node[midway,above]{$S_{i+1}$};
\draw[line,thick] (environment.160) -- (P |- environment.160)
node[midway,above]{$R_{i+1}$};
\draw[line] (P |- environment.200) -- ++ (-1.4,0) |- (agent.160)
node[left, pos=0.25, align=right] {state\\ $s_t$};
\draw[line,thick] (P |- environment.160) -- ++ (-0.8,0) |- (agent.200)
node[right,pos=0.25,align=left] {reward\\ $R_t$};
\end{tikzpicture}
\end{document}

ADDENDUM: If you want to adjust the distances of the two lines, there is no need to introduce new coordinat (However, your way to achieve this with additional coordinates is very elegant, I think. Just in the original question the coordinates (ael) and (aer) were not necessarily a good choice as they interrupted paths. In principle one could avoid introducing P in the following by switching to decorations.markings, but this might be an overkill.)

\documentclass{standalone}
\usepackage[utf8]{inputenc}
\usepackage{tikz}
\usetikzlibrary{arrows.meta,shadows,positioning}

\tikzset{
  frame/.style={
    rectangle, draw,
    text width=6em, text centered,
    minimum height=4em,drop shadow,fill=white,
    rounded corners,
  },
  line/.style={
    draw, -{Latex},rounded corners=3mm,
  }
}

\begin{document}
\begin{tikzpicture}[font=\small\sffamily\bfseries,very thick,node distance = 4cm]
\node [frame] (agent) {Agent};
\node [frame, below=1.2cm of agent] (environment) {Environment};
\draw[line] (environment) -- ++ (3.5,0) |- (agent) 
node[right,pos=0.25,align=left] {action\\ $A_t$};
\coordinate[left=12mm of environment] (P);
\draw[thin,dashed] (P|-environment.north) -- (P|-environment.south);
\pgfmathsetmacro{\Ldist}{4mm}
\draw[line] ([yshift=-\Ldist]environment.west) -- 
([yshift=-\Ldist]environment.west -| P) node[midway,above]{$S_{i+1}$};
\draw[line,thick] ([yshift=\Ldist]environment.west) -- ([yshift=\Ldist]environment.west
-|P) node[midway,above]{$R_{i+1}$};
\draw[line] ([yshift=-\Ldist]environment.west -| P) -- ++ (-12mm-\Ldist,0) |- 
([yshift=\Ldist]agent.west) node[left, pos=0.25, align=right] {state\\ $S_t$};
\draw[line,thick] ([yshift=\Ldist]environment.west -| P) -- ++ (-12mm+\Ldist,0) 
|- ([yshift=-\Ldist]agent.west) node[right,pos=0.25,align=left] {reward\\ $R_t$};
\end{tikzpicture}
\end{document}

Answer 2

You can remove the blank spaces in the middle by declaring (ael) and (aer) as \coordinate.

Currently Reward and State are same node in the question. Shift the State node left to get some separation between them. This can be done with positioning library:

\coordinate [left=1cm of ael] (aell) {}

Note that I have used left=1cm of ael not left of=. As @marmot mentioned, writing left of= is not using the loaded library positioning. This can reduce the number of nodes required also.

\documentclass[border=3mm]{standalone}
\usepackage[utf8]{inputenc}
\usepackage{tikz}
\usetikzlibrary{arrows,shadows,positioning}

\tikzset{
  frame/.style={
    rectangle, draw,
    text width=6em, text centered,
    minimum height=4em,drop shadow,fill=white,
    rounded corners,
  },
  line/.style={
    draw, -latex',rounded corners=3mm,
  }
}

\begin{document}
\begin{tikzpicture}[font=\small\sffamily\bfseries,very thick,node distance = 2cm]
\node [frame] (agent) {Agent};
\coordinate [below right=of agent] (aer) {};
\coordinate [below left=of agent] (ael) {};
\coordinate [left=0.9 cm of ael] (aell) {};
\node [frame, below left=of aer] (environment) {Environment};

\path [line] (agent) -| node[right, pos=1,align=left] {action\\ $A_t$} (aer) |- (environment);
\path [line] (environment.160) -| node[right,pos=1,align=left] {reward\\ $R_t$} (ael) |- (agent.200);
\path [line] (environment.200) -| node[left,pos=1,align=right]  {state\\ $S_t$} (aell) |- (agent.160);

\draw[-latex] (environment.160) -- ++(-1,0) node[above,midway]{$R_{i+1}$} coordinate(c1);
\draw[-latex] (environment.200) -- ++(-1,0) node[above,midway]{$S_{i+1}$} coordinate(c2);
\draw[dashed,shorten >=-3mm,shorten <=-3mm] (c1) -- (c2);
\end{tikzpicture}
\end{document}