tikz unruly tree

Question

Please forgive the very specific question that may not help many others. I need a kind of decision tree drawn. I am struggling with control.

I started with a dot (graphviz) tree. After a lot of tinkering, I got it almost to where I wanted it to be...

except that a bug in graphviz messes up the placement of the arrows on the far right going into the cascade box: C3R4 should be between C3R3 and C3R5, not above C3R1. (C2R1 should be higher up, too.) [please ignore box color.] graphviz was so close, but no cigar.

after experimenting with converting it into tikz via dot2tex, I gave up and decided to go native with tikz instead.

as usual, I tried to find a similar example on the web and modify it. my best result was

\documentclass[border=30pt]{standalone}

\usepackage{charter}
\usepackage{tikz}
\usetikzlibrary{arrows}

\tikzset{
  treenode/.style = {shape=rectangle, rounded corners, draw, align=center,
                     top color=white, bottom color=blue!20},
  root/.style     = {treenode, font=\Large, bottom color=red!30},
  fancy/.style    = {circle,draw},
  prect/.style    = {rectangle,draw},
  dot/.style      = {draw},
}


\begin{document}

\begin{tikzpicture}
  [
    grow                    = right,
    sibling distance        = 10em,
    level distance          = 10em,
    edge from parent/.style = {draw, -latex},
    % every node/.style       = {font=\footnotesize},
    sloped  %% for the labels, according to angle of link
  ]

  \node [root] {START}
  child {
    node [fancy] {R11}
    edge from parent node [below] {$\ell1$}
  }
    % 
  child {
    node [fancy] {A11}
    child {
      node [prect] {coin}
      child {
        node [fancy] {R33}
        child { node {A4L5} } % edge from parent node [above] {$h2XA5$}
        child { node {A4L6} } % edge from parent node [above] {$h2XA6$}
        edge from parent node [below] {1/2}
      }
      child {
        node [fancy] {A32}
        child {
          node {A4X1} } % edge from parent node [above] {$h35$}
        child {
          node {A4X2} }  % edge from parent node [above] {$h36$}
        edge from parent node [above] {1/2}
      }
      edge from parent node [below] {$\ell2$}
    }
%
    child {
      node [dot] {}
      child {
        node [fancy] {A31}
        child { node {A41}  }
        child { node {A42} }
      }
      edge from parent node [above] {h2}
    }
    edge from parent node [above] {$h1$}
  };
\end{tikzpicture}

\end{document}

this got me to

not even a cigarette now

1. I need the little rectangle between C1R1 and C2R1 to disappear (and C2R1 needs to remain above C2R2);
2. I need C2R1 to be higher up (and still link to C3R1 and C3R2);
3. I need C2R2 to continue to link to C3R3 and C3R4 nearby;
4. I obviously don't want C3R6 and C3R4 to share a slot.
5. I have not even gotten to attempting to draw the dashed box with text around C3R1 through C3R4.

I tried sibling angle=xx on some child nodes, but this seemed to be ignored. (level distance worked.) I also have not gotten to visual fine-tuning yet, which sibling angle and level distance could help with.

Nuisance: how do replace the circles with (fatter) ovals?

advice highly appreciated...

Answer 1

My approach using tikz

\documentclass[border=5pt]{standalone}

\usepackage{tikz}
\usetikzlibrary{calc,fit,shapes,backgrounds}
\usepackage{makecell}

\tikzset{%
    every node/.style={font=\small},
}
\tikzstyle{arrowline}=[-latex, shorten >=2pt, shorten <=2pt]
\tikzstyle{elor}=[ellipse, minimum width=20pt, minimum height=10pt, fill=orange!50!red!50!white]
\tikzstyle{elwh}=[draw, fill=white, ellipse, minimum width=20pt, minimum height=10pt]
\tikzstyle{elcy}=[ellipse, minimum width=20pt, minimum height=10pt, fill=cyan!25!white]
\tikzstyle{cicy}=[fill=cyan!25!white, circle, inner sep=5pt]

\begin{document}
    \begin{tikzpicture}
        
        %nodes
        
        \node (a1) [cicy] at (0,0) {};
        \node (b1) [elwh, anchor=west] at ($ (a1.east) + (2,1) $) {C1R1};
        \node (b2) [elwh, anchor=west] at ($ (a1.east) + (2,-1) $) {C1R2};
        \node (c1) [draw, minimum width=40pt, anchor=west] at ($ (b1.east)!0.5!(b2.east) + (2,0) $) {\makecell[c]{coin \\ flip}};
        \node (d2) [elwh, anchor=west] at ($ (c1.east) + (2,0) $) {C2R2};
        \node (d1) [elwh] at ($ (d2) + (0,3) $) {C2R1};
        \node (d3) [elcy] at ($ (d2) + (0,-3) $) {C2R3};
        \node (e1) [elor] at ($ (d1.east) + (2,2) $)  {C3R1};
        \node (e2) [elor] at ($ (e1) + (0,-2) $)  {C3R2};
        \node (e3) [elor] at ($ (e2) + (0,-2) $) {C3R3};
        \node (e4) [elor] at ($ (e3) + (0,-2) $)  {C3R4};
        \node (e5) [elwh] at ($ (e4) + (0,-2) $) {C3R5};
        \node (e6) [elwh] at($ (e5) + (0,-2) $) {C3R6};
        
        % arrows
        
        \draw[arrowline] (a1) edge node [pos=0.5, above] {h1} (b1);
        \draw[arrowline] (a1) edge node [pos=0.5, below] {l1} (b2);
        \draw[arrowline] (b1) edge node [pos=0.5, above] {l2} (c1);
        \draw[arrowline] (b1) edge node [pos=0.5, above] {h2} (d1);
        \draw[arrowline] (c1) edge node [pos=0.5, above] {1/2} (d2);
        \draw[arrowline] (c1) edge node [pos=0.5, above] {1/2} (d3);
        \draw[arrowline] (d1) edge node [pos=0.5, above] {h3} (e1);
        \draw[arrowline] (d1) edge node [pos=0.5, above] {l3} (e2);
        \draw[arrowline] (d2) edge node [pos=0.5, above] {h4} (e3); 
        \draw[arrowline] (d2) edge node [pos=0.5, above] {h4} (e4);
        \draw[arrowline] (d3) edge node [pos=0.5, above] {h5} (e5);
        \draw[arrowline] (d3) edge node [pos=0.5, above] {l5} (e6);
        
        \begin{pgfonlayer}{background}
            \node[fit=(e1)(e4), densely dashed, draw=blue, inner sep=5pt, label=above:cascade, rounded corners=5pt] {};
        \end{pgfonlayer}
    
    \end{tikzpicture}
\end{document}

Answer 2

When something is a tree, use forest. :) Oval styles from Excelsior's answer. Edge label code from here.

Reading your question carefully, it seems that the tree that you produced in GraphViz is not actually how you want the tree to look. If that's the case, here's a more regular version of the tree without the crossed branch and misplaced C3R4 node.

\documentclass{article}
\usepackage{forest}
\forestset{start/.style={draw, shape=rectangle, rounded corners, align=center,
                     top color=white, bottom color=blue!20, content=START},
           ell/.style={ellipse, minimum width=20pt, minimum height=10pt,draw,},
           orangefill/.style={fill=orange!50!red!50!white},
           bluefill/.style={fill=cyan!30!white},
           % edge label code from https://tex.stackexchange.com/a/425271/2693
           my edge label/.style={
    if={
      > O_= {n'}{1}
    }{
      edge label={node [midway, above] {$#1$} }
    }{
      edge label={node [midway, below,] {$#1$} }
    },
  },
  my tree/.style={where n children=0{tier=leaf}{},
    for tree={s sep=.5in,l=1in,grow=east, edge={->,thick}},
    before typesetting nodes={
      for tree={
        split option={content}{:}{content, my edge label},
      },
    }}}
\begin{document}
\begin{forest}my tree
      [,start [C1R2:l1,ell,tier=two,]
        [C1R1:h1,ell,
            [{coin flip}:l2,draw 
                [C2R3:1/2,ell,bluefill,tier=three
                     [C3R6:l5,ell] 
                     [C3R5:h5,ell]] 
                [C2R2:1/2,ell,name=C2R2,tier=three
                    [C3R4:l4,ell,orangefill,name=C3R4]
                    [C3R3:h4,ell,orangefill ]]] 
             [C2R1:h2,ell,tier=three
                [C3R2:l3,ell,orangefill]
                [C3R1:h3,ell,orangefill,name=C3R1]
            ]]]
\node[draw=blue,very thick,dashed,rounded corners, fit=(C3R1) (C3R4),label=above:cascade] {};
\end{forest}
\end{document}

Alternate version

In case you (or others) want to see how to reproduce the GraphViz picture, here's that code too:

\documentclass{article}
\usepackage{forest}
\forestset{start/.style={draw, shape=rectangle, rounded corners, align=center,
                     top color=white, bottom color=blue!20, content=START},
           ell/.style={ellipse, minimum width=20pt, minimum height=10pt,draw,},
           orangefill/.style={fill=orange!50!red!50!white},
           bluefill/.style={fill=cyan!30!white},
           % edge label code from https://tex.stackexchange.com/a/425271/2693
           my edge label/.style={
    if={
      > O_= {n'}{1}
    }{
      edge label={node [midway, above] {$#1$} }
    }{
      edge label={node [midway, below,] {$#1$} }
    },
  },
  my tree/.style={where n children=0{tier=leaf}{},
    for tree={s sep=.5in,l=1in,grow=east, edge={->,thick}},
    before typesetting nodes={
      for tree={
        split option={content}{:}{content, my edge label},
      },
    }}}
\begin{document}
\begin{forest}my tree
      [,start 
        [C1R2:l1,ell,tier=two,]
        [C1R1:h1,ell,
            [{coin flip}:l2,draw 
                [C2R3:1/2,ell,bluefill,tier=three
                     [C3R6:l5,ell] 
                     [C3R5:h5,ell]] 
                [C2R2:1/2,ell,name=C2R2,tier=three
                    [C3R3:h4,ell,orangefill,name=C3R3 ]]]
             [C2R1:h2,ell,tier=three
                [C3R2:l3,ell,orangefill]
                [C3R1:h3,ell,orangefill]
                [C3R4,ell,orangefill,no edge,name=C3R4]
            ]]]
\draw[->,thick] (C2R2.north) [in=180,out=60,] to node [near start,left] {$l4$} (C3R4.west);
\node[draw=blue,very thick,dashed,rounded corners, fit=(C3R3) (C3R4),label=above:cascade] {};
\end{forest}
\end{document}