Styles in TikZ trees not working as expected

Question

I want to draw a "cut-out" region of a tree, and visually underline this by having dashed edges on top and on the bottom of my diagram (to symbolize the levels further up and further down).

I am attempting to use styles defined in the options of the \tikzpicture environment, but failing to achieve the desired result, which would look something like this:

For me, the "outermost" edges (which I want to be dashed) are just normal lines.

\documentclass{beamer}

\usepackage{tikz}

\begin{document}
\begin{tikzpicture}[
  every node/.style={minimum size=4mm, inner sep=0.5mm},
  normal/.style={circle,draw},
  invis/.style={draw=none},
  border/.style={ edge from parent/.style={dashed,draw} },
  acc/.style={circle,thick,draw=green!50,fill=green!2},
  rej/.style={circle,thick,draw=red!50,fill=red!20},
  semithick]

  node[invis] (root) {}
    child[level distance=11mm] { node[border,normal] {x}
      child {node[acc] {x1}
        child {node[border,invis] {} }
        child {node[border,invis] {} }
        child {node[border,invis] {} }
      }
      child {node[rej] {x2} }
      child {node[normal] {x3}
        child {node[invis,border] {} }
        child {node[invis,border] {} }
        child {node[invis,border] {} }
      }
  }; 
\end{tikzpicture}
\end{document}

My first hunch was maybe a node could only have one style defined in its options, but even after swapping the two styles for the last tree nodes, it makes no difference. I also tried applying a style and setting an option in the same []-block, and it worked, so this doesn't seem to be the problem.

I am aware that the author of TikZ notes in the manual that "placing node options at the right place is an arcane art", but (after carefully reading) I thought basically it worked like this:

• Options defined in a node are local in effect.
• Options defined in a child command are inherited to all of its child nodes.

When trying to pass the border style to the child command as an alternative , I do get dashed edges which work for the bottom, but if I pass it to the first visible child node (x), all the lines in the tree are dashed (even the node outlines!) because of the inheritance mechanism.

What am I doing wrong/ should I change to achieve the desired result?

Answer 1

Just a quick demonstration with Forest. The major advantages of Forest lie in its power, flexibility and concise syntax. Regularities in content and styling can be turned into automated configuration rules and trees themselves can be specified very concisely. Since Forest is based on TikZ, the power of the host package is also available.

For example,

\begin{forest}
  bordered tree,
  [
    [x, border
      [x1, acc, for descendants=border
        [][][]
      ]
      [x2, rej]
      [x3, for children=border
        [][][]
      ]
    ]
  ]
\end{forest}

specifies the target tree:

while the following partly explains the first and partly shows one or two additional tricks,

\begin{forest}
  bordered tree,
  [
    [x, border, normal
    [x1, acc, for descendants=border, label=left:\texttt{for descendants}
        [][.,label=left:bordered children [][]][]
      ]
      [x2, rej]
      [x3, normal, for children=border, label=right:\texttt{for children}
        [][.,label=right:unbordered children [][]][, for current and ancestors={edge+=blue}]
      ]
    ]
  ]
\end{forest}

which produces:

Complete code:

\documentclass[border=10pt]{standalone}
\usepackage{forest}
\forestset{
  declare boolean={border}{0},
  bordered tree/.style={
    for tree={
      minimum size=4mm, 
      inner sep=0.5mm,
      edge+={semithick},
      semithick,
    },
    before typesetting nodes={
      where border={
        edge+={dashed, draw},
      }{},
      where={isodd(n_children)}{
        tempcounta/.process={
          Ow+n {n children}{(##1+1)/2}
        },
        for n={
          >  R {tempcounta} % doesn't work to plug the above in directly ??'
        }{calign with current edge},
      }{},
      where content={}{}{normal},
    },
  },
  /tikz/.cd,
  normal/.style={circle,draw},
  invis/.style={draw=none},
  acc/.style={circle,thick,draw=green!50,fill=green!2},
  rej/.style={circle,thick,draw=red!50,fill=red!20},
}
\begin{document}
\begin{forest}
  bordered tree,
  [
    [x, border
      [x1, acc, for descendants=border
        [][][]
      ]
      [x2, rej]
      [x3, for children=border
        [][][]
      ]
    ]
  ]
\end{forest}
\begin{forest}
  bordered tree,
  [
    [x, border, normal
    [x1, acc, for descendants=border, label=left:\texttt{for descendants}
        [][.,label=left:bordered children [][]][]
      ]
      [x2, rej]
      [x3, normal, for children=border, label=right:\texttt{for children}
        [][.,label=right:unbordered children [][]][, for current and ancestors={edge+=blue}]
      ]
    ]
  ]
\end{forest}
\end{document}

Answer 2

The main problem is that your styles for the edges should be applied to the child commands not the node ones. This is what I show in the second solution. Probably better is styling the levels. I'll give this first.

Styling the levels

\documentclass{article}

\usepackage{tikz}

\begin{document}

\begin{tikzpicture}[
  every node/.style={minimum size=4mm, inner sep=0.5mm},
  level 1/.style={every child/.style={edge from parent/.style={draw,dashed}}},
  level 2/.style={every child/.style={edge from parent/.style={draw,solid}}},
  level 4/.style={every child/.style={edge from
  parent/.style={draw,dashed}},sibling distance=5mm},
  normal/.style={circle,draw,solid},
  acc/.style={circle,thick,solid,draw=green!50,fill=green!2},
  rej/.style={circle,thick,solid,draw=red!50,fill=red!20},
  semithick]

  \node (root) {}
    child[level distance=11mm] { node[normal] {x}
      child {node[acc] {x1}
        child { node[normal] {} child child}
        child { node[normal] {} child child}
      }
      child {node[rej] {x2} }
      child {node[normal] {x3}
        child {node[normal] {} child }
        child {node[acc] {} child child }
        child {node[normal] {} child child child}
      }
  }; 
\end{tikzpicture}

\end{document}

This assumes cut off at uniform depth. The style for level 2 is inherited by lower levels, so you specify styles for the first, second and last levels. If the cut off is not at uniform depth, you can instead write children on the last level as

child { edge from parent[dashed] }

producing a descending dashed line.

Style on child

\documentclass{article}

\usepackage{tikz}

\begin{document}

\begin{tikzpicture}[
  every node/.style={minimum size=4mm, inner sep=0.5mm},
  normal/.style={circle,draw},
  invis/.style={draw=none},
  border/.style={ edge from parent/.style={dashed,draw} },
  normaledge/.style={ edge from parent/.style={solid,draw}},
  acc/.style={circle,thick,draw=green!50,fill=green!2},
  rej/.style={circle,thick,draw=red!50,fill=red!20},
  semithick]

  \node[invis] (root) {}
    child[level distance=11mm,border] { node[normal] {x}
      child[normaledge] {node[acc] {x1}
        child[border] {node {} }
        child[border] {node {} }
        child[border] {node {} }
      }
      child[normaledge] {node[rej] {x2} }
      child[normaledge] {node[normal] {x3}
        child[border] {node[invis] {} }
        child[border] {node[invis] {} }
        child[border] {node[invis] {} }
      }
  }; 
\end{tikzpicture}

\end{document}

I have introduced a normaledge style for edges that are not borders.

Answer 3

Another solution to use the istgame package. The istgame environment is just like the tikzpicture environment, so you can use any macros and options that work in the tikzpicture environment.

\documentclass{standalone}

\usepackage{istgame}

\begin{document}   

\begin{istgame}[font=\footnotesize]
\setistEllipseNodeStyle{6mm}
\xtdistance{15mm}{15mm}
\istroot[north](0)
  \istb[dashed] \endist
\istrooto(0){x}
  \istb \istb \istb \endist
\xtdistance{15mm}{10mm}
\istrooto(1)(0-1){x1}
  \istb[dashed] \istb[dashed] \istb[dashed] \endist
\istrooto(2)(0-2){x2}
  \endist
\istrooto(3)(0-3){x3}
  \istb[dashed] \istb[dashed] \istb[dashed] \endist
\end{istgame}

\end{document}