How to draw squares and circles in my decision tree?

Question

I'd like to create a decision tree like the one shown below.

\tikzstyle{decision} = [rectangle, minimum height=18pt, minimum width=18pt, draw=blue, fill=none, ultra thick, inner sep=0pt]
\tikzstyle{chance} = [circle, minimum width=18pt, draw=blue, fill=none, ultra thick, inner sep=0pt]
\tikzstyle{line} = [draw=none]

Answer 1

You can choose the shape of each individual child.

There's much, much more you can do though, including not needing to hardcode. I don't know much about TikZ trees / mindmaps, but there's a tutorial in the TikZ documentation, examples on TeXample, plenty of packages to draw trees (e.g. forest or tikz-qtree, among others), and the trees tag on this website.

\documentclass[border=1cm]{standalone}

\usepackage{tikz}
\usetikzlibrary{shapes}

\begin{document}

\begin{tikzpicture}[%
level 1/.style={sibling distance=5cm},
level 2/.style={sibling distance=3cm},
every node/.style = {draw, minimum width=1.5cm, minimum height=.75cm, anchor=north},
edge from parent path={(\tikzparentnode.south) -- (\tikzchildnode.north)}]
]
\node[shape=ellipse, minimum height=1.25cm, minimum width=2cm] {R}
child { node[shape=rectangle, rounded corners] {A}
    child { node[shape=rectangle] {A} 
        child { node[shape=ellipse] {B} }
        child { node[shape=ellipse] {B} }
    }
    child { node[shape=ellipse] {B} }
}
child { node[shape=rectangle, rounded corners] {A} 
    child { node[shape=ellipse] {B} }
    child { node[shape=ellipse] {B} }
};
\end{tikzpicture}

\end{document}

Using the tikzstyles you defined:

\documentclass[border=1cm]{standalone}

\usepackage{tikz}

\tikzstyle{decision} = [rectangle, minimum height=18pt, minimum width=18pt, draw=blue, fill=none, ultra thick, inner sep=0pt]
\tikzstyle{chance} = [circle, minimum width=18pt, draw=blue, fill=none, ultra thick, inner sep=0pt]
\tikzstyle{line} = [draw=none]

\begin{document}

\begin{tikzpicture}[%
level 1/.style={sibling distance=5cm},
level 2/.style={sibling distance=3cm},
every node/.style = {draw},
edge from parent path={(\tikzparentnode.south) -- (\tikzchildnode.north)}]
]
\node[chance] {R}
child { node[decision] {A}
    child { node[decision] {A} 
        child { node[chance] {B} }
        child { node[chance] {B} }
    }
    child { node[chance] {B} }
}
child[decision] { node {A} 
    child { node[chance] {B} }
    child { node[chance] {B} }
};
\end{tikzpicture}

\end{document}