Hobby path realization in convex hull approach

Question

Motivation

In the answer Highlight a group of nodes in a tikz tree, Jake suggested combining the convex hull approach from padded boundary of convex hull with the hobby path and I was really intrigued by the possibility.

Preliminary work

At first I tried to modify at least as possible the \convexpath:

\documentclass[a4paper,11pt]{article}
\usepackage{tikz}
\usetikzlibrary{hobby,backgrounds,calc,trees}

\newcommand{\myconvexpath}[2]{
[   
    create hobbyhullnodes/.code={
        \global\edef\namelist{#1}
        \foreach [count=\counter] \nodename in \namelist {
            \global\edef\numberofnodes{\counter}
            \node at (\nodename) [draw=none,name=hobbyhullnode\counter] {};
        }
        \node at (hobbyhullnode\numberofnodes) [name=hobbyhullnode0,draw=none] {};
        \pgfmathtruncatemacro\lastnumber{\numberofnodes+1}
        \node at (hobbyhullnode1) [name=hobbyhullnode\lastnumber,draw=none] {};
    },
    create hobbyhullnodes
]
($(hobbyhullnode1)!#2!-90:(hobbyhullnode0)$)
\foreach [
    evaluate=\currentnode as \previousnode using \currentnode-1,
    evaluate=\currentnode as \nextnode using \currentnode+1
    ] \currentnode in {1,...,\numberofnodes} {
   let \p1 = ($(hobbyhullnode\currentnode)!#2!-90:(hobbyhullnode\previousnode) - (hobbyhullnode\currentnode)$),
    \n1 = {atan2(\x1,\y1)},
    \p2 = ($(hobbyhullnode\currentnode)!#2!90:(hobbyhullnode\nextnode) - (hobbyhullnode\currentnode)$),
    \n2 = {atan2(\x2,\y2)},
    \n{delta} = {-Mod(\n1-\n2,360)}
  in 
    {arc [start angle=\n1, delta angle=\n{delta}, radius=#2]}
     ..($(hobbyhullnode\nextnode)!0.5!(hobbyhullnode\currentnode)$)
     ..($(hobbyhullnode\nextnode)!#2!-90:(hobbyhullnode\currentnode)$)
}
--cycle
}

\begin{document}
\begin{tikzpicture}[use Hobby shortcut]
\node (f) {f}
    child { node (g) {g}
      child { node (a) {a}
    }
      child { node (b) {b}
    }
  }
    child { node (h) {h}
      child { node (c) {c}
    }
  };
  \begin{pgfonlayer}{background}
  \fill[draw,blue, opacity=0.3] \myconvexpath{f,h,c,g}{12pt};
  \fill[draw,red, opacity=0.3] \myconvexpath{g,b,a}{12pt};
  \end{pgfonlayer}
  \end{tikzpicture}

\end{document}

which leads to:

I suspected the combination of arcs with the hobby path was the cause of cusps, so in another example I tried with:

\documentclass[a4paper,11pt]{article}
\usepackage{tikz}
\usetikzlibrary{hobby,backgrounds,calc,trees}

\newcommand{\myconvexpath}[2]{
[   
    create hobbyhullnodes/.code={
        \global\edef\namelist{#1}
        \foreach [count=\counter] \nodename in \namelist {
            \global\edef\numberofnodes{\counter}
            \node at (\nodename) [draw=none,name=hobbyhullnode\counter] {};
        }
        \node at (hobbyhullnode\numberofnodes) [name=hobbyhullnode0,draw=none] {};
        \pgfmathtruncatemacro\lastnumber{\numberofnodes+1}
        \node at (hobbyhullnode1) [name=hobbyhullnode\lastnumber,draw=none] {};
    },
    create hobbyhullnodes
]
($(hobbyhullnode1)!#2!-90:(hobbyhullnode0)$)
\foreach [
    evaluate=\currentnode as \previousnode using \currentnode-1,
    evaluate=\currentnode as \nextnode using \currentnode+1
    ] \currentnode in {1,...,\numberofnodes} {
   let \p1 = ($(hobbyhullnode\currentnode)!#2!-90:(hobbyhullnode\previousnode)$),
    \n1 = {atan2(\x1,\y1)},
    \p2 = ($(hobbyhullnode\currentnode)!#2!90:(hobbyhullnode\nextnode)$),
    \n2 = {atan2(\x2,\y2)},
    \n{delta} = {-Mod(\n1-\n2,360)},
    \n{end}={add(\n1,\n{delta})}
  in 
    {..([in angle=\n1]$(hobbyhullnode\currentnode)!#2!-90:(hobbyhullnode\previousnode)$)..([out angle=\n{end}]$(hobbyhullnode\currentnode)!#2!90:(hobbyhullnode\nextnode)$)}
     ..($(hobbyhullnode\nextnode)!0.5!(hobbyhullnode\currentnode)$)
     ..($(hobbyhullnode\nextnode)!#2!-90:(hobbyhullnode\currentnode)$)
}
--cycle
}

\begin{document}
\begin{tikzpicture}[use Hobby shortcut]
\node (f) {f}
    child { node (g) {g}
      child { node (a) {a}
    }
      child { node (b) {b}
    }
  }
    child { node (h) {h}
      child { node (c) {c}
    }
  };
  \begin{pgfonlayer}{background}
  \fill[draw,blue, opacity=0.3] \myconvexpath{f,h,c,g}{12pt};
  \fill[draw,red, opacity=0.3] \myconvexpath{g,b,a}{12pt};
  \end{pgfonlayer}
  \end{tikzpicture}

\end{document}

that gives a not promising result:

Question

Is there a way to automatically recognize the node angle a path will fall when arrives near it? Doing things by hand, one can force a path to follow the desired direction, for example, h.north -> h.east -> h.south, but how is it possible to do it automatically without the arc syntax?

Notice that, for some shapes, one could proceed as follows:

\documentclass[a4paper,11pt]{article}
\usepackage{tikz}
\usetikzlibrary{hobby,backgrounds,calc,trees}

\newcommand{\hobbyconvexpath}[2]{
[   
    create hobbyhullnodes/.code={
        \global\edef\namelist{#1}
        \foreach [count=\counter] \nodename in \namelist {
            \global\edef\numberofnodes{\counter}
            \node at (\nodename) [draw=none,name=hobbyhullnode\counter] {};
        }
        \node at (hobbyhullnode\numberofnodes) [name=hobbyhullnode0,draw=none] {};
        \pgfmathtruncatemacro\lastnumber{\numberofnodes+1}
        \node at (hobbyhullnode1) [name=hobbyhullnode\lastnumber,draw=none] {};
    },
    create hobbyhullnodes
]
($(hobbyhullnode1)!#2!-40:(hobbyhullnode0)$)
\foreach [
    evaluate=\currentnode as \previousnode using \currentnode-1,
    evaluate=\currentnode as \nextnode using \currentnode+1
    ] \currentnode in {1,...,\numberofnodes} {
  let \p1 = ($(hobbyhullnode\currentnode)!#2!-90:(hobbyhullnode\previousnode) $),
    \n1 = {atan2(\x1,\y1)},
    \p2 = ($(hobbyhullnode\currentnode)!#2!-90:(hobbyhullnode\nextnode)$),
    \n2 = {atan2(\x2,\y2)},
    \n{delta} = {-Mod(\n1-\n2,360)},
    \n{fin}={add(\n1,\n{delta})}
  in 
      {..($(hobbyhullnode\currentnode)!#2!-220:(hobbyhullnode\previousnode)$)..($(hobbyhullnode\currentnode)!#2!40:(hobbyhullnode\nextnode)$)}
   %{arc [start angle=\n1, end angle=\n{fin}, radius=#2]}
     ..($(hobbyhullnode\nextnode)!0.5!(hobbyhullnode\currentnode)$)
     ..($(hobbyhullnode\nextnode)!#2!-40:(hobbyhullnode\currentnode)$)
}
--cycle
}

\begin{document}

\begin{tikzpicture}[use Hobby shortcut]
  \foreach \place/\text in {{(1,0)/a},{(0,-1)/b},{(-1,0)/c},{(0,1)/d}}
  \node[name=\text] at \place {\text};
  \begin{pgfonlayer}{background}
  \fill[draw,green, opacity=0.3] \hobbyconvexpath{a,b,c,d}{10pt};
  \end{pgfonlayer}
\end{tikzpicture}

\end{document}

but in general is not a valid approach and it is still to improve, to get at least the same result of Highlight a group of nodes in a tikz tree.

Answer 1

I'm answering because I think there's a quite good solution on the problem. First of all I'd like to thank Andrew Stacey because without the discussion in chat I wouldn't be able to solve this and, more important, he founds the major issue in the code.

---

What I learnt

The answer to the question:

Is there a way to automatically recognize the node angle a path will fall when arrives near it?

is actually: TikZ already does it automatically. Notice indeed that:

\documentclass{article}
\usepackage{tikz}
\begin{document}
\begin{tikzpicture}
\node[circle,fill=red] (a) at (1,1) {}; 
\node[circle,fill=blue] (b) at  (3,2) {};
\draw[color=black] (a) -- (b);
\end{tikzpicture}
\end{document}

correctly leads to:

and the user should not have to care about the angle the line arrives near the blue node.

The major issue

Building on that, however, was not sufficient. Andrew recognized that the paths drawn were not correctly realized due to the fact that the hobby path shortcut construct separately each piece; this was the cause of the bizarre behaviour as per the second figure displayed in the question.

The intermediate result

With his help it was possible to get (code here):

and notice how this solved the problems mentioned before.

A final result

Starting from the intermediate result I noticed that actually the non-perfect fit around some nodes was basically due to the out angle, that is the the path from hypothetically ($(hobbyhullnode1)!10pt!-90:(hobbyhullnode0)$)..($(hobbyhullnode1)!10pt!90:(hobbyhullnode0)$). With a bit of care, the right angle should have been set to 180 rather than 90.

Here is a MWE:

\documentclass[tikz,border=2bp]{standalone}
\usetikzlibrary{backgrounds,calc,trees,hobby}

\pgfdeclarelayer{background}
\pgfsetlayers{background,main}

\newcommand{\hobbyconvexpath}[2]{
[   
    create hobbyhullnodes/.code={
        \global\edef\namelist{#1}
        \foreach [count=\counter] \nodename in \namelist {
            \global\edef\numberofnodes{\counter}
            \node at (\nodename)
[draw=none,name=hobbyhullnode\counter] {};
        }
        \node at (hobbyhullnode\numberofnodes)
[name=hobbyhullnode0,draw=none] {};
        \pgfmathtruncatemacro\lastnumber{\numberofnodes+1}
        \node at (hobbyhullnode1)
[name=hobbyhullnode\lastnumber,draw=none] {};
    },
    create hobbyhullnodes
]
($(hobbyhullnode1)!#2!-90:(hobbyhullnode0)$)
\pgfextra{
  \gdef\hullpath{}
\foreach [
    evaluate=\currentnode as \previousnode using \currentnode-1,
    evaluate=\currentnode as \nextnode using \currentnode+1
    ] \currentnode in {1,...,\numberofnodes} {
    \pgfmathtruncatemacro\thecurrentnode\currentnode
    \pgfmathtruncatemacro\thepreviousnode\previousnode
    \pgfmathtruncatemacro\thenextnode\nextnode
    \xdef\hullpath{\hullpath    
  ..($(hobbyhullnode\thecurrentnode)!#2!180:(hobbyhullnode\thepreviousnode)$)
  ..($(hobbyhullnode\thenextnode)!0.5!(hobbyhullnode\thecurrentnode)$)}
    \ifx\currentnode\numberofnodes
    \xdef\hullpath{\hullpath .. cycle}
    \else
    \xdef\hullpath{\hullpath
  ..($(hobbyhullnode\thenextnode)!#2!-90:(hobbyhullnode\thecurrentnode)$)}
    \fi
}
}
\hullpath
}

\begin{document}
\begin{tikzpicture}[use Hobby shortcut,scale=3,transform shape]
\node (f) {f}
    child { node (g) {g} 
      child { node (a) {a}
    }
      child { node (b) {b}
    }
  }
    child { node (h) {h}
      child { node (c) {c}
      }
  };

\begin{pgfonlayer}{background}
\fill[red,opacity=0.3] \hobbyconvexpath{a,g,b}{10pt};
\end{pgfonlayer}

\draw[blue,dashed]($(hobbyhullnode1)!10pt!-90:(hobbyhullnode0)$)--
($(hobbyhullnode1)!10pt!-90:(hobbyhullnode0)$)..($(hobbyhullnode1)!10pt!180:(hobbyhullnode0)$)..
($(hobbyhullnode2)!0.5!(hobbyhullnode1)$) ..
($(hobbyhullnode2)!10pt!-90:(hobbyhullnode1)$)..($(hobbyhullnode2)!10pt!180:(hobbyhullnode1)$)..($(hobbyhullnode3)!0.5!(hobbyhullnode2)$)..
($(hobbyhullnode3)!10pt!-90:(hobbyhullnode2)$)..($(hobbyhullnode3)!10pt!180:(hobbyhullnode2)$)
..($(hobbyhullnode4)!0.5!(hobbyhullnode3)$) .. cycle;

\begin{pgfonlayer}{background}
\fill[green!50!lime,opacity=0.4] \hobbyconvexpath{g,f,h,c}{10pt};
\end{pgfonlayer}

\draw[blue,dashed]($(hobbyhullnode1)!10pt!-90:(hobbyhullnode0)$)--
($(hobbyhullnode1)!10pt!-90:(hobbyhullnode0)$)..($(hobbyhullnode1)!10pt!180:(hobbyhullnode0)$)..
($(hobbyhullnode2)!0.5!(hobbyhullnode1)$) ..
($(hobbyhullnode2)!10pt!-90:(hobbyhullnode1)$)..($(hobbyhullnode2)!10pt!180:(hobbyhullnode1)$)..($(hobbyhullnode3)!0.5!(hobbyhullnode2)$)..
($(hobbyhullnode3)!10pt!-90:(hobbyhullnode2)$)..($(hobbyhullnode3)!10pt!180:(hobbyhullnode2)$)
..($(hobbyhullnode4)!0.5!(hobbyhullnode3)$) ..
($(hobbyhullnode4)!10pt!-90:(hobbyhullnode3)$)..($(hobbyhullnode4)!10pt!180:(hobbyhullnode3)$)
..($(hobbyhullnode5)!0.5!(hobbyhullnode4)$) .. cycle;
\end{tikzpicture}

\begin{tikzpicture}[use Hobby shortcut,scale=3,transform shape]
\node (f) {f}
    child { node (g) {g} 
      child { node (a) {a}
    }
      child { node (b) {b}
    }
  }
    child { node (h) {h}
      child { node (c) {c}
      }
  };

\begin{pgfonlayer}{background}
\fill[orange,opacity=0.4] \hobbyconvexpath{a,g,h,b}{13pt};
\end{pgfonlayer}
\end{tikzpicture}

\begin{tikzpicture}[use Hobby shortcut,scale=3,transform shape]
\node (f) {f}
    child { node (g) {g} 
      child { node (a) {a}
    }
      child { node (b) {b}
    }
  }
    child { node (h) {h}
      child { node (c) {c}
      }
  };

\begin{pgfonlayer}{background}
\fill[cyan,opacity=0.4] \hobbyconvexpath{b,g,f,h}{10pt};
\end{pgfonlayer}
\end{tikzpicture}
\end{document}

with the following result:

Another example (since I noticed that the shape highlighted is actually the same in all the three previous figures):

\begin{tikzpicture}[use Hobby shortcut,scale=3,transform shape]
\node (f) {f}
    child { node (g) {g} 
      child { node (a) {a}
        child { node (i) {i}}
    }
      child { node (b) {b}
            child { node (d) {d}}
            child { node (e) {e}}
    }
  }
    child { node (h) {h}
      child { node (c) {c}
      }
  };

\begin{pgfonlayer}{background}
\fill[cyan,opacity=0.4] \hobbyconvexpath{a,g,f,b,e,d}{10pt};
\end{pgfonlayer}
\end{tikzpicture}