Side effects of changing blackboard bold font with \setmathfont

Question

I'm using Latin Modern Math as math font but, since I don't like the blackboard bold letters that come with it, I replace them with the ones from TeX Gyre Termes Math. I believe to achieve this, the following is correct

\usepackage{unicode-math}
\setmainfont{Latin Modern Roman}
\setmathfont{Latin Modern Math}
\setmathfont{TeX Gyre Termes Math}[range=bb] % blackboard letters

Unfortunately the last line also introduces white space after \big* delimiters:

Compare with:

Any idea why this is happening?

\documentclass{article}
\usepackage{amsmath,amssymb,amsfonts}
\usepackage{unicode-math}
\setmainfont{Latin Modern Roman}
\setmathfont{Latin Modern Math}
\setmathfont{TeX Gyre Termes Math}[range=bb] % blackboard letters
\begin{document}

\[
\Bigl(\frac{d\symbf\varphi}{d\symbf\theta}\Bigr)^{\top} \frac{d\eta}{d\mu}
\quad \mathbb{R}^2
\]

\end{document}

Answer 1

With \left(...\right) instead and, of course, an upright differential operator:

\documentclass{article}
\usepackage{amsmath}
\usepackage{unicode-math}
\setmainfont{Latin Modern Roman}
\setmathfont{Latin Modern Math}
\setmathfont{TeX Gyre Termes Math}[range=bb] % blackboard letters
\begin{document}

    \[
    \left(\frac{\mathrm{d}\mbfitphi}{\mathrm{d}\mbfittheta}\right)^{\top} 
    \frac{\mathrm{d}\eta}{\mathrm{d}\mu}
    \quad \BbbR^2
    \]

\end{document}

Answer 2

The problem seem to be gone with LuaTeX in TL 2016. With XeTeX it is still broken, but Unicode math typesetting is in general broken in XeTeX.

Also amssymb and amsfonts are superfluous as unicode-math has all the symbols.

\documentclass{article}
\pagestyle{empty}
\usepackage{amsmath}
\usepackage{unicode-math}
\setmainfont{Latin Modern Roman}
\setmathfont{Latin Modern Math}
\setmathfont{TeX Gyre Termes Math}[range=\mathbb] % blackboard letters
\begin{document}

\[
\Bigl(\frac{d\symbf\varphi}{d\symbf\theta}\Bigr)^{\top} \frac{d\eta}{d\mu}
\quad \mathbb{R}^2
\]

\end{document}

With TeX Gyre Termes

Without TeX Gyre Termes