When I am using STIX Math in combination with LuaLaTeX with unicode-math, the spacing in formulas looks quite horrible. With other math fonts (I've tried with TeX Gyre Pagella, Latin Modern, and some others), things appear to look fine.
I'm running MacTeX-2016 including all updates. The MWE below gives the following output:
The problem appears similar to this one for XeTeX. There it was mentioned that switching to LuaTeX solved the problem. In my case switching to XeTeX doesn't appear to resolve the problem.
MWE:
\documentclass[10pt]{article}
\usepackage{amsmath}
\usepackage{fontspec,xunicode}
\usepackage[math-style=ISO]{unicode-math}
\setmainfont{STIX}\setmathfont{STIX Math}
% \setmainfont{Tex Gyre Pagella}\setmathfont{texgyrepagella-math.otf}
% \setmainfont{Latin Modern Roman}\setmathfont{Latin Modern Math}
% \setmainfont{TeX Gyre Termes}\setmathfont{TeX Gyre Termes Math}
% \setmainfont{TeX Gyre Schola}\setmathfont{TeX Gyre Schola Math}
% \setmainfont{TeX Gyre Bonum}\setmathfont{TeX Gyre Bonum Math}
\renewcommand{\vec}[1]{\symbf{#1}}
\begin{document}
This is a test.
\begin{align}
P(\vec{k}) = \int_a^b
e^{i\vec{k} \cdot \vec{R}}
P(\vec{R}) \,\mathrm{d}\vec{R}.
\end{align}
\end{document}
\usepackage{stix}
instead of through fontspec then everything is fine, also.