I believe this is related, but my issue is not math-specific. The problem is:
%!TEX TS-program = xelatex
%!TEX encoding = UTF-8 Unicode
\documentclass[12pt]{article}
% PACKAGES
\usepackage{geometry}
\usepackage{fontspec}
\usepackage{xltxtra}
\defaultfontfeatures{Mapping=tex-text,Numbers=OldStyle}
\setromanfont[Mapping=tex-text]{Linux Libertine O}
\setsansfont[Scale=MatchLowercase,Mapping=tex-text]{Linux Biolinum O}
\setmonofont[Scale=MatchLowercase]{Linux Libertine Mono O}
\begin{document} % ==============================================================
\newfontfamily{\HT}{Hoefler Text}
\newfontfamily{\BV}{Baskerville}
When using and transitioning from roman to \textit{italics} and back, \LaTeX\ appears to preserve character spacing at the baseline.
{\Huge It is more \textit{noticeable} in larger font sizes.}
{\Large\bfseries The problem persists at \textit{other} weights.}
\textsf{...also with Linux Biolinum, the {\itshape associated} sans-serif font.} {\tiny ← That one is \textit{really} noticeable!}
{\HT This does not appear to be a \textit{problem} in other typefaces.}
{\BV Where \LaTeX\ evidently \textit{preserves} character spacing at mid-ex-height or so.}
Some graphic (if contrived) examples:
Linux Libertine: l\textit{l}l
{\HT Hoefler Text: l\textit{l}l}
{\BV Baskerville: l\textit{l}l}
\end{document}