LuaLaTeX: Either wrong spacing with integrals or wrong mathrm font

Question

This question is related to newtxmath + libertine: integral sign spacing is wrong. I'm using LuaLaTeX from TeXLive 2016 (Version 0.95.0).

There seems to be a known spacing problem with lualatex and integrals, as described e. g. in the question linked above or in Too much space after integral sign with unicode-math and lualatex. I have a similar problem with the following code (where I want to use Libertine both as a text and math font):

\documentclass{minimal}

\usepackage[libertine]{newtxmath}
\usepackage{fontspec}
\setmainfont{Linux Libertine O}
\setmathrm{Linux Libertine O}

\begin{document}

cos $\cos$
\[ \int_0^1 \cos x \]

\end{document}

For me the output looks like this:

So here the integral sign is too small and the limits are too far away from it.

By experimenting I found out that this spacing problem does not occur if I load fontspec with the no-math option (leaving the above code unchanged otherwise). But then the line

\setmathrm{Linux Libertine O}

has no effect any more (actually it wasn't necessary before), i. e. \mathrm commands are now typeset in Computer Modern, and the output now looks like this:

Now, the answer https://tex.stackexchange.com/a/326837/25031 to the aforementioned question suggests that I could use the type1 version of the Libertine font, adding a line

\usepackage{libertine-type1}

after the fontspec call. This does produce the correct output, but I wonder if there is some "cleaner" way to achieve this? In fact, in my real document the usage of libertine-type1 leads to lots of conflicts with other packages.

Answer 1

As mentioned as addition in my answer https://tex.stackexchange.com/a/328047/2388 the core of the problem is a side effect of the math constants table in the open type Linux Libertine =. I discussed this shortly on the luatex list (http://tug.org/pipermail/luatex/2016-September/006172.html and following) and imho you can sum up the discussion that if you mix in math legacy fonts and open type fonts it can have bad side effects with some open type fonts -- and libertine is one of this problematic font.

So either switch completely to open type fonts with unicode-math as suggested by Henri, or force newtxmath to use only type1/legacy math fonts:

\documentclass{article}

\def\rmdefault{LinuxLibertineT-LF}
\usepackage[libertine]{newtxmath}
\usepackage[no-math]{fontspec}

\setmainfont{Linux Libertine O}


\begin{document}

cos $\cos \mathrm{cos} $
\[ \int_0^1 \cos x \]

\end{document}

Answer 2

I would go for a completely different approach and just use the Libertinus font (the successor of Linux Libertine) which offers a complete Unicode math font (samples). This is not only easier to use but also makes the document look more consistent and you get matching font families throughout text and math.

\documentclass{article}

\usepackage{unicode-math}
\setmainfont{Libertinus Serif}
\setmathfont{Libertinus Math}

\begin{document}

\[ \int_0^1 \cos x \]

\end{document}