I wish to use protrusion and expansion from microtype as I like the look you get with them. I use amsthm and typeset theorems in italic as usual (with \theoremstyle{plain}
). With microtype enabled, LaTeX fails to break some lines where the last item on the line is something in italic. I get three failures to break nicely with my included example.
I have microtype 2.5 and use lualatex with Latin Modern Roman. I have a working example, though it's not very minimal. If I remove much more the ancillary changes to formatting hide the problem I suspect is a bug. The issue doesn't seem to be with the fact I'm using hyphenated words like Erdos-Hajnal, since the line break failure also occurs on other lines, and problems even occur in my bibliography where some portions of the reference text are in italics. The problem also occurs for me if I remove the \usepackage{fontspec}
line. If I remove that line and compile with pdflatex there are no problems.
(Fairly) Minimal Example:
\documentclass[a4paper,twoside]{scrartcl}
\usepackage[UKenglish]{babel}
\usepackage{amsmath,amsthm}
\usepackage{fontspec,blindtext}
\usepackage[babel,protrusion=true,expansion]{microtype}
\theoremstyle{plain}
\newtheorem{thm}{Theorem}[section]
\newtheorem{prop}[thm]{Proposition}
\newtheorem{conj}[thm]{Conjecture}
\begin{document}
\begin{conj}
For all graphs $H$ there is an $\epsilon > 0$ such that for all sufficiently large $n$, and for all $G\in\mathcal{G}^n$ either $H$ is an induced subgraph of $G$, or $G$ contains a homogeneous set of size at least $n^\epsilon$.
\end{conj}
\begin{prop}
A short dummy paragraph.
Let $\epsilon_2=\delta\epsilon_1$, where $\delta<1/(2k+1)$. Then there exists $n_0$ such that all graphs on $n\geq n_0$ vertices which do not contain homogeneous sets of size $n^{\epsilon_2}$ have $H$ as an induced subgraph. That is, $H$ has the Erd\H{o}s-Hajnal property.
\end{prop}
\begin{thm}
Let the graphs $H$ and $F$ have the Erd\H{o}s-Hajnal property, and let $V(H) = \{v_1,\dotsc,v_k\}$. Then $H(F, v_2,\dotsc, v_k)$ obtained by substituting $v_1$ for $F$ also has the Erd\H{o}s-Hajnal property.
\end{thm}
\blindtext
\end{document}
EDIT: this is clearly nothing to do with amsmath, and a simple way to reproduce a similar italic-linebreaking problem is with:
% !TEX TS-program = lualatex
\documentclass{scrartcl}
\usepackage{fontspec}
\usepackage[UKenglish,latin]{babel}
\usepackage{microtype}
\usepackage{lipsum}
\begin{document}
\lipsum[1-2]
\textit{\lipsum[1-2]}
\end{document}