# Can/does hyphenation ignore accents?

LuaLaTeX MWE:

\documentclass[11pt]{article}

\usepackage{polyglossia}
\setotherlanguage{russian}
\usepackage{tabu}
top=72pt]{geometry}
\usepackage{fontspec}
\setmainfont{CMU Serif}

\begin{document}

\begin{tabu}{X|X}
Е́сли вы изу́чите ру́сский язы́к, вы полу́чите хоро́шую пабо́ту.
& If you master (will master) Russian, you will get a good job. \\

Е́сли вы изу́чите ру́сский язы́к, вы по\-лу́\-чите хо\-ро́\-шую пабо́ту.
& If you master (will master) Russian, you will get a good job. \\
\end{tabu}

\end{document}


Notice that I've used U+0301, "combining acute accent," to add accents to the Russian. (Showing where the accents are is common in English texts designed for learning Russian.) Here's the Russian half (the left half of the table) from the resulting PDF:

On the first row of the table I get Underfull \hbox (badness 6332) in paragraph at lines 19--19, presumably because LuaLaTeX doesn't know how to hyphenate the words with added accents. If I explicitly hint the hyphenation, as I did on the second row, it gets hyphenated correctly.

Is there any way to tell LuaLaTeX/Polyglossia/whatever to ignore the added accents when hyphenating words?

• As David says, you need to tell the engine what language is to be used: if English is the current language, the engine doesn't know how to hyphenate Russian. It seems that XeLaTeX is able to ignore the combining accent when choosing hyphenation points, whereas LuaLaTeX isn't. – egreg Apr 1 '14 at 16:20
• @egreg my original plan was to dynamically modify the pattern file to add the combining accent everywhere as luatex can load pattern files late but didn't try that the word hyphenated anyway 9in a different place) – David Carlisle Apr 1 '14 at 16:21
• @DavidCarlisle It seems not needed. I assumed that \hyphenrules{russian} works the same in XeLaTeX and LuaLaTeX; it seems not the case. – egreg Apr 1 '14 at 16:25

I don't really know ployglossia but if I explicitly select russian the word is hyphenated:

\documentclass[11pt]{article}

\usepackage{polyglossia}
\setotherlanguage{russian}
\usepackage{tabu}
top=72pt]{geometry}
\usepackage{fontspec}
\setmainfont{CMU Serif}

\begin{document}
%
%\input{hyph-ru}

\begin{tabu}{X|X}
%\hyphenrules{russian}
\leavevmode\selectlanguage{russian}\righthyphenmin=3 \lefthyphenmin=2
Е́сли вы изу́чите ру́сский язы́к, вы полу́чите хоро́шую пабо́ту.
& If you master (will master) Russian, you will get a good job. \\

Е́сли вы изу́чите ру́сский язы́к, вы по\-лу́\-чите хо\-ро́\-шую пабо́ту.
& If you master (will master) Russian, you will get a good job. \\
\end{tabu}

\end{document}

• With XeLaTeX, but not with LuaLaTeX, it seems. Probably \begin{tabu}{>{\hyphenrules{russian}}X|X} is better. – egreg Apr 1 '14 at 16:18
• @egreg the image was made with lulatex? – David Carlisle Apr 1 '14 at 16:20
• Doh! Quirks with \hyphenrules, it appears. – egreg Apr 1 '14 at 16:22
• @egreg but with hyphenrules \righthyphenmin gets set (I guess) so hyphenation occurs in same place, i updated my answer thanks. – David Carlisle Apr 1 '14 at 16:25
• Thanks for the attempt. Today I learned about the \hyphenrules command. Am I right that the consensus is that this only works for XeLaTeX and not LuaLaTeX? It didn't work here for me using LuaLaTeX, at least. – dsedivec Apr 2 '14 at 14:55

Based on David Carlisle's solution and egreg's comment I came up with something that seems to work on my TeX installation:

\documentclass[11pt]{article}

\usepackage[main=english,russian]{babel}
\usepackage{tabu}
top=72pt]{geometry}
\usepackage{fontspec}
\setmainfont{CMU Serif}

\begin{document}

\begin{tabu}{>{\hyphenrules{russian}}X|X}
Е́сли вы изу́чите ру́сский язы́к, вы полу́чите хоро́шую пабо́ту.
& discretionary discretionary discretionary discretionary
discretionary \\

Е́сли вы изу́чите ру́сский язы́к, вы по\-лу́\-чите хо\-ро́\-шую пабо́ту.
& discretionary discretionary discretionary discretionary
discretionary \\
\end{tabu}

\end{document}


The output is now hyphenated properly:

I changed the English text to words that were likely to need hyphenation, so that I could see if I accidentally broke English hyphenation during testing.

I guess Babel now works under LuaLaTeX. I thought it did not, which is why I was using Polyglossia.

I'm new to multi-lingual LaTeX documents, so I'm a bit concerned that it looks like I have to manually tell LaTeX what kind of language each section of text is in, despite it being painfully obvious to a human. I was hoping I could simultaneously load both English and Russian hyphenation rules, and have LaTeX choose which one to use based on the surrounding characters. Manually marking each little Russian phrase as being in the Russian language is going to kind of suck.

• spotting that the alphabet is cyrillic as opposed to latin could be automated but spotting language is harder, think english and american english or french and spanish or russian and Ukranian. Given a large enough sample google translate (for example) make a reasonable guess but TeX can't really do that. – David Carlisle Apr 2 '14 at 20:03