# \tableofcontents encoding issue with lualatex

I have a issue when trying to generate table of contents with lualatex in russian. Following example fails to build with ! Undefined control sequence. \contentsname ->{\cyr \CYRS \cyro \cyrd \cyre \cyrr \cyrzh \cyra \cyrn \cyri... which stands for 'Содержание' ('Contents' in russian)

\documentclass{article}

\usepackage{fontspec}
\usepackage{xunicode}
\setmainfont[Ligatures=TeX]{DejaVu Serif}
\usepackage[russian]{babel}

\begin{document}
\tableofcontents
\clearpage
\section{тест 1}
\section{тест 2}
\section{тест 3}
\end{document}


When I load english version of babel it ok, execept of table of contents title in english "Contents". Is there a way to correctly specify encoding for babel in lualatex?

-

The following covers the Russian alphabet:

\documentclass{article}
\usepackage[russian]{babel}

\DeclareTextCommand{\CYRA}{EU2}{А}
\DeclareTextCommand{\cyra}{EU2}{а}
\DeclareTextCommand{\CYRB}{EU2}{Б}
\DeclareTextCommand{\cyrb}{EU2}{б}
\DeclareTextCommand{\CYRV}{EU2}{В}
\DeclareTextCommand{\cyrv}{EU2}{в}
\DeclareTextCommand{\CYRG}{EU2}{Г}
\DeclareTextCommand{\cyrg}{EU2}{г}
\DeclareTextCommand{\CYRD}{EU2}{Д}
\DeclareTextCommand{\cyrd}{EU2}{д}
\DeclareTextCommand{\CYRE}{EU2}{Е}
\DeclareTextCommand{\cyre}{EU2}{е}
\DeclareTextCommand{\CYRYO}{EU2}{Ё}
\DeclareTextCommand{\cyryo}{EU2}{ё}
\DeclareTextCommand{\CYRZH}{EU2}{Ж}
\DeclareTextCommand{\cyrzh}{EU2}{ж}
\DeclareTextCommand{\CYRZ}{EU2}{З}
\DeclareTextCommand{\cyrz}{EU2}{з}
\DeclareTextCommand{\CYRI}{EU2}{И}
\DeclareTextCommand{\cyri}{EU2}{и}
\DeclareTextCommand{\CYRISHRT}{EU2}{Й}
\DeclareTextCommand{\cyrishrt}{EU2}{й}
\DeclareTextCommand{\CYRK}{EU2}{К}
\DeclareTextCommand{\cyrk}{EU2}{к}
\DeclareTextCommand{\CYRL}{EU2}{Л}
\DeclareTextCommand{\cyrl}{EU2}{л}
\DeclareTextCommand{\CYRM}{EU2}{М}
\DeclareTextCommand{\cyrm}{EU2}{м}
\DeclareTextCommand{\CYRN}{EU2}{Н}
\DeclareTextCommand{\cyrn}{EU2}{н}
\DeclareTextCommand{\CYRO}{EU2}{О}
\DeclareTextCommand{\cyro}{EU2}{о}
\DeclareTextCommand{\CYRP}{EU2}{П}
\DeclareTextCommand{\cyrp}{EU2}{п}
\DeclareTextCommand{\CYRR}{EU2}{Р}
\DeclareTextCommand{\cyrr}{EU2}{р}
\DeclareTextCommand{\CYRS}{EU2}{С}
\DeclareTextCommand{\cyrs}{EU2}{с}
\DeclareTextCommand{\CYRT}{EU2}{Т}
\DeclareTextCommand{\cyrt}{EU2}{т}
\DeclareTextCommand{\CYRU}{EU2}{У}
\DeclareTextCommand{\cyru}{EU2}{у}
\DeclareTextCommand{\CYRF}{EU2}{Ф}
\DeclareTextCommand{\cyrf}{EU2}{ф}
\DeclareTextCommand{\CYRH}{EU2}{Х}
\DeclareTextCommand{\cyrh}{EU2}{х}
\DeclareTextCommand{\CYRC}{EU2}{Ц}
\DeclareTextCommand{\cyrc}{EU2}{ц}
\DeclareTextCommand{\CYRCH}{EU2}{Ч}
\DeclareTextCommand{\cyrch}{EU2}{ч}
\DeclareTextCommand{\CYRSH}{EU2}{Ш}
\DeclareTextCommand{\cyrsh}{EU2}{ш}
\DeclareTextCommand{\CYRSHCH}{EU2}{Щ}
\DeclareTextCommand{\cyrshch}{EU2}{щ}
\DeclareTextCommand{\CYRHRDSN}{EU2}{Ъ}
\DeclareTextCommand{\cyrhrdsn}{EU2}{ъ}
\DeclareTextCommand{\CYRERY}{EU2}{Ы}
\DeclareTextCommand{\cyrery}{EU2}{ы}
\DeclareTextCommand{\CYRSFTSN}{EU2}{Ь}
\DeclareTextCommand{\cyrsftsn}{EU2}{ь}
\DeclareTextCommand{\CYREREV}{EU2}{Э}
\DeclareTextCommand{\cyrerev}{EU2}{э}
\DeclareTextCommand{\CYRYU}{EU2}{Ю}
\DeclareTextCommand{\cyryu}{EU2}{ю}
\DeclareTextCommand{\CYRYA}{EU2}{Я}
\DeclareTextCommand{\cyrya}{EU2}{я}

\usepackage{fontspec}
\setmainfont[Ligatures=TeX]{DejaVu Serif}

\begin{document}
\tableofcontents
\clearpage
\section{тест 1}
\section{тест 2}
\section{тест 3}
\end{document}


A more abstract way for getting (almost) all the known cyrillic characters can be obtained with code by David Carlisle (see http://tex.stackexchange.com/a/110016/4427)

\documentclass{article}

\usepackage{fontspec}
\setmainfont[Ligatures=TeX]{DejaVu Serif}

% This code must go after xunicode is loaded (by fontspec),
% but before babel.
\makeatletter
\let\KeptDeclareUnicodeCharacter\DeclareUnicodeCharacter
\def\DeclareUnicodeCharacter#1#2{\@declare{#1}#2!!}%
\def\@declare#1#2#3!!{%
\ifdefined#2\else
\begingroup
\lccodeE=E \lccodeU=U
\lccodeA="#1\relax
\lowercase{\endgroup\DeclareTextCommand{#2}{EU2}{A}}%
\typeout{Redefined \string#2}
\fi}
\input{x2enc.dfu}% this contains all the cyrillic characters
\let\DeclareUnicodeCharacter\KeptDeclareUnicodeCharacter
\let\KeptDeclareUnicodeCharacter\@undefined
\let\@declare\@undefined
\makeatother

\usepackage[russian]{babel}

\begin{document}
\tableofcontents
\clearpage
\section{тест 1}
\section{тест 2}
\section{тест 3}
\end{document}


Another way might be to redefine \captionsrussian and the other relevant commands (taken from gloss-russian.ldf of polyglossia)

\def\captionsrussian{%
\def\prefacename{Предисловие}%
\def\refname{Список литературы}%
\def\abstractname{Аннотация}%
\def\bibname{Литература}%
\def\chaptername{Глава}%
\def\appendixname{Приложение}%
\ifdefined\thechapter
\def\contentsname{Оглавление}%
\else
\def\contentsname{Содержание}%
\fi
\def\listfigurename{Список иллюстраций}%
\def\listtablename{Список таблиц}%
\def\indexname{Предметный указатель}%
\def\authorname{Именной указатель}%
\def\figurename{Рис.}%
\def\tablename{Таблица}%
\def\partname{Часть}%
\def\enclname{вкл.}%
\def\ccname{исх.}%
\def\pagename{с.}%
\def\seename{см.}%
\def\alsoname{см.~также}%
\def\proofname{Доказательство}%
}
\def\daterussian{%
\def\today{\number\day%
\space\ifcase\month\or%
января\or
февраля\or
марта\or
апреля\or
мая\or
июня\or
июля\or
августа\or
сентября\or
октября\or
ноября\or
декабря\fi%
\space \number\year\space г.}}
\makeatletter
\def\@Asbuk#1{\ifcase#1\or
А\or Б\or В\or Г\or Д\or Е\or Ж\or
З\or И\or К\or Л\or М\or Н\or О\or
П\or Р\or С\or Т\or У\or Ф\or Х\or
Ц\or Ч\or Ш\or Щ\or Э\or Ю\or Я\else\@ctrerr\fi}
\def\@asbuk#1{\ifcase#1\or
а\or б\or в\or г\or д\or е\or ж\or
з\or и\or к\or л\or м\or н\or о\or
п\or р\or с\or т\or у\or ф\or х\or
ц\or ч\or ш\or щ\or э\or ю\or я\else\@ctrerr\fi}
\makeatother


Added April 2013: the forthcoming version 3.9 of babel should solve the problem by checking whether the typesetting engine understands Unicode natively and setting the fixed words accordingly. This version of babel is already on CTAN, but still hasn't found its way in TeX Live. It will soon.

-
First. Why you are declaring \cyr.. commands for EU2 encoding? Future unicode solution will not need any \cyr... command, hence this should be named something like OU2 (old unicode). Second. In fact, I am using cited set of \DeclareTextCommand{\cyrya}{EU2}{...} commands for some time. It really works, but I wonder where EU2 encoding is activated? As far as I know, by default, fontspec declares EU1 encoding. –  Igor Kotelnikov Oct 6 '11 at 15:16
@Igor fontspec declares EU1 encoding for XeLaTeX and EU2 for LuaLaTeX. Of course one has to declare those commands for EU2 encoding, because babel uses them in the .ldf file for Russian and so they must be known –  egreg Oct 6 '11 at 21:40
Thank you. Perhaps, it is worthy to elaborate unicode russian stuff for babel, it would be named russian_u_.ldf or even russian.ldf since currently babel comes with russian_b_.ldf. however, I am not sure that babel is the only package which relies on \cyr.. commands. I will try... –  Igor Kotelnikov Oct 7 '11 at 3:20

Use updated russianb.ldf from http://www.ctan.org/pkg/russian instead of that that came with babel 3.8. Updated russianb.ldf will be included with babel 3.9.

-

There isn't yet either (full) babel or (any) polyglossia support for lualatex. So unless you need the specific extras such as direct lua code, you should use xelatex + polyglossia for the time being or work with the solutions posted by Ulrike and egreg.

-

Sadly polyglossia is still not adapted for lualatex. It uses xetex specific commands for which no lualatex equivalent exist. But as far as I can see gloss-russian.ldf doesn't need them so you can try this:

\documentclass[a4paper]{article}
\usepackage{fontspec}
\setmainfont[Ligatures=TeX]{DejaVu Serif}
\def\newXeTeXintercharclass#1{} %hack to fool polyglossia
\usepackage{polyglossia}
\setmainlanguage{russian}
\begin{document}

abc
\end{document}


Attention! This will probably break horribly if you load a polyglossia language like french which actually needs the \XeTeXintercharclass feature. In this case one the work-arounds of egreg is better.

-
That's right! Adding French results in subtle errors, even if babel is used. I'll add an example of such errors soon in separate post. –  Igor Kotelnikov Jun 14 '11 at 7:04
Although lualatex does run without crashing, it seems that this hack does not allow the hyphenation patterns to be loaded, even in languages that do not need the \XeTeXintercharclass feature. –  Mateus Araújo Oct 19 '11 at 22:49
@Mateus: I haven't the time to test now. But: lualatex loads hyphenations differently than xelatex (there are loaded on the fly). Perhaps polyglossia needs some adaption. You could contact the new maintainer of polyglossia. –  Ulrike Fischer Oct 20 '11 at 9:11
@UlrikeFischer: Indeed! The systems are independent! The issue was that I was trying the hyphenation patterns for the language brazil, which XeTeX knows about, but LuaTeX doesn't. I changed it to portuguese, and it worked. Probably someone should add the synonym to language.dat.lua. Thanks! –  Mateus Araújo Oct 20 '11 at 14:31

Try to use [russian] as an argument to \documentclass instead of Babel. At least for me, using lualatex 0.60 with texlive 2010 including all updates, that works.

-
It doesn't work with lualatex 0.70.1 –  Martin Schröder Jun 30 '11 at 21:52

I have not much experience in using LuaTeX. Your code seems to be more for XeLaTeX. Perhaps you should replace babel with polyglossia. The example below works at least with XeLaTeX.

\documentclass{article}
\usepackage{fontspec}
\usepackage{xunicode}
\setmainfont[Ligatures=TeX]{DejaVu Serif}
\usepackage{polyglossia}
\setmainlanguage{russian}

\begin{document}
\tableofcontents
\clearpage
\section{тест 1}
\section{тест 2}
\section{тест 3}
\end{document}

-
No, both lualatex and xelatex use fontspec. But the problem is that there isn't yet either babel or polyglossia` support for lualatex. So unless Avsej needs the specific extras such as direct lua code, he should use xelatex for the time being. –  Alan Munn Jan 17 '11 at 19:18
@Alan: I think this comment is worth to be transformed to an answer. –  Thorsten Donig Jan 17 '11 at 19:43
Thank you. Frankly speaking I using xelatex now. But I think that it is good thing to have embedded lua :) Shall I close this question? There are many posts in internets about lack of babel/polyglossia for lualatex. Is there some roadmap for this features? –  avsej Jan 18 '11 at 20:27