# Why does hyphsubst not work with LuaTeX?

Consider the following MWE:

\RequirePackage[ngerman=ngerman-x-latest]{hyphsubst}
\documentclass{scrartcl}

\usepackage[english]{babel}
\usepackage{blindtext}

\begin{document}
\Blindtext
\end{document}


This compiles fine with pdflatex but not with lualatex. Why is there an error when using the latest hyphenation patterns? Can I make them work with lualatex?

This is LuaTeX, Version beta-0.70.1-2011061410 (rev 4277)
restricted \write18 enabled.
(./BabelTest.tex
LaTeX2e <2011/06/27>
LuaTeX adaptation of babel <v3.8l-luatex-1.5> and hyphenation patterns for engl
(/usr/local/texlive/2011/texmf-dist/tex/generic/oberdiek/hyphsubst.sty
(/usr/local/texlive/2011/texmf-dist/tex/generic/oberdiek/infwarerr.sty))
(./scrbook.cls
Document Class: scrbook 2012/03/08 v3.10a KOMA-Script document class (book)
(/usr/local/texlive/2011/texmf-dist/tex/latex/koma-script/scrkbase.sty
(/usr/local/texlive/2011/texmf-dist/tex/latex/koma-script/scrbase.sty
(/usr/local/texlive/2011/texmf-dist/tex/latex/graphics/keyval.sty)
(/usr/local/texlive/2011/texmf-dist/tex/latex/koma-script/scrlfile.sty

))) (/usr/local/texlive/2011/texmf-dist/tex/latex/koma-script/tocbasic.sty)
(/usr/local/texlive/2011/texmf-dist/tex/latex/koma-script/scrsize11pt.clo)
(/usr/local/texlive/2011/texmf-dist/tex/latex/koma-script/typearea.sty
Package typearea, 2012/03/08 v3.10a KOMA-Script package (type area)

)) (/usr/local/texlive/2011/texmf-dist/tex/generic/babel/babel.sty
(/usr/local/texlive/2011/texmf-dist/tex/generic/babel/english.ldf
(/usr/local/texlive/2011/texmf-dist/tex/generic/babel/babel.def
! LuaTeX error ...11/texmf-dist/tex/luatex/hyph-utf8/luatex-hyphen.lua:60: luat
ex-hyphen: \language0 should be dumped in the format
stack traceback:
[C]: in function 'error'
...11/texmf-dist/tex/luatex/hyph-utf8/luatex-hyphen.lua:22: in function 'err'
anguage'
<\directlua >:1: in main chunk.
\bbl@patterns ...pestring {#1}", \the \language )}
\fi \fi \fi
l.131 \main@language{english}

-
No errors here, what is the version of Babel printed in the top of your log? Here I've LuaTeX adaptation of babel <v3.8l-luatex-1.5>, I suspect earlier versions to cause an error (recently the issue with Babel aliases was fixed). –  Khaled Hosny May 4 '12 at 18:33
I am sorry, I put in the wrong documentclass. It now turns out it just does not work with the KOMA Script classes. Can you compile the above code as well? See also the error message. –  Ingo May 4 '12 at 22:25
Still no error here. –  Khaled Hosny May 4 '12 at 22:35

I've got no error when compiling with LuaTeX, but hyphsubst doesn't seem to work with LuaTeX. You can verify by running the following document with LuaTeX:

\RequirePackage[german=ngerman-x-latest]{hyphsubst}
\documentclass{minimal}
\usepackage[german]{babel}
\begin{document}
\showhyphens{Meister}
\end{document}


It should report Meis-ter but reports Mei-ster. With luatex -recorder test.tex you can look into the generated .fls file and see that only the file hyph-de-1901.pat.txt gets loaded.

-
No wonder, loading hyphenation patterns is completely different under lualatex. –  Khaled Hosny May 4 '12 at 19:49
@KhaledHosny And why shouldn't hyphsubst be able to switch between different backends as it does now with LaTeX and TeX? I don't see a technical difficulty. –  topskip May 4 '12 at 19:58
I’ve no idea how hyphsubst works, but without it’s author explicitly adding support for LuaTeX, it is unlikely to work out of box. We have LuaTeX specific version of Babel’s hyphen.cfg to handle loading patterns in a LuaTeX specific way. –  Khaled Hosny May 4 '12 at 20:10
@PatrickGundlach I've added some code; can you point to a word that's hyphenated differently with ngerman and ngerman-x-latest? –  egreg May 5 '12 at 13:43
@egreg ab-fal-lauf-be-rei-tung (ngerman) and ab-fall-auf-be-rei-tung (latest) –  topskip May 5 '12 at 14:56

hyphsubst simply changes the number associated to the control sequence that LaTeX uses when choosing a language.

For instance \selectlanguage{ngerman} executes, among other things, \language\l@ngerman and \l@ngerman is defined at format creation to be the number "1E (on my machine, it can vary according to the language.dat file).

The number corresponding to ngerman-x-latest is (on my machine) "4.

When one says

\RequirePackage[german=ngerman-x-latest]{hyphsubst}


the basic action is to change the meaning of \l@ngerman into "4; more precisely, the action is

\chardef\l@ngerman=\csname l@ngerman-x-latest\endcsname


Of course, if the languages are not preloaded, as it happens with LuaLaTeX, this can't work (precautions are taken in the package so as a correct assignment is performed even if the meaning of \csname l@ngerman-x-latest\endcsname was \relax). That's why we end with the patterns for English, I guess.

## A possible step towards a solution

Apparently, LuaLaTeX knows about the correct numbers, so what's needed is to force loading the correct language. A test:

\documentclass{article}

%% print the numbers
\typeout{ngerman: \number\csname l@ngerman\endcsname}
\typeout{ngerman-x-latest: \number\csname l@ngerman-x-latest\endcsname}

\usepackage{hyphsubst}

\usepackage[ngerman]{babel}

\begin{document}

\typeout{Current language: \number\language}

\HyphSubstLet{ngerman}{ngerman-x-latest}

\typeout{Current language: \number\language}

\end{document}


The relevant part of the log file shows

ngerman: 30
ngerman-x-latest: 4
[...]
Current language: 30
Current language: 4


so the value of \language seems to be correctly set. However the .fls file reports only

INPUT /usr/local/texlive/2011/texmf-dist/tex/generic/hyph-utf8/patterns/txt/hyph-de-1996.pat.txt
INPUT /usr/local/texlive/2011/texmf-dist/tex/generic/hyph-utf8/patterns/txt/hyph-de-1996.pat.txt
INPUT /usr/local/texlive/2011/texmf-dist/tex/generic/hyph-utf8/patterns/txt/hyph-de-1996.hyp.txt


which seems to mean that the hyphenation patterns for ngerman-x-latest are not loaded.

I've tried to add something like

%% Load the new language
\setbox0=\vbox{
\everypar{}\begin{hyphenrules}{ngerman-x-latest}abc\end{hyphenrules}
}


in the preamble or in the document, in order to see if the patterns are loaded, but this doesn't seem so.

-
Check tug.org/svn/texhyphen/trunk/hyph-utf8/doc/luatex/hyph-utf8/… for how patterns are loaded with lualatex. –  Khaled Hosny May 5 '12 at 14:30
@KhaledHosny The macro responsible for it seems to be \bbl@patterns, which is called with ngerman-x-latest, but doesn't seem to load the patterns. :( –  egreg May 5 '12 at 14:48

To answer the last question of the OP "Can I make them work with lualatex?":

Patterns from package dehyph-exptl are already included in package hyph-utf8 and those patterns are activated for UTF-8 aware TeX engines, e.g., LuaTeX and XeTeX. Because German patterns for TeX and pdfTeX are frozen, package hyph-utf8 contains conventional German patterns, too, but activates those only for said 8 bit TeX engines.

That is, when using LuaLaTeX you don't need to care about packages dehyph-exptl and hyph-subst at all. This example

\documentclass{article}
% \usepackage[ngerman]{babel}
\usepackage[german]{babel}
\begin{document}
\showhyphens{modernste}
\end{document}


when compiled with LaTeX and LuaLaTeX should show the following results:

             german           ngerman

LaTeX        mo-d-ern-ste     mo-d-erns-te
LuaLaTeX     mo-dern-ste      mo-derns-te


You can find out what version of experimental German patterns are shipped with package hyph-utf8 (usually the latest) by typing

\$head kpsewhich hyph-de-1901.lic.txt


on the shell. Windows users type

>for /F "usebackq" %f in (kpsewhich hyph-de-1901.lic.txt) do more "%f"


on the command-line. Note the backquotes surrounding the kpsewhich calls!

-
Welcome to TeX.sx! You don't have to sign with your name since it automatically appears in the lower right corner of your post. –  doncherry Jun 22 '12 at 16:00