2

When generating documents in Khmer, we noticed that some ligatures do not work. We found that turning off XeTeXlinebreaklocale fixes all ligatures, however this causes problems with linebreaks, and lines extend past the right margin if there aren't any zero width spaces between words. (In Khmer words within the same sentence or phrase are generally run together with no spaces between them.)

Our objective is to have all ligatures work, and to use XeTeXlinebreaklocale at the same time.

Please note that we are not looking for a manual workaround for this particular example. We are looking for a fix to the root cause that we can then use in our automated system where we cannot fix individual instances of this problem.

MWE 1: XeTeXlinebreaklocale turned off, ligatures work (text in red), however text runs into right margin.

MWE 1: Ligatures work

\documentclass[12pt]{article}

%\XeTeXlinebreaklocale "km"
\usepackage{color}
\usepackage{fontspec}
\setmainfont[Ligatures=Required]{Khmer OS Content}

\begin{document}
មានមនុស្សម្នាក់ដែលបានធ្វើការមួយដែលអាចដឹងបាន នោះគឺជាកា{\color{red}ស្រ្តូ}នៅទីនេះ។ បាទ អ្នកទាំងអស់គ្នា។ គាត់បានចុះទៅទីនោះ។ ហើយពួកនាយទុនមានអស់ទាំងព្រនង់ ហើយនឹងអ្វីៗដែល… វាគឺឋិតនៅក្នុងស្តង់ដាមាស ដូចជាសហរដ្ឋអាមេរិកដែរ។ តើគាត់បានធ្វើអ្វី? គាត់បានទិញអស់ទាំងបណ្ណ័ភាគហ៊ុន។ គឺរកលុយតាមគ្រប់ទាំងវិធីដែលគាត់អាចធ្វើទៅបាន។ បន្ទាប់មកតើគាត់ធ្វើអ្វីទៀត? គាត់ក៏បានក្លែងបន្លំក្រដាស់ប្រាក់ ហើយបានប្ដូរវា។ ហើយក៏ដាក់ទៅវិញ។ នេះហើយគឺជាអ្វីតែមួយដែលជាតិសាសន៍នេះអាចធ្វើបាន។
\end{document}

MWE 2: XeTeXlinebreaklocale set to "km", line breaks work, however some ligatures are broken (text in red).

enter image description here

\documentclass[12pt]{article}

\XeTeXlinebreaklocale "km"
\usepackage{color}
\usepackage{fontspec}
\setmainfont[Ligatures=Required]{Khmer OS Content}

\begin{document}
មានមនុស្សម្នាក់ដែលបានធ្វើការមួយដែលអាចដឹងបាន នោះគឺជាកា{\color{red}ស្រ្តូ}នៅទីនេះ។ បាទ អ្នកទាំងអស់គ្នា។ គាត់បានចុះទៅទីនោះ។ ហើយពួកនាយទុនមានអស់ទាំងព្រនង់ ហើយនឹងអ្វីៗដែល… វាគឺឋិតនៅក្នុងស្តង់ដាមាស ដូចជាសហរដ្ឋអាមេរិកដែរ។ តើគាត់បានធ្វើអ្វី? គាត់បានទិញអស់ទាំងបណ្ណ័ភាគហ៊ុន។ គឺរកលុយតាមគ្រប់ទាំងវិធីដែលគាត់អាចធ្វើទៅបាន។ បន្ទាប់មកតើគាត់ធ្វើអ្វីទៀត? គាត់ក៏បានក្លែងបន្លំក្រដាស់ប្រាក់ ហើយបានប្ដូរវា។ ហើយក៏ដាក់ទៅវិញ។ នេះហើយគឺជាអ្វីតែមួយដែលជាតិសាសន៍នេះអាចធ្វើបាន។
\end{document}

I used TexShop 3.62 on Mac, typesetting it as XeLatex with XeTeX, Version 3.14159265-2.6-0.99996 (TeX Live 2016).

Is there a way to change our template so that both XeTeXlinebreaklocale and ligatures work?

  • Can't help much but if I get the font I can confirm the same happens in the latest xetex with texlive 2018 pretest – David Carlisle Apr 9 '18 at 18:41
  • @DavidCarlisle thanks for looking into this. The font we use is from the same place as the one you linked to. You can download the font we actually use here: drive.google.com/file/d/1YuoheWcKu9jS0cyZ-LU9g2ohicCMd5yX/… – Jo Hund Apr 9 '18 at 20:40
  • you might be better to ask on xetex list (xetex@tug.org) as this is going deep into the interaction between xetex's tex derived code and the harfbuzz font support, the manual gives no information, I looked around the sources a bit but unless you know the source layout better than I do working out how XeTeXlinebreaklocale works is somewhat taxing:-) – David Carlisle Apr 9 '18 at 20:46
  • @DavidCarlisle thanks for having a look. I will reach out to xetex@tug.org – Jo Hund Apr 9 '18 at 20:51
1

I got a solution from Jonathan Kew at the xetex@tug.org mailing list. He suggested to add \XeTeXinterwordspaceshaping=2 to our document, and this did indeed fix our problem.

I quote him:

The problem arises because activating XeTeXlinebreaklocale effectively makes xetex insert something like \penalty0 or \hskip0pt or similar (depending on the settings of \XeTeXlinebreakpenalty and ...skip) at each potential line-break position, so that the normal TeX line-breaking algorithm will be able to find and use these breaks.

But the inserted penalty and/or skip interrupts the sequence of characters that is passed to the OpenType shaping engine, and so features like ligatures will not work across the boundary.

A possible workaround would be to set \XeTeXinterwordspaceshaping=2 in your document. This will cause xetex to re-shape runs of text after line-breaking, and at this point your ligatures should work.

There are some caveats: in particular you'll notice if you try this that your red coloring of the example text fragment gets lost. This is because the \special{}s that \color inserts will be moved out of the run of text that is now being shaped as a unit. But depending on the needs of your documents, this may be an acceptable trade-off.

Oh, by the way: you can change the \XeTeXinterwordspaceshaping setting within the document if you like, but its effect does not respect the usual TeX scoping rules; if I remember correctly, \XeTeXinterwordspaceshaping=2 basically operates on a whole-page basis, so what matters is the value at the time the page is completed.

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.