# \- doesn't work in footnote

In the following example the \- in the footnote doesn't work. In Indian languages "dh" is one letter and should not be separated by hyphenation. I know I could use \discretionary here, but I want to understand why \- doesn't work. I have many cases like that.

\documentclass{memoir}
\usepackage[ruled,para*]{manyfoot}
\usepackage{quoting}
\usepackage{fontspec,xltxtra,xunicode}
\newcounter{versenum}
\DeclareNewFootnote[para]{A}[arabic]
\newcommand{\lem}{\textbf{]} \hspace{0.2em}} % TODO: textbf doesn't work yet.
\newcommand{\fn}[2]{\FootnotetextA{}{{\textbf{\theversenum#1}} #2}}
\newcommand\mynocolon{ }
\newcommand{\myversetmp}[1]{\begin{quoting}[rightmargin=0cm,vskip=0cm,indentfirst=false]\stepcounter{versenum}#1\end{quoting}}
\begin{document}
\setcounter{versenum}{31}
\fn{x}{xxxxxxxxxxxxxx xxxx xxx}
\myversetmp{umārdhaṃ vāmato khyātaṃ harārdhaṃ dakṣiṇe śṛṇu |\\
puṃsābharaṇasampanno yathoktalakṣaṇānvitaḥ || 32 ||%
\fn{a}{vāmato\lem incorrect sandhi, probably aiśa}\fn{b}{harār\-dhaṃ\lem em.; harārdha\mynocolon A}\fn{c}{pumsābharaṇasampanno\lem em.; pumāṃbharaṇasampaṃnā\mynocolon A}\fn{d}{yathoktalakṣaṇānvitaḥ\lem em.; yathoktaktalakṣaṇānvitam\mynocolon A}}%
\end{document}


Switching to bigfoot for the footnotes is not an option either, as the result looks very different, and not good: (just changed manyfoot for bigfoot in the above example.)

Update: This problem is not a problem of manyfoot only. I was now able to reproduce the same problem also with bigfoot, see below:

\documentclass{article}
\usepackage{fontspec}
\usepackage{bigfoot}
\DeclareNewFootnote[para]{A}[arabic]
\begin{document}
\footnoteA{XXXXXXXXX XXXXXXXXXXXXXXXX XXXXXXXXXXXXX XXXXXXX  harār\-dhaṃ}
\end{document}


• Don't load xltxtra and xunicode. – egreg Sep 17 '15 at 8:36
• You might try bigfoot instead of manyfoot. – egreg Sep 17 '15 at 9:17
• @egreg bigfoot's (default, not sure if that can be changed) behaviour for paragraphed footnotes is quite different from manyfoot's. It doesn't seem to like breaking an entry across lines, see the sample output above, which doesn't look good for a critical apparatus. – muk.li Sep 17 '15 at 9:55
• @egreg The problem also appears with bigfoot, see above. – muk.li Sep 18 '15 at 19:51
• perhaps mark-up as \mbox{harār}\-dhaṃ\lem. After all, you already added \- and \lem. – user4686 Aug 28 '18 at 16:20

Comment with a picture:

With polyglossia language set to sanskrit and script option set as [script=Latin], I can't get dh to hyphenate, whichever combinations of text widths, fonts and arrangements I try.

Is there a test file with more extensive cases, to try out all the hyphenations?

MWE

\documentclass{memoir}
\usepackage[ruled,para*]{manyfoot}
\usepackage{quoting}
\setlength{\textwidth}{1.5in}%to force hyphenation
\usepackage{fontspec}
\usepackage{polyglossia}
%\setmainfont{Noto Serif}
\setmainlanguage[script=Latin]{sanskrit}
\setotherlanguage{english}
\newfontfamily\englishfont{Noto Serif}

\newcounter{versenum}
\DeclareNewFootnote[para]{A}[arabic]
\newcommand{\lem}{\textbf{]} \hspace{0.2em}} % TODO: textbf doesn't work yet.
\newcommand{\fn}[2]{\FootnotetextA{}{{\textbf{\theversenum#1}} #2}}
\newcommand\mynocolon{ }
\newcommand{\myversetmp}[1]{\begin{quoting}[rightmargin=0cm,vskip=0cm,indentfirst=false]\stepcounter{versenum}#1\end{quoting}}
\begin{document}
\setcounter{versenum}{31}
\fn{x}{xxxxxxxxxxxxxx xxxx xxx}
\myversetmp{umārdhaṃ vāmato khyātaṃ harārdhaṃ dakṣiṇe śṛṇu |\\
puṃsābharaṇasampanno yathoktalakṣaṇānvitaḥ || 32 ||%
\fn{a}{vāmato\lem incorrect sandhi, probably aiśa}\fn{b}{harār\-dhaṃ\lem em.; harārdha\mynocolon A}\fn{c}{pumsābharaṇasampanno\lem em.; pumāṃbharaṇasampaṃnā\mynocolon A}\fn{d}{yathoktalakṣaṇānvitaḥ\lem em.; yathoktaktalakṣaṇānvitam\mynocolon A}}%

harārdhaṃ x harārdhaṃ x harārdhaṃ xxx x harārdhaṃ x b b b harārdhaṃ x harārdhaṃ x harārdhaṃ x harārdhaṃ x harārdhaṃ x harārdhaṃ xx harārdhaṃ x harārdhaṃ x harārdhaṃ x harārdhaṃ x harārdhaṃ x harārdhaṃ x harārdhaṃ xxxx harārdhaṃ x harārdhaṃ x harārdhaṃ x harārdhaṃ x harārdhaṃ x harārdhaṃ xx harārdhaṃ x harārdhaṃ x harārdhaṃ x harārdhaṃ x harārdhaṃ x harārdhaṃ x harārdhaṃ xxx harārdhaṃ x harārdhaṃ x harārdhaṃ x harārdhaṃ x harārdhaṃ x
\end{document}

• Do you want a hyphenation such as harārd-ham? That would be undesired, as dh is only the transliteration for the single letter ध. Generally you would hyphenate Sanskrit at the end of syllables, normally consisting of consonant[-cluster, but sometimes also no consonant], vowel and optionally a modifier such as ḥ or ṃ. So the possible places would be ha-rā-rdhaṃ. So the output you show seems fine to me. – muk.li Jun 30 at 11:21
• But then that is just the standard convention hyphenating transliterated Sanskrit as if it were written in an Indic script such as Devanāgarī. I see my old MWE had a harār-dhaṃ which is more a European hyphenation convention, not really standard. – muk.li Jun 30 at 11:30

The question didn't say explicitly, but the point is that an explicit discretionary hyphenation (\-) should suppress automatically chosen hyphenation in the word.

This is a general problem with xetex having some sort of "optimization" in which it throws away the discretionary information in the typeset words as it formats a paragraph: the hbox lines for the formatted paragraph contain no discretionaries, but the characters chosen from the discretionaries while breaking the paragraph into lines. If such a box is captured and emptied into another paragraph, xetex treats it like pristine text that gets hyphenated afresh, whereas TeX (and luatex and pdftex) retain the original discretionaries (and more discretionaries may be added in the second pass through paragraph building).

Here is a demonstration that shows the difference between the engines

\documentclass{article}
\pretolerance=-1

\begin{document}

\parbox{6cm}{Blathering demonstrates not\-a\-word consternation.%
{\tracingall\showlists}
\par
{\tracingall\showlists}
}

\end{document}


See how with xelatex the first display of the text contains the explicit discretionaries but they are missing from the final display, although the final display shows a hyphenation occurred (in "consternation"). (XeLaTeX also compacts the logging to a word per line, rather than a letter per line.) Contrarily, with pdflatex or lualatex, the initial horizontal list contains the explicit discretionaries and the final paragraph contains those same discretionaries as well as others based on hyphenation.

I noticed this some time back when XeLaTeX caused magaz.sty to break.

The solution in this case is to avoid xelatex, which also necessitates avoiding xltxtra and xunicode. Try it with lualatex instead.