# Allow Latex to hyphenate words with dashes

I know from this and this question that LaTeX won't hyphenate a word that already contains a hyphen.

This can be circumvented by using the extdash package that provides a special replacement that can be used instead of a hyphen that won't stop the word from being hyphenated.

My question now is whether it is possible to make normal hyphens behave in that exact way. I don't want to have to write all my hyphens in a special way. Instead I want LaTeX to apply that behavior automatically (unless I use a special hyphen-command for that exact purpose).

Is this doable? And if yes: How?

• If there was a way to do this on all TeX engines, then the package you mention (extdash) would have probably got this feature already. Anyway, the thing you ask for is probably doable with LuaTeX, with no downsides. You can read the LuaTeX manual and give it a shot. :-) – ShreevatsaR Mar 1 '18 at 1:07
• With LuaTeX? Nice that's exactly what I'm using anyways! Will I find it in the manual directly or do I have to search for a specific keyword other than "dash" and/or "hyphenate"? :) – Raven Mar 1 '18 at 7:10
• See also tex.stackexchange.com/a/364550/2388 for a solution with \defaulthyphenchar. – Ulrike Fischer Mar 1 '18 at 21:03

## 2 Answers

Traditional typographical practice is to avoid hyphenating such words, so such a rule was coded by Knuth into the TeX program.

However, it is still possible to insert a hyphen that allows breaks, by fooling TeX into thinking we have two separate words (by inserting empty glue). This can be quite easily done if we're willing to type a special sequence like \Hyphdash or \-/ instead of a hyphen, which is what is provided by packages like extdash. But if we insist on typing regular hyphens into the TeX file, then “classical” TeX engines (meaning, other than LuaTeX) provide only one way of having regular hyphens be treated as something else, and that is to change the catcode of - to active, so that it becomes a macro that expands to something longer. But this would cause problems whenever - is used anywhere other than regular text, which is quite common (see examples of the problem in Steven's answer). I imagine that if a good solution existed for this in the classical TeX world, some package like extdash would have covered it.

(Actually, there's a perfectly sensible approach which for some reason is not very popular in the TeX world: and that is to preprocess your input file with some “smart” external script, replacing only the hyphens that occur in regular text with \Hyphdash (or its expansion) or whatever. But for some reason, people want to do everything inside TeX.)

With LuaTeX, which allows greater access to TeX internals, a lot of things become possible, and a lot of catcode and macro-related headaches are gone. For example, LuaTeX has a hyphenate callback which can be used to modify the regular hyphenation it does. This callback gets a list of nodes, into which we can insert discretionary nodes (hyphens) wherever we choose. This is much easier than having to do everything with macros (it is hard to set up appropriate input which later expands into exactly what we need, with no other interference). And there's even a lang.hyphenate function, which we can use to simply do what the regular hyphenation does.

The following seems to work: put the following in a file, say hyphenateall.lua:

function hyphenate_always(head, tail)
local n = head
while n do
if node.type(n.id) == 'glyph' and n.char == string.byte('-') then
-- Insert an infinite penalty before, and a zero-width glue node after, the hyphen.
-- Like writing "\nobreak-\hspace{0pt}" or equivalently "\penalty10000-\hskip0pt"
local p = node.new(node.id('penalty'))
p.penalty = 10000
head, p = node.insert_before(head, n, p)
local g = node.new(node.id('glue'))
head, g = node.insert_after(head, n, g)
n = g
end
n = n.next
end
lang.hyphenate(head, tail)
end

luatexbase.add_to_callback('hyphenate', hyphenate_always, 'Hyphenate even words containing hyphens')


and just put \directlua{require('hyphenateall')} in your .tex file.

For example (borrowing the example from Steven's answer):

\documentclass{article}
% Not needed -- used in this example just to show it's not needed :-)
\newcommand\?{\nobreak-\hspace{0pt}}
\begin{document}

\begin{minipage}[t]{0in}
\hbox{Before:} often hungry Works: often\?hungry What about: often-hungry
\end{minipage}\hspace{1in}%
\directlua{require('hyphenateall')}%
\begin{minipage}[t]{0in}
\hbox{After:} often hungry Works: often\?hungry What about: often-hungry
\end{minipage}

\end{document}


Note that after the \directlua call, we did not need the special syntax to get the extra hyphenation.

• Why did shower not hyphenate? – Steven B. Segletes Mar 1 '18 at 10:47
• Does hyphenateall.lua has to be located in the same directory as the .tex file or can it be put somewhere so that TeX will find it no matter which document it compiles? – Raven Mar 1 '18 at 13:24
• @Raven Yes, the Lua file can be put somewhere that works for all documents, by (1) changing dofile to require (done in the answer), and (2) putting the file in a path that is looked up; see answers to this question. – ShreevatsaR Mar 1 '18 at 17:40
• @StevenB.Segletes Actually, on my computer I don't get shower hyphenated with pdflatex either. So maybe I just have different hyphenation rules on my system, that are being used by both pdfTeX and LuaTeX. – ShreevatsaR Mar 1 '18 at 18:04
• It's weird. It's the 2nd time I was questioned on the hyphenation performed by my LaTeX installation. Apparantly, it also doesn't know how to hyphenate "meaning", tex.stackexchange.com/questions/415084/… – Steven B. Segletes Mar 1 '18 at 18:15

You can do it by changing the catcode of -, but that breaks all kind of stuff, since - is not only a dash but also a negative sign in math (as well as a dimensioning sign, as in \kern-1pt, \vspace{-\baselineskip}, \rule[-1pt]{1pt}{3pt}).

So this MWE below is a mere demonstration, not an actual recommendation. Shown in the MWE for two zero-width minipages, without then with active -.

\documentclass{article}
\let\svdash-
{\catcode-=\active %
\gdef-{\nobreak\svdash\nobreak\hspace{0pt}}
}
\begin{document}
\begin{minipage}[t]{0in}
\catcode-=12 %
First,
asteroid

Then,
shower

Last,
asteroid-shower
\end{minipage}\hspace{1in}%
\begin{minipage}[t]{0in}
\catcode-=\active %
First,
asteroid

Then,
shower

Last,
asteroid-shower
\end{minipage}
\end{document}


Therefore, the better approach would be to (as noted for the extdash package), assign a special macro to be this special type of hyphen. Here I choose \?, resulting in the identical output as the prior MWE:

\documentclass{article}
\newcommand\?{\nobreak-\nobreak\hspace{0pt}}
\begin{document}
\begin{minipage}[t]{0in}
First,
asteroid

Then,
shower

Last,
asteroid-shower
\end{minipage}\hspace{1in}%
\begin{minipage}[t]{0in}
First,
asteroid

Then,
shower

Last,
asteroid\?shower
\end{minipage}
\end{document}

• Wouldn't it be possible to "change the catcode back" when entering math mode? – Raven Feb 28 '18 at 15:52
• Sure, but negatives are used for dimensioning in all sorts of places outside of mathmode, e.g., \kern-1pt, \vspace{-\baselineskip}, \rule[-1pt]{1pt}{3pt}, etc. It would be easier to just change the \catcode at the beginning of a block that you know is normal text, and turn it off at the end of the text block. – Steven B. Segletes Feb 28 '18 at 15:55
• Ah yes of course... Totally forgot about those. – Raven Feb 28 '18 at 16:08
• Interesting: if the latter \nobreak is left out, then pdfTeX inserts a blank line for some reason. But if the \nobreak is kept, then LuaTeX honours it and does not break there. So neither option works in both engines! – ShreevatsaR Mar 1 '18 at 8:16
• The discrepancy between the engines is interesting (also, using -\penalty0\hskip0pt\relax` as recommended at tex.ac.uk/FAQ-nohyph.html gave wrong results in LuaTeX). May be a good question to ask on the site :-) – ShreevatsaR Mar 1 '18 at 8:38