Force LaTeX to break a nonsensical line with a hyphen

I have a long text containing many sequences of International Phonetic Alphabet (IPA) characters, and obviously they do not make up any meaningful English words.

I would like to insert this long text in my thesis, but because they do not consist meaningful words, the IPA sequences do not linebreak with hyphens.

Minimal code (XeLaTex):

\documentclass{article}
\usepackage{fontspec}
\setmainfont{XITS}

\begin{document}

∅ p b f v m t d s z n c ɟ ʃ ʒ ɲ k ɡ x h l r r̝ j [pbfvmtdszncɟʃʒɲkɡxhlrr̝j][pbtdɡfvszʃʒxjrr̝lmnɲ] [tdkvszxhjrr̝lpbɡfʃʒxmnɲ][pbtdkfvszʃʒxhr̝mnɲ][pbtdcɟkvsʃʒxhmnɲjr̝rl] [thlpbtdfvszʃʒmnɲ][tkszxhr̝dʃʒ][pbtdkvsʃxhmnɲ][vmnɲjr̝rl] [fvsz][tdsz]kvj

\end{document}


Result:

How can I force LaTeX to just break the long string anywhere with a hyphen? I can't break lines manually, because like I said, there are many sequences like this, so I can't go through every one of them manually.

• Adding hyphenation patterns for certain "words" using \hyphenation would also not be feasable, I guess? Aug 4, 2022 at 7:35
• @JasperHabicht Unfortunately no, because pretty much every long sequence is unique.
– Ian
Aug 4, 2022 at 7:37
• Aug 4, 2022 at 8:23
• I mean they could represent English words. But what are you doing here? Aug 4, 2022 at 17:13

It wasn't clear if you want a hyphen at the break, I show without and with - here, also it wasn't clear if the space between some ] [ was meaningful, I do nothing outside [...] here so ] [ allows a break (and shows a space if no break occurs) and ][ shows no space and does not allow a break.

\documentclass{article}
\usepackage{fontspec}
\setmainfont{XITS}
\def\x#1{#1\linebreak[0]\x}
\def\xstop#1\x{}
\begin{document}

∅ p b f v m t d s z n c ɟ ʃ ʒ ɲ k ɡ x h l r r̝ j [\x pbfvmtdszncɟʃʒɲkɡxhlrr̝j\xstop][\x pbtdɡfvszʃʒxjrr̝lmnɲ\xstop] [\x tdkvszxhjrr̝lpbɡfʃʒxmnɲ\xstop][\x pbtdkfvszʃʒxhr̝mnɲ\xstop][\x pbtdcɟkvsʃʒxhmnɲjr̝rl\xstop] [\x thlpbtdfvszʃʒmnɲ\xstop][\x tkszxhr̝dʃʒ\xstop][\x pbtdkvsʃxhmnɲ\xstop][\x vmnɲjr̝rl\xstop] [\x fvsz\xstop][\x tdsz\xstop]kvj

\def\x#1{#1\-\x}

∅ p b f v m t d s z n c ɟ ʃ ʒ ɲ k ɡ x h l r r̝ j [\x pbfvmtdszncɟʃʒɲkɡxhlrr̝j\xstop][\x pbtdɡfvszʃʒxjrr̝lmnɲ\xstop] [\x tdkvszxhjrr̝lpbɡfʃʒxmnɲ\xstop][\x pbtdkfvszʃʒxhr̝mnɲ\xstop][\x pbtdcɟkvsʃʒxhmnɲjr̝rl\xstop] [\x thlpbtdfvszʃʒmnɲ\xstop][\x tkszxhr̝dʃʒ\xstop][\x pbtdkvsʃxhmnɲ\xstop][\x vmnɲjr̝rl\xstop] [\x fvsz\xstop][\x tdsz\xstop]kvj

\end{document}

• (alternatively, manually add \linebreak[0] at every location that can be hyphenated. This code shows how to do that automatically, with some caveat e.g. cannot use any command, does not work in pdflatex with Unicode character) Aug 4, 2022 at 8:22
• @user202729 the OP said that every location can be hyphenated, you could do this in pdftex with a suitable loop eg tex.stackexchange.com/a/652894/1090 Aug 4, 2022 at 8:25
• I know . // by "cannot use any command" I mean e.g. if you insert \textbf{123} inside there would be a problem Aug 4, 2022 at 8:25
• @user202729 yes well I was trying to keep my loop overhead down to half a dozen tokens, Joseph's version can take more care over stepping over stuff:-) Aug 4, 2022 at 8:28
• Beware that this breaks combining characters. Aug 4, 2022 at 8:34

This adds \- between all characters, but care is taken not to do it too early (three leading characters and three trailing ones are not considered) and not to tamper with combining characters (from U+0301 up to U+036F).

You probably want a better name than \foo.

\documentclass[twocolumn]{article}
\usepackage{fontspec}
\setmainfont{XITS}

\ExplSyntaxOn
\NewDocumentCommand{\foo}{m}
{
[
% split the input at its items (characters)
\seq_set_split:Nnn \l_tmpa_seq { } { #1 }
% map the input, considering also the index of each item
% ##1 is the index, ##2 is the item
\seq_map_indexed_inline:Nn \l_tmpa_seq
{
% check that the character is not a combining one
% and that we're have at least three characters
% before and after
\bool_lazy_and:nnT
{
! \int_compare_p:n { "0301 <= ##2 <= "036F } % not a combining
}
{
\int_compare_p:n { 3 < ##1 < \seq_count:N \l_tmpa_seq - 1 } % we're in the middle
}
{ \- } % add a discretionary
% print the character
##2
}
]
}
\ExplSyntaxOff

\begin{document}

∅ p b f v m t d s z n c ɟ ʃ ʒ ɲ k ɡ x h l r r̝ j
\foo{pbfvmtdszncɟʃʒɲkɡxhlrr̝j}\foo{pbtdɡfvszʃʒxjrr̝lmnɲ}
\foo{tdkvszxhjrr̝lpbɡfʃʒxmnɲ}\foo{pbtdkfvszʃʒxhr̝mnɲ}\foo{pbtdcɟkvsʃʒxhmnɲjr̝rl}
\foo{thlpbtdfvszʃʒmnɲ}\foo{tkszxhr̝dʃʒ}\foo{pbtdkvsʃxhmnɲ}\foo{vmnɲjr̝rl}
\foo{fvsz}\foo{tdsz}kvj

%\parbox{0pt}{\hspace{0pt}\foo{pbfvmtdszncɟʃʒɲkɡxhlrr̝j} \foo{pbtdkvsʃxhmnɲ}}

\end{document}


Uncomment the \parbox line to see all the points where \- is added.

If you're free to use LuaLaTeX, it's straightforward to set up a Lua function that modifies the string so that line breaks are possibly everywhere. The code can handle pretty much all utf8-encoded characters -- they need not be ascii-encoded.

Please note that this approach does not generate hyphenation characters at line ends. In my view, the absence of hyphenation characters may actually be an advantage, since there's now no ambiguity as to whether a - character that happens to occur at the very end of a line is a hyphenation character inserted by LaTeX or is already present in the long string. If you do require a line to end with a hyphenation character, just change '%1\\hspace{0pt}' to '%1\\-' in the code below.

\documentclass{article}
\usepackage{fontspec}
\setmainfont{XITS}
\usepackage{luacode} % for 'luacode' env. and '\luastringN' macro
\begin{luacode}

function gentle_split ( s )
s = unicode.utf8.gsub ( s , '(.)' , '%1\\hspace{0pt}' )
tex.sprint ( s )
end

\end{luacode}
\newcommand{\GentleSplit}[1]{\directlua{gentle_split(\luastringN{#1})}}

\begin{document}

\noindent
\GentleSplit{∅ p b f v m t d s z n c ɟ ʃ ʒ ɲ k ɡ x h l r r̝ j [pbfvmtdszncɟʃʒɲkɡxhlrr̝j][pbtdɡfvszʃʒxjrr̝lmnɲ] [tdkvszxhjrr̝lpbɡfʃʒxmnɲ][pbtdkfvszʃʒxhr̝mnɲ][pbtdcɟkvsʃʒxhmnɲjr̝rl] [thlpbtdfvszʃʒmnɲ][tkszxhr̝dʃʒ][pbtdkvsʃxhmnɲ][vmnɲjr̝rl] [fvsz][tdsz]kvj}

\medskip
\noindent
\GentleSplit{[ɟʃʒɟʃʒɟʃʒɟʃʒɟʃʒɟʃʒɟʃʒɟʃʒɟʃʒɟʃʒɟʃʒɟʃʒɟʃʒɟʃʒɟʃʒɟʃʒɟʃʒɟʃʒɟʃʒɟʃʒɟʃʒɟʃʒɟʃʒɟʃʒɟʃʒɟʃʒɟʃʒɟʃʒɟʃʒɟʃʒɟʃʒɟʃʒɟʃʒɟʃʒɟʃʒɟʃʒɟʃʒɟʃʒɟʃʒɟʃʒɟʃʒɟʃʒɟʃʒɟʃʒɟʃʒɟʃʒɟʃʒɟʃʒɟʃʒɟʃʒɟʃʒɟʃʒɟʃʒɟʃʒɟʃʒɟʃʒɟʃʒɟʃʒɟʃʒɟʃʒɟʃʒɟʃʒɟʃʒɟʃʒɟʃʒɟʃʒɟʃʒɟʃʒɟʃʒɟʃʒɟʃʒɟʃʒɟʃʒɟʃʒɟʃʒɟʃʒɟʃʒɟʃʒɟʃʒɟʃʒɟʃʒɟʃʒɟʃʒɟʃʒɟʃʒɟʃʒɟʃʒɟʃʒɟʃʒɟʃʒɟʃʒɟʃʒ]}
\end{document}

• @DavidCarlisle - I've updated the code so that line breaks are permissible anywhere in the input string, even if the string isn't ascii-encoded.
– Mico
Aug 4, 2022 at 8:36

Still another approach based on luatex, which allows \patterns in the document. A new language is declared and a set of hyphenation patterns are then assigned to it.

\documentclass{article}

\newlanguage\ipalang
\begingroup
\language\ipalang
\patterns{1∅ 1p 1b 1f 1v 1m 1t 1d 1s 1z 1n 1c 1ɟ 1ʃ 1ʒ 1ɲ 1k 1ɡ 1x 1h
1l 1r 1r̝ 1j}
% \prehyphenchar=-1 % If you don’t want hyphens
\endgroup

\usepackage{fontspec}
\setmainfont{XITS}

\newcommand\ipatext[1]{{%
\language\ipalang
\lefthyphenmin=1
\righthyphenmin=1
#1}}

\begin{document}

\hsize1pt

Start paragraph
\ipatext{[vmtdszncɟʃʒɲkɡxhlrr̝j][dɡfvszʃʒxjrr̝lmnɲ]}
End paragraph.

\end{document}
`