Forbidding line breaks after specific symbols in inline math?

I'm writing a text in which order relations (e.g. <, \prec, \ll) often appear as arguments of functions rather than binary operators (e.g. $O(P,<)$). Every so often this leads to an inline formula's line breaking after the order relation and leaving a closing bracket or a comma (when there's another argument after the order relation) at the beginning of the next line. This, of course, I'd like to avoid.

I'm hoping for a more flexible solution than \relpenalty=10000 because I don't like it doing way more than is necessary and possibly messing with the way other formulas are typeset. I only really want to forbid line breaks after the symbols <, \prec, \ll as well as these symbols with a subscript or superscript added. However, I can't find a way to adjust the penalty for breaking after a specific symbol. Perhaps, there's a way of doing this by changing the \mathcode of said symbols but I can't put my finger on how exactly that could work.

Update. To be absolutely clear, I do not want to forbid the line break ad hoc at every instance but I want to do it a the preamble level. Also, if there is a way of forbidding line breaks before closing brackets and commas that would also work (perhaps that would even be better, although it wouldn't match the question's title, lol).

• Not tested. I think that if you change the category of such a relation to Ord, the breakability will be suppressed. This can be done by wrapping it in (grouping) braces: $O(P,{<})$. May 22 at 20:04
• @barbarabeeton Thanks! Is there a way of doing this at the preamble level? Is this some specific \mathcode, perhaps? (There seem to be plenty of ways of forbidding a line break at a given position in a given formula but I was really hoping to avoid doing that.) May 22 at 20:15
• You could give it a "name" -- \newcommand{\LT}{{<}} -- in the preamble, and then use that name instead of the symbol. The double braces are essential, and if this might appear in a "moving argument" (a caption or section name, for example), even triple braces might be a good idea. (Don't use \< for the name; that's already in use.) May 22 at 20:21
• There are some people here who might try to concoct an approach that would parse the string and probably check to see if the < is following a comma. My coding chops aren't good enough for that. I'd rather predict the possibility of a problem from the start, so that I wouldn't have to go back and change anything. Of course, using search-and-replace in a good editor, search on ,< and replace that by ,{<} (or with a space between comma and <) should take care of that in short order. May 22 at 20:40
• No problem. I just hope you don't use < very often as a real relation. May 22 at 22:53

This will prohibit line breaks after every occurrence of <, \prec and \ll, not just in the given context. You may add \penalty\relpenalty in case you have problems with line breaking in other contexts.

\documentclass{article}
\usepackage{amsmath}

% save the necessary commands and codes
\mathchardef\imakhlinless=\mathcode<
\mathchardef\imakhlinprec=\prec
\mathchardef\imakhlinll=\ll

% define a generic not breaking combination
% accepting subscripts and superscripts before
% issuing \nobreak
\NewDocumentCommand{\nobreaking}{me{^_}}{%
#1\IfValueT{#2}{^{#2}}\IfValueT{#3}{_{#3}}\nobreak
}

% renew the desired commands
\renewcommand{\prec}{\nobreaking\imakhlinprec}
\renewcommand{\ll}{\nobreaking\imakhlinll}
\newcommand{\aliasless}{\nobreaking\imakhlinless}
\begingroup\lccode~=< \lowercase{\endgroup\let~}\aliasless
\mathcode<="8000

\begin{document}

\parbox{0pt}{$O(P,<)$
$O(P,<_1)$
$O(P,\ll)$
$O(P,\prec_1^2)$
}

\end{document}


The \parbox with zero width is meant to force all possible line breaks.

• Thank you, this approach is better than mine because it does not affect the spacing. I noticed a couple of side effects though: (A) writing <' results in an awkward space between the "<" and the prime (but not for <^\prime or any other superscript); (B) more importantly, I can't use < and the other two as subscripts or superscripts anymore (unless I put them in braces). May 22 at 22:42
• Also, a minor comment: shouldn't you include xparse to use \NewDocumentCommand? May 23 at 1:55
• @imakhlin _< is wrong markup and should always be _{<}. Something like _x works only when it does, the standard example being _\notin. xparse is needed only for LaTeX prior to 2020-10-01 May 23 at 8:24
• Thanks for explaining! Not sure I follow the "something like _x works only when it does" part. I mean, I understand why _\notin doesn't work but how is _< any worse than _1? Or is the latter also bad markup? As for xparse, I can only say that the current default compiler at Overleaf (TeX Live 2020, pdfLatex) won't compile it without the inclusion. May 23 at 11:00

This is partially inspired by the comments of barbara beeton. One can actually use \mathcode to change the symbol class of <. I added the following to my preamble, it does the trick.

\mathcode<="013C
\let\relprec\prec
\renewcommand{\prec}{{\relprec}}
\let\relll\ll
\renewcommand{\ll}{{\relll}}


Here in the last 4 lines I simply redefine \prec to mean {\prec} and \ll to mean {\ll}, so that after these commands lines break as they do after Ord (class 0) symbols rather than Rel (class 3) symbols. This, however, does not work for < for the obvious reasons and instead I change its class to 0 via \mathcode in the first line. In 013C the first digit 0 is the new class, 1 is the cursive font (I can't use the usual family 0, because this leads to < being typeset as a "¡", fortunately, using cursive does not affect the shape of "<") and 3C is the character code of "<".

Of course, it is also possible to use \mathcode<="8000 for a more conceptual approach, as is done in egreg's answer. Also, as someone pointed out in a deleted comment, changing the class will mess with the spacing around the symbols when I actually use them as infix binary operators, although I can't say I mind too much because I always feel like these spaces are too big.

An alternative approach is the one I mentioned in the update: forbidding line breaks before commas and closing brackets. This is done by

\mathchardef\newbracket=\mathcode)
\mathcode)="8000
\begingroup
\catcode) \active
\gdef){\nolinebreak\newbracket}
\endgroup

\mathchardef\newcomma=\mathcode,
\mathcode,="8000
\begingroup
\catcode, \active
\gdef,{\nolinebreak\newcomma}
\endgroup


The upside of this solution for me personally is that it doesn't create difficulties with subscripts, since I do use _< and _\prec but not _) or _,. It also keeps the spacing as it is, unlike my other answer.

Of course, as I pointed out, this solves my problem but doesn't answer the precise question in the title. Also, I'm nowhere near as TeXnologically literate as most people who post answers here, so please let me know if there's something grossly inappropriate about the above code.