4

I was wondering if there's an automatic way to avoid getting short words at line edges -- specifically avoiding short words after punctuation at the right edge, and short words before punctuation at the left edge?

For example, if a sentence starts with I or If, or there's an and right after a comma, can we get latex to push it onto the next line automatically while still maintaining basic spacing rules? This question is similar, but it looks like the solution still requires a manual fix (unless I'm interpreting it wrong).

(I'm working in pdflatex)

  • What's an example where you want to do this (push word to next line) when the short word is before punctuation? Or do you mean squeeze on previous line in that case? How much badness (stretch / shrink of spaces) are you willing to tolerate? There's a chance that this may be possible in LuaTeX, but it seems the problem is still underspecified at the moment. – ShreevatsaR Jul 8 '17 at 1:28
  • Indeed, I've updated the question to be more specific. Ideally I'm looking for a solution that will balance this with the normal conditions, and, say, trade this for an extra hyphen in a long word rather than messing with line internal spacing too much. – Sam Zukoff Jul 8 '17 at 1:36
  • 2
    To me, this objective smells of "a bridge too far." Even if you managed this somehow, what about the rare case in which you do not want to prevent the line break, because the whole paragraph falls apart? I just liberally use ~ instead of a space wherever I feel like it. – Michael Palmer Jul 8 '17 at 1:37
  • That's certainly a good idea I'll keep in mind... but I've already got a couple hundred pages written and wanna see if I can achieve the desired result without going back and changing it by hand. Getting a solution here certainly isn't crucial -- file this under productive procrastination. – Sam Zukoff Jul 8 '17 at 1:44
  • 1
    The simplest solution may be to declare punctuation (dots and commas) as particularly good places to break (negative penalty, large in magnitude). If you want absolutely no chance of a short word though, it gets messier AFAICT. – ShreevatsaR Jul 8 '17 at 1:58
4

There are two goals here:

  • don't break after a short word that immediately follows punctuation,
  • don't break before a short word that immediately precedes punctuation,

subject to regular constraints of good line breaking.

One simple solution is to declare punctuation as particularly good places to break (a negative penalty, sufficiently large in magnitude). This will let TeX trade off trying to break at punctuation with its other line-breaking considerations (badness, demerits, other penalties), but will not guarantee that there are absolutely no breaks of that kind.

Here's a before-and-after, to illustrate:

before-after-annotated

As you can see,

  • In the first paragraph, the , it at the end of the third line has moved to the next line after the change.
  • In the second paragraph, the el. at the beginning of the fourth line and the at, at the beginning of the sixth line have moved to the previous line after the change.
  • The third paragraph has been included to show that this trick isn't a guarantee: the it. at the beginning of the fourth line remains there, because there's simply no way to fit it on the previous line.

This was achieved with:

\catcode`.=\active \def.{\char`.\penalty -200\relax}
\catcode`,=\active \def,{\char`,\penalty -200\relax}

in the following document:

\documentclass{article}
\begin{document}
\frenchspacing % Makes it easier
\hsize=20em
\parskip=10pt

% First, three paragraphs with the default settings
Lorem ipsum dolor sit amet, consectetur adipiscing elit. Ut blandit placerat justo, sed dictum sem. Donec erat elit, tincidunt non, it vel, tincidunt vehicula velit. Etiam pharetra ante at porta elementum. In nulla purus, faucibus non accumsan non, consequat eget.

Natis nulla blandit luctus tellus, sit amet posuere lacus maxius quis. In sit amet mattis est, a vehiula velit. Nam interum solicitudin el. In faucibus vulputate purus nec consectelur crass metus ipsum, blandit iln ullamcorpert at, portitor vita dolor. Duis sed mauris i inset inculis malesuada. Quisque laoret eu dui eget sage melittis corpum verborum.

Volutpat libero ac auctor. Donec semper, as id ultrices rhoncus, lectus nulla consequat nisi, ac sagitis risus lectus vel felis. Ut gravida it. Nam malesuada ante turpis eget. Ipsum factum verbum verdit.

\pagebreak

% Now the same text, with the meanings of . and , changed.
\catcode`.=\active \def.{\char`.\penalty -200\relax}
\catcode`,=\active \def,{\char`,\penalty -200\relax}

Lorem ipsum dolor sit amet, consectetur adipiscing elit. Ut blandit placerat justo, sed dictum sem. Donec erat elit, tincidunt non, it vel, tincidunt vehicula velit. Etiam pharetra ante at porta elementum. In nulla purus, faucibus non accumsan non, consequat eget.

Natis nulla blandit luctus tellus, sit amet posuere lacus maxius quis. In sit amet mattis est, a vehiula velit. Nam interum solicitudin el. In faucibus vulputate purus nec consectelur crass metus ipsum, blandit iln ullamcorpert at, portitor vita dolor. Duis sed mauris i inset inculis malesuada. Quisque laoret eu dui eget sage melittis corpum verborum.

Volutpat libero ac auctor. Donec semper, as id ultrices rhoncus, lectus nulla consequat nisi, ac sagitis risus lectus vel felis. Ut gravida it. Nam malesuada ante turpis eget. Ipsum factum verbum verdit.

% Change it back
\catcode`.=12 \catcode`,=12
\pagebreak

% Same text again, to show that nothing's permanently changed.
Lorem ipsum dolor sit amet, consectetur adipiscing elit. Ut blandit placerat justo, sed dictum sem. Donec erat elit, tincidunt non, it vel, tincidunt vehicula velit. Etiam pharetra ante at porta elementum. In nulla purus, faucibus non accumsan non, consequat eget.

Natis nulla blandit luctus tellus, sit amet posuere lacus maxius quis. In sit amet mattis est, a vehiula velit. Nam interum solicitudin el. In faucibus vulputate purus nec consectelur crass metus ipsum, blandit iln ullamcorpert at, portitor vita dolor. Duis sed mauris i inset inculis malesuada. Quisque laoret eu dui eget sage melittis corpum verborum.

Volutpat libero ac auctor. Donec semper, as id ultrices rhoncus, lectus nulla consequat nisi, ac sagitis risus lectus vel felis. Ut gravida it. Nam malesuada ante turpis eget. Ipsum factum verbum verdit.

\end{document}

Notes:

  • I wouldn't be surprised if changing the meanings of . and , like this breaks something. (In fact I was surprised that nothing got messed up in this example, then I realized that catcode changes don't apply to tokens that have already been read in.)
  • You can tweak the penalties: I used -200 just as an example, but anything from -1 to -9999 will have some effect. (In this example the threshold for all these changes to take effect seems to be -175, though one change happens even at -100.) A penalty that is ≤ -10000 forces a line break, which is not what you want.
  • You can do the same for more punctuation characters (?!:;) or have different penalties for different punctuation characters.
  • Things are a bit harder with \nonfrenchspacing (the default), where spaces are larger after punctuation. It may be doable but coming up with these examples was a lot of work so I haven't pursued it. Left as an exercise :-)
  • With LuaTeX you can even change the line-breaking algorithm, which would be a cool way to guarantee no short words at line edges (if that's what you need).

Edit: I couldn't resist implementing the “guaranteed” solution in LuaTeX. This version should work with both \frenchspacing and \nonfrenchspacing. What it does is to detect certain sequences and insert infinite (10000) penalties to prevent a break:

(punct, space, short_word, space) -> (punct, space, short_word, penalty, space)

and

(space, short_word, punct) -> (penalty, space, short_word, punct)

For the example above, this produces:

with LuaTeX

Note the overfull box in the last paragraph because the constraints are quite strict, but that's what we asked for. (Anyway, you probably won't have overfull boxes with wider and longer paragraphs, and you can fix them in the usual ways, of rewriting or adding \emergencystretch and so on.)

The code that produced the above (and even the idea) quite possibly has bugs in it that may even cause your LuaTeX compilation to crash, but here it is:

\documentclass{article}
\directlua{dofile("strict.lua")}
\begin{document}
\frenchspacing % Keeping same example as before
\hsize=20em
\parskip=10pt

Lorem ipsum dolor sit amet, consectetur adipiscing elit. Ut blandit placerat justo, sed dictum sem. Donec erat elit, tincidunt non, it vel, tincidunt vehicula velit. Etiam pharetra ante at porta elementum. In nulla purus, faucibus non accumsan non, consequat eget.

Natis nulla blandit luctus tellus, sit amet posuere lacus maxius quis. In sit amet mattis est, a vehiula velit. Nam interum solicitudin el. In faucibus vulputate purus nec consectelur crass metus ipsum, blandit iln ullamcorpert at, portitor vita dolor. Duis sed mauris i inset inculis malesuada. Quisque laoret eu dui eget sage melittis corpum verborum.

Volutpat libero ac auctor. Donec semper, as id ultrices rhoncus, lectus nulla consequat nisi, ac sagitis risus lectus vel felis. Ut gravida it. Nam malesuada ante turpis eget. Ipsum factum verbum verdit.
\end{document}

where strict.lua is:

function is_punct(n)
   if node.type(n.id) ~= 'glyph' then return false end
   if n.char > 127 then return false end
   c = string.char(n.char)
   if c == '.' or c =='?' or c == '!' or c == ':' or c == ';' or c == ',' then
      return true
   end
   return false
end

function no_punct_short_word_eol(head)
   -- Prevents having a line that ends like "<punctuation><space><short_word>"
   -- How we do this:
   --   (1) detect such short words (punct, space, short_word, space)
   --   (2) insert a penalty of 10000 between the short_word and the following space.
   -- More concretely:
   --   * A punctuation is one of .?!:;, which are the ones affected by \frenchspacing
   --   * A space is any glue node.
   --   * A short_word is a sequence of only glyph and kern nodes.
   -- So we maintain a state machine: default -> seen_punct -> seen_space -> seen_word
   -- where in the last state we maintain length. If we're in seen_word state and we see
   -- a glue, and length is less than threshold, insert a penalty before the glue.
   state = 'default'
   root = head
   while head do
      if state == 'default' then
         if is_punct(head) then
            state = 'seen_punct'
         end
      elseif state == 'seen_punct' then
         if node.type(head.id) == 'glue' then
            state = 'seen_space'
         else
            state = 'default'
         end
      elseif state == 'seen_space' then
         if node.type(head.id) == 'glyph' then
            state = 'seen_word'
            length = 1
         elseif is_punct(head) then
            state = 'seen_punct'
         else
            state = 'default'
         end
      elseif state == 'seen_word' then
         if node.type(head.id) == 'glue' and length <= 2 then
            -- Moment of truth
            penalty = node.new('penalty')
            penalty.penalty = 10000
            root, new = node.insert_before(root, head, penalty)
            -- TODO: Is 'head' invalidated now? Docs don't say anything...
            state = 'default'
         elseif node.type(head.id) == 'glyph' or node.type(head.id) == 'kern' then
            if node.type(head.id) == 'glyph' then length = length + 1 end
         else
            state = 'default'
         end
      else
         assert(false, string.format('Impossible state %s', state))
      end
      head = head.next
   end
   return root
end
luatexbase.add_to_callback('pre_linebreak_filter', no_punct_short_word_eol, 'Prevent short words after punctuation at end of sentence')

function no_bol_short_word_punct(head)
   -- Prevents having a line that starts like "<short_word><punctuation>"
   -- How we do this:
   --   (1) detect such short words (space, short_word, punct)
   --   (2) insert a penalty of 10000 between the space and the following short_word.
   -- More concretely:
   --   * A punctuation is one of .?!:;, which are the ones affected by \frenchspacing
   --   * A space is any glue node.
   --   * A short_word is a sequence of only glyph and kern nodes.
   -- So we maintain a state machine: default -> seen_space -> seen_word
   -- where in the last state we maintain length. If we're in seen_word state and we see
   -- a punct, and length is less than threshold, insert a penalty before the glue.
   -- Note that for this to work, we need to maintain a pointer to where we saw the glue.
   state = 'default'
   root = head
   before_space = nil
   while head do
      if state == 'default' then
         if node.type(head.id) == 'glue' then
            state = 'seen_space'
            before_space = head.prev
         end
      elseif state == 'seen_space' then
         if node.type(head.id) == 'glyph' then
            state = 'seen_word'
            length = 1
         else
            state = 'default'
         end
      elseif state == 'seen_word' then
         if is_punct(head) and length <= 2 then
            -- Moment of truth
            penalty = node.new('penalty')
            penalty.penalty = 10000
            root, new = node.insert_after(root, before_space, penalty)
            -- TODO: Is 'head' invalidated now? Docs don't say anything...
            state = 'default'
         elseif node.type(head.id) == 'glyph' or node.type(head.id) == 'kern' then
            if node.type(head.id) == 'glyph' then length = length + 1 end
         elseif node.type(head.id) == 'glue' then
            state = 'seen_space'
            before_space = head.prev
         else
            state = 'default'
         end
      else
         assert(false, string.format('Impossible state %s', state))
      end
      head = head.next
   end
   return root
end
luatexbase.add_to_callback('pre_linebreak_filter', no_bol_short_word_punct, 'Prevent short words at beginning of sentence before punctuation')
  • This seems like the perfect solution! ...but it seems to not be playing nice with citations (I'm using NatBib) and labels/references (I'm using hyperref) with the punctuation. For example, I have colons in my labels, and when I add the new spacing definition for colon, it spits out an error. This seems to hold for all the punctuation. Any thoughts on a work-around? – Sam Zukoff Jul 9 '17 at 19:29
  • @SamZukoff I don't know the full context of the error, but I can guess that the labels/citations expect a sequence of characters as their tokens, and having these penalties etc. in there probably messes them up. (You can try seeing whether \def:{\char`:} still throws the error, which may be illuminating at least.) Anyway, I don't have much thoughts on a workaround: you'll have to turn off these definitions in those contexts, or turn them on only for “normal” text, but I haven't thought about how to do that. – ShreevatsaR Jul 9 '17 at 19:40
  • The error messages (at least some of them) seem to have to do with csname. One example is given here: ! Missing \endcsname inserted. <to be read again> \char l.205 ... ...... (\ref{Gk:data1}a), ...... ... The control sequence marked <to be read again> should not appear between \csname and \endcsname. ! Extra \endcsname. \NR@setref ... \NR@@setref \csname r@#1\endcsname l.205 ... ...... (\ref{Gk:data1} ...... ... I'm ignoring this, since I wasn't doing a \csname. The same thing happens with just \def:{\char`:}. – Sam Zukoff Jul 9 '17 at 19:51
  • This document has citations and cross-references in just about every paragraph, so I probably won't bother with turning it and off all the time. But it looks like this is a great solution to the problem otherwise! – Sam Zukoff Jul 9 '17 at 19:55
  • 1
    Somebody downvoted this… I wonder why. A comment would be helpful. – ShreevatsaR Jul 10 '17 at 1:42

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.