Artificially improving default TeX algo results for balancing interword spacing

Question

I'm trying to build rectangular paragraphs. That is, paragraphs which are both left as well as right aligned. For large paragraphs I can tell TeX I'd like a rectangular end result.

Consider the below example,

\documentclass{article}
\usepackage{microtype}
\usepackage[numbers]{kantlipsum}

\begin{document}

\parindent=0pt
\parfillskip=0pt
\parskip=6pt % to show paragraph breaks

\kant[1-160]

\end{document}

As pointed out by egreg, this (simple) example document produces the desired rectangular paragraphs (only paragraph 154 is not perfectly rectangular, which can be adjusted using a higher \tolerance) and uses \parfillskip=0pt to get it done. For smaller paragraphs issues arise and mostly the last line of the paragraph is likely to produce aesthetically displeasing results.

I'd like to explore how this can be improved on, starting with the option of artificially increasing or decreasing the interword spacing, potentially on a paragraph-by-paragraph basis. Steven B. Segletes mentioned there may be some possibilities towards getting it done, but I'm open to suggestions in general. Naturally, e.g. extreme fine-tuning of each paragraph would be an option, that is, adjust tracking on a paragraph-by-paragraph basis. A more generally applicable and/or automated solution would obviously have preference.

Answer 1

See ADDENDUM, if one permits a solution that allows solutions utilizing both letter-spacing as well as inter-word spacing.

At this stage, this result is manual, in that a pt value of per word space insertion or deletion is required to optimize the result. Also, this simple algorithm is not able to handle macros in the text, though I suppose if one wanted to go down this path, it could be made more robust.

The approach is to parse the input word by word and following each word, kern in the extra (manually specified) space, then an \hfill, and finally a regular \ space.

In this MWE, I take the Gettysburg address, either in full, or truncated sentence by sentence. I show results both expanding it out to the right flush the last line, or compressing it to remove the last partial line. As it was pointed out in the chat room, short sentences are hard to make look good. The syntax I use is \comparepars{expansion}{compression}{text}.

Hopefully, you could use this manual approach to determine whether or not it is worth generalizing for your needs. If the result of rectangularizing a paragraph is not what you were expecting, then you determined that at little cost.

\documentclass{article}
\usepackage{tabto}
\def\theLetterSpace{0pt}
\newcommand\spaceout[2][\theLetterSpace]{%
  \edef\tmp{#2}%
  \def\LocalLetterSpace{#1}\expandafter\spaceouthelpA#2 \relax\relax}
\def\spaceouthelpA#1 #2\relax{%
  #1\ifx\relax#2\else%
  \kern\LocalLetterSpace\hfill\ \spaceouthelpA#2\relax\fi%
}
\newcommand\comparepar[2]{%
ORIGINAL:\par#2\par 
SPACED OUT:\par\spaceout[#1]{#2}\par
\clearpage%
}
\newcommand\comparepars[3]{%
ORIGINAL:\par#3\par 
SPACED OUT:\par\spaceout[#1]{#3}\par
SPACED IN:\par\spaceout[#2]{#3}\par
\clearpage%
}
\def\sA{Four score and seven years ago our fathers brought forth on this continent a new nation, conceived in liberty, and dedicated to the proposition that all men are created equal.}
\def\sB{Now we are engaged in a great civil war, testing whether that nation, or any nation so conceived and so dedicated, can long endure. We are met on a great battlefield of that war.}
\def\sC{We have come to dedicate a portion of that field, as a final resting place for those who here gave their lives that that nation might live.}
\def\sD{It is altogether fitting and proper that we should do this.}
\def\sE{But, in a larger sense, we can not dedicate, we can not consecrate, we can not hallow this ground.}
\def\sF{The brave men, living and dead, who struggled here, have consecrated it, far above our poor power to add or detract.}
\def\sG{The world will little note, nor long remember what we say here, but it can never forget what they did here.}
\def\sH{It is for us the living, rather, to be dedicated here to the unfinished work which they who fought here have thus far so nobly advanced.}
\def\sI{It is rather for us to be here dedicated to the great task remaining before us—--that from these honored dead we take increased devotion to that cause for which they gave the last full measure of devotion—that we here highly resolve that these dead shall not have died in vain—that this nation, under God, shall have a new birth of freedom—and that government of the people, by the people, for the people, shall not perish from the earth.}
\begin{document}
\parindent 0pt
\noindent\hrulefill

\edef\txt{\sA\ \sB\ \sC\ \sD\ \sE\ \sF\ \sG\ \sH\ \sI}
\comparepars{.3pt}{-1.1pt}{\txt}

\edef\txt{\sA\ \sB\ \sC\ \sD\ \sE\ \sF\ \sG\ \sH}
\comparepar{0pt}{\txt}

\edef\txt{\sA\ \sB\ \sC\ \sD\ \sE\ \sF\ \sG}
\comparepars{1.3pt}{-.8pt}{\txt}

\edef\txt{\sA\ \sB\ \sC\ \sD\ \sE\ \sF}
\comparepars{.05pt}{-2.6pt}{\txt}

\edef\txt{\sA\ \sB\ \sC\ \sD\ \sE}
\comparepars{1.9pt}{-1.2pt}{\txt}

\edef\txt{\sA\ \sB\ \sC\ \sD}
\comparepars{-.04pt}{-.04pt}{\txt}

\edef\txt{\sA\ \sB\ \sC}
\comparepars{2pt}{-1.6pt}{\txt}

\edef\txt{\sA\ \sB}
\comparepars{1.5pt}{-3.1pt}{\txt}

\edef\txt{\sA}
\comparepars{8pt}{-4pt}{\txt}

\end{document}

Here it is on a longer passage, in which expanding it to fill the line or compressing it to one less line looks not too bad.

But here it is on a single sentence. As you can see either expanding it to fill the line or compressing it to one less line leaves a result that is not particularly desirable.

---

ADDENDUM

Here is the final result (single sentence input) using the modified algorithm described below. Compare it to the above result that uses only the inter-word spacing approach.

If inter-letter spacing is allowed as part of the solution, here is a comparable approach to above, but which also compresses/expands letterspacing within the specified bounds of

\def\MaxLetterSpace{0.5pt}
\def\MinLetterSpace{-0.5pt}

Inter-word spacing changes may exceed this, but letterspacing changes are limited by these parameters.

The result, in my view, looks better because it can achieve the flushed result with smaller overall changes (for example, compare the expand/squeeze parameters between the two versions), and because the contrast between word and letter spacing is made less stark.

\documentclass{article}
\usepackage{xcolor}
\def\theLetterSpace{0.0pt}
\def\theWordSpace{1.5\LocalLetterSpace}
\def\MaxLetterSpace{0.5pt}
\def\MinLetterSpace{-0.5pt}
\newlength\LocalLetterSpace
\newcommand\spaceout[2][\theLetterSpace]{%
  \ifdim#1>\MaxLetterSpace\setlength\LocalLetterSpace{\MaxLetterSpace}%
  \else\ifdim#1<\MinLetterSpace\setlength\LocalLetterSpace{\MinLetterSpace}%
    \else\setlength\LocalLetterSpace{#1}\fi%
  \fi
  \expandafter\spaceouthelpA#2 \relax\relax}
\def\spaceouthelpA#1 #2\relax{%
  \spaceouthelpB#1\relax\relax%
  \ifx\relax#2\else\kern\theWordSpace\hfill\ \spaceouthelpA#2\relax\fi
}
\def\spaceouthelpB#1#2\relax{%
  #1%
  \ifx\relax#2\else
    \kern\LocalLetterSpace\spaceouthelpB#2\relax%
  \fi
}

\newcommand\comparepars[3]{%
ORIGINAL:\par#3\par 
\def\theWordSpace{#1}%
SPACED OUT:\par\spaceout[#1]{#3}\par
\def\theWordSpace{#2}%
SPACED IN:\par\spaceout[#2]{#3}\par
\clearpage%
}
\def\sA{Four score and seven years ago our fathers brought forth on this continent a new nation, conceived in liberty, and dedicated to the proposition that all men are created equal.}
\def\sB{Now we are engaged in a great civil war, testing whether that nation, or any nation so conceived and so dedicated, can long endure. We are met on a great battlefield of that war.}
\def\sC{We have come to dedicate a portion of that field, as a final resting place for those who here gave their lives that that nation might live.}
\def\sD{It is altogether fitting and proper that we should do this.}
\def\sE{But, in a larger sense, we can not dedicate, we can not consecrate, we can not hallow this ground.}
\def\sF{The brave men, living and dead, who struggled here, have consecrated it, far above our poor power to add or detract.}
\def\sG{The world will little note, nor long remember what we say here, but it can never forget what they did here.}
\def\sH{It is for us the living, rather, to be dedicated here to the unfinished work which they who fought here have thus far so nobly advanced.}
\def\sI{It is rather for us to be here dedicated to the great task remaining before us—--that from these honored dead we take increased devotion to that cause for which they gave the last full measure of devotion—that we here highly resolve that these dead shall not have died in vain—that this nation, under God, shall have a new birth of freedom—and that government of the people, by the people, for the people, shall not perish from the earth.}

\begin{document}

\parindent 0pt
\noindent\hrulefill

\edef\txt{\sA\ \sB\ \sC\ \sD\ \sE\ \sF\ \sG\ \sH\ \sI}
\comparepars{.04pt}{-.22pt}{\txt}

\edef\txt{\sA\ \sB\ \sC\ \sD\ \sE\ \sF\ \sG\ \sH}
\comparepars{0pt}{-.43pt}{\txt}

\edef\txt{\sA\ \sB\ \sC\ \sD\ \sE\ \sF\ \sG}
\comparepars{0.29pt}{-.16pt}{\txt}

\edef\txt{\sA\ \sB\ \sC\ \sD\ \sE\ \sF}
\comparepars{.03pt}{-0.75pt}{\txt}

\edef\txt{\sA\ \sB\ \sC\ \sD\ \sE}
\comparepars{0.3pt}{-.3pt}{\txt}

\edef\txt{\sA\ \sB\ \sC\ \sD}
\comparepars{-0.03pt}{-1.8pt}{\txt}

\edef\txt{\sA\ \sB\ \sC}
\comparepars{.4pt}{-.28pt}{\txt}

\edef\txt{\sA\ \sB}
\comparepars{.35pt}{-1.4pt}{\txt}

\edef\txt{\sA}
\comparepars{5pt}{-.8pt}{\txt}
\end{document}

As the paragraphs to be operated on become longer, the results quickly look even better:

Answer 2

It isn't clear from your question what effect you need other than setting \parindent and \parfillskip to zero to remove the white space at the start and end.

I borrowed Stephen's text to make comparison easier, each version of the text is set 4 times, justified as normal, then again with no parfillskip and then trying again to be one line longer or shorter.

\documentclass{article}
\parindent=0pt
\emergencystretch\textwidth

\def\x{\clearpage
\bigskip
\textbf{ORIGINAL}\par{\txt\par}
\bigskip
\textbf{A}\par{\parfillskip=0pt \txt\par}
\bigskip
\textbf{B}\par{\looseness=1 \parfillskip=0pt \txt\par}
\bigskip
\textbf{C}\par{\looseness=-1 \parfillskip=0pt \txt\par}
}

\def\sA{Four score and seven years ago our fathers brought forth on this continent a new nation, conceived in liberty, and dedicated to the proposition that all men are created equal.}
\def\sB{Now we are engaged in a great civil war, testing whether that nation, or any nation so conceived and so dedicated, can long endure. We are met on a great battlefield of that war.}
\def\sC{We have come to dedicate a portion of that field, as a final resting place for those who here gave their lives that that nation might live.}
\def\sD{It is altogether fitting and proper that we should do this.}
\def\sE{But, in a larger sense, we can not dedicate, we can not consecrate, we can not hallow this ground.}
\def\sF{The brave men, living and dead, who struggled here, have consecrated it, far above our poor power to add or detract.}
\def\sG{The world will little note, nor long remember what we say here, but it can never forget what they did here.}
\def\sH{It is for us the living, rather, to be dedicated here to the unfinished work which they who fought here have thus far so nobly advanced.}
\def\sI{It is rather for us to be here dedicated to the great task remaining before us―--that from these honored dead we take increased devotion to that cause for which they gave the last full measure of devotion―that we here highly resolve that these dead shall not have died in vain―that this nation, under God, shall have a new birth of freedom―and that government of the people, by the people, for the people, shall not perish from the earth.}
\begin{document}
\parindent 0pt
\noindent\hrulefill

\edef\txt{\sA\ \sB\ \sC\ \sD\ \sE\ \sF\ \sG\ \sH\ \sI}
\x

\edef\txt{\sA\ \sB\ \sC\ \sD\ \sE\ \sF\ \sG\ \sH}
\x

\edef\txt{\sA\ \sB\ \sC\ \sD\ \sE\ \sF\ \sG}
\x

\edef\txt{\sA\ \sB\ \sC\ \sD\ \sE\ \sF}
\x

\edef\txt{\sA\ \sB\ \sC\ \sD\ \sE}
\x

\edef\txt{\sA\ \sB\ \sC\ \sD}
\x

\edef\txt{\sA\ \sB\ \sC}
\x

\edef\txt{\sA\ \sB}
\x

\edef\txt{\sA}
\x

\end{document}