Automatically adjust interletter spacing (tracking) in XeTeX

Question

Background: We know that getting fully justified text, without resorting to hyphenation, is a hard problem and invariably requires awful spaces. The situation would be slightly improved if inter-letter spacing could also be increased and not just inter-word spacing. Here is an illustration from the soul documentation:

3.5 Dirty tricks

Narrow columns are hard to set, because they don’t allow much spacing flexibility, hence long words often cause overfull boxes. A macro could use \so to insert stretchability between the single characters. Table 2 shows…

The middle one is what I want (won't have so much letterspacing though!), in XeTeX. (Have to use XeTeX because the document requires OpenType fonts and preferably Unicode input, and LuaTeX does not support the script I need.) I'd like it to apply to the entire book, without having to surround each paragraph with \foo{...}.

Here's a document:

\documentclass{article}
\usepackage{fontspec}
\usepackage{polyglossia}
\setmainlanguage{kannada}
\setmainfont{Noto Sans Kannada}

\begin{document}

\hsize=300pt
\hyphenpenalty=10000
\emergencystretch=100em

ಶ್ರೀವನಿತೆಯರಸನೆ ವಿಮಲ ರಾಜೀವ ಪೀಠನ ಪಿತನೆ ಜಗಕತಿ ಪಾವನನೆ ಸನಕಾದಿ
ಸಜ್ಜನನಿಕರ ದಾತಾರ. ರಾವಾಣಾಸುರ ಮಥನ ಶ್ರವಣ ಸುಧಾ ವಿನೂತನ ಕಥನ,
ಕಾರಣ ಕಾವುದಾನತ ಜನವ. ಗದುಗಿನ ವೀರನಾರಯಣ ಶರಣಸಂಗವ್ಯಸನ.
ಭುಜಗಾಭರಣನಮರ ಕಿರೀಟ ಮಂಡಿತಚರಣ ಚಾರುಚರಿತ್ರ. ನಿರುಪಮ ಭಾಳಶಿಖಿನೇತ್ರ
ಕರಣನಿರ್ಮಲ ಭಜಕರಘ. ಸಂಹರಣ ದಂತಿ. ಚಮೂರು ಚರ್ಮಾಂಬರನೆ
ಸಲಹುಗೆ ಭಕುತ ಜನರನು ಪಾರ್ವತೀರಮಣ. ವರಮಣಿಗಳಿಂದೆಸೆವ ಮೌಳಿಯ
ಸರಸಿಜಾರಿಯ ಕಿರಣದೋಳಿಯ ವಿರಚಿಸಿದ ಸಿಂಧೂರಭಾಳದಿ ಕುಣಿವ ಕುಂತಳದ
ಕರಿ ನಿಭಾಕೃತಿಯೆನಿಪ ವದನದ ಕರದ ಪಾಶದ ಮೋದಕದ ವಿಸ್ತರದ ಗಣಪತಿ
ಮಾಡೆಮಗೆ ನಿರ್ವಿಘ್ನದಾಯಕವ.

\end{document}

which produces

Note the wide spaces especially on line 6 of 9. I'd like to decrease these spaces, by slightly increasing the spaces between the letters on that line (letterspacing aka tracking).

In general, I'd like to increase the spaces between the letters when a line is underfull, by an amount up to some limit I consider aesthetically acceptable, but keep them at their normal values by default. (That is, stretch only when necessary, and only up to some limit.) I know people will have mixed opinions about the typographic propriety of doing this, but in this case I'm trying to match the look of an existing book and that's what the publisher seems to have done.

All the questions I could find on this site are about indiscriminately and persistently increasing letter spacing:

• Enlarging tracking (= letter spacing)
• Interletter spacing in plain TeX
• LetterSpace (tracking) of monospace fonts in Xetex/fontspec etc.

I have read the TeX FAQ on letterspacing and looked at letterspacing.tex (documented in TUGboat), at microtype and at soul, in all of which I found only “increase letterspacing globally” and not the feature I was looking for (except for the above note from the soul documentation).

Answer 1

microtype helps a bit but you can add space via character classes:

\documentclass{article}
\usepackage{fontspec}
\usepackage{polyglossia}
\setmainlanguage{kannada}
\setmainfont{NotoSansKannada-Regular.ttf}
\XeTeXinterchartokenstate=1
\newXeTeXintercharclass\Kclass
\newXeTeXintercharclass\Kclassb
\makeatletter
\count@"00C80
\loop
\XeTeXcharclass\count@=\Kclass
\ifnum\count@<"00D00
\advance\count@\@ne
\repeat

\XeTeXcharclass"0C81=\Kclassb
\XeTeXcharclass"0C82=\Kclassb
\XeTeXcharclass"0C83=\Kclassb
\XeTeXcharclass"0CBC=\Kclassb
\XeTeXcharclass"0CBE=\Kclassb
\XeTeXcharclass"0CBF=\Kclassb
\XeTeXcharclass"0CC0=\Kclassb
\XeTeXcharclass"0CC1=\Kclassb
\XeTeXcharclass"0CC2=\Kclassb
\XeTeXcharclass"0CC3=\Kclassb
\XeTeXcharclass"0CC4=\Kclassb
\XeTeXcharclass"0CC6=\Kclassb
\XeTeXcharclass"0CC7=\Kclassb
\XeTeXcharclass"0CC8=\Kclassb
\XeTeXcharclass"0CCA=\Kclassb
\XeTeXcharclass"0CCB=\Kclassb
\XeTeXcharclass"0CCC=\Kclassb
\XeTeXcharclass"0CCD=0 % Don't insert hskip before or after this character
\XeTeXcharclass"0CD5=\Kclassb
\XeTeXcharclass"0CD6=\Kclassb
\XeTeXcharclass"0CE2=\Kclassb
\XeTeXcharclass"0CE3=\Kclassb
\XeTeXcharclass"0CF1=\Kclassb
\XeTeXcharclass"0CF2=\Kclassb

\XeTeXinterchartoks\Kclass\Kclass{\nobreak\hskip0pt plus .5pt\relax}
\XeTeXinterchartoks\Kclassb\Kclass{\nobreak\hskip0pt plus .5pt\relax}
\makeatother

\usepackage{microtype}
\begin{document}
\XeTeXinterchartokenstate=1
\hsize=300pt
\hyphenpenalty=10000
\emergencystretch=100em

ಶ್ರೀವನಿತೆಯರಸನೆ ವಿಮಲ ರಾಜೀವ ಪೀಠನ ಪಿತನೆ ಜಗಕತಿ ಪಾವನನೆ ಸನಕಾದಿ
ಸಜ್ಜನನಿಕರ ದಾತಾರ. ರಾವಾಣಾಸುರ ಮಥನ ಶ್ರವಣ ಸುಧಾ ವಿನೂತನ ಕಥನ,
ಕಾರಣ ಕಾವುದಾನತ ಜನವ. ಗದುಗಿನ ವೀರನಾರಯಣ ಶರಣಸಂಗವ್ಯಸನ.
ಭುಜಗಾಭರಣನಮರ ಕಿರೀಟ ಮಂಡಿತಚರಣ ಚಾರುಚರಿತ್ರ. ನಿರುಪಮ ಭಾಳಶಿಖಿನೇತ್ರ
ಕರಣನಿರ್ಮಲ ಭಜಕರಘ. ಸಂಹರಣ ದಂತಿ. ಚಮೂರು ಚರ್ಮಾಂಬರನೆ
ಸಲಹುಗೆ ಭಕುತ ಜನರನು ಪಾರ್ವತೀರಮಣ. ವರಮಣಿಗಳಿಂದೆಸೆವ ಮೌಳಿಯ
ಸರಸಿಜಾರಿಯ ಕಿರಣದೋಳಿಯ ವಿರಚಿಸಿದ ಸಿಂಧೂರಭಾಳದಿ ಕುಣಿವ ಕುಂತಳದ
ಕರಿ ನಿಭಾಕೃತಿಯೆನಿಪ ವದನದ ಕರದ ಪಾಶದ ಮೋದಕದ ವಿಸ್ತರದ ಗಣಪತಿ
ಮಾಡೆಮಗೆ ನಿರ್ವಿಘ್ನದಾಯಕವ.

\XeTeXinterchartokenstate=0


ಶ್ರೀವನಿತೆಯರಸನೆ ವಿಮಲ ರಾಜೀವ ಪೀಠನ ಪಿತನೆ ಜಗಕತಿ ಪಾವನನೆ ಸನಕಾದಿ
ಸಜ್ಜನನಿಕರ ದಾತಾರ. ರಾವಾಣಾಸುರ ಮಥನ ಶ್ರವಣ ಸುಧಾ ವಿನೂತನ ಕಥನ,
ಕಾರಣ ಕಾವುದಾನತ ಜನವ. ಗದುಗಿನ ವೀರನಾರಯಣ ಶರಣಸಂಗವ್ಯಸನ.
ಭುಜಗಾಭರಣನಮರ ಕಿರೀಟ ಮಂಡಿತಚರಣ ಚಾರುಚರಿತ್ರ. ನಿರುಪಮ ಭಾಳಶಿಖಿನೇತ್ರ
ಕರಣನಿರ್ಮಲ ಭಜಕರಘ. ಸಂಹರಣ ದಂತಿ. ಚಮೂರು ಚರ್ಮಾಂಬರನೆ
ಸಲಹುಗೆ ಭಕುತ ಜನರನು ಪಾರ್ವತೀರಮಣ. ವರಮಣಿಗಳಿಂದೆಸೆವ ಮೌಳಿಯ
ಸರಸಿಜಾರಿಯ ಕಿರಣದೋಳಿಯ ವಿರಚಿಸಿದ ಸಿಂಧೂರಭಾಳದಿ ಕುಣಿವ ಕುಂತಳದ
ಕರಿ ನಿಭಾಕೃತಿಯೆನಿಪ ವದನದ ಕರದ ಪಾಶದ ಮೋದಕದ ವಿಸ್ತರದ ಗಣಪತಿ
ಮಾಡೆಮಗೆ ನಿರ್ವಿಘ್ನದಾಯಕವ.

\end{document}

Here I have given the whole range the class Kclass, but then re-assigned combining characters to class Kclassb so that space is not inserted between the base and the combining character. It is hopefully more readable than the first version that only used one class, but as I can't read it, it is hard to be sure. More classes could be used if some characters could have bigger spaces than others.