Ligature suppression algorithm fails if the word in question contains an Umlaut (dieresis) before the ligature

Question

This is a follow-up posting prompted by Taco Hoekwater's recent excellent answer to a question I posed a few months ago. It concerns the use of the ligature suppression algorithm for words (mainly, but not exclusively, German-language words) that contain Umlaute (diereses).

Taking Taco's MWE as the starting point (not reproduced below because it's quite long), add the instructions

\suppressligature{lffach}{lf|fach} %% no f-ligature in words containing the string "lffach"
\suppressligature{rflich}{rf|lich} %% ditto for words containing the string "rflich"
\suppressligature{mpfflug}{mpf|flug}

and the following six words (which all contain potential f-ligature glyphs):

elffach zwölffach                %% ff ligature to be suppressed
verwerflich dörflich             %% fl ligature to be suppressed
Kampfflugzeug Düsenkampfflugzeug %% ffl ligature to be replaced with f-fl

after \begin{document}. This results in the following output:

Notice that the three words which do not contain an Umlaut before the | ligature suppression point are treated correctly, i.e., the ff, fl, and ffl ligatures are broken up in the manner indicated by the applicable \suppressligature commands. In contrast, ligature suppression fails for the second set of three words -- apparently because they contain an Umlaut. This happens with words containing an ä Umlaut as well, by the way.

Furthermore, I've determined that this problem only appears to happen with words that contain an Umlaut before the ligature suppression point. Words that contain an Umlaut after the ligature suppression point indicated by a corresponding \suppressligature instruction are treated correctly. E.g., when supplying the instruction \suppressligature{uflös}{uf|lös}, the words auflösen and Auflösung are both rendered without the fl ligature.

A word about my TeX setup: I mostly run TeXLive2011 and use TeXWorks as my editor program. However, the problem also occurs under MiKTeX2.9, with all the latest updates installed, and with WinEdt7.0 as the editor. Both editors are fully unicode-aware.

Question: How does Taco's code need to be edited/modified to circumvent the problem(s) described in this posting?

Answer 1

The problem is that string.find returns a byte location, and in this case you want to have a character index when comparing nodes to strings. The solution is to replace

table.insert(throwliga,{match,n,k})

with

table.insert(throwliga,{prefix_length(f,match),n,k})

where prefix_length is defined like this:

  local prefix_length = function(word,byte)
      return unicode.utf8.len(string.sub(word,0,byte))
  end

Which simply returns the length in characters of the prefix of word that ends at byte.

Note: This change only fixes the current problem, it does not solve cases where there are unicode characters inside the strings in the noliga table. Please do not ask me to write that as well, there should be some challenge left ;)