# Unicode normalization and Hebrew in ConTeXt

I am typesetting a document in Hebrew that includes pointing (e.g., vowels, shin and sin dots, dagesh, etc.) using ConTeXt. The Hebrew text that I want to typeset has been normalized into Unicode's NFC canonical form. It is well-known that the Unicode canonical ordering of Hebrew points conflicts with the recommended mark ordering of specific points based on their functions (see https://www.sbl-site.org/Fonts/SBLHebrewUserManual1.5x.pdf for more on this topic). Thankfully, many typesetting engines automatically reorder the points to ensure that they are combined according to the specifications of many fonts. I'm pretty sure that XeLaTeX is one of these, as it typesets Hebrew letters with multiple points correctly even when the Hebrew text is in NFC form.

My question is, can ConTeXt with LuaTeX handle the same situation correctly? In the following minimal example, ConTeXt typesets pointed Hebrew correctly when the characters are in the typographically recommended order, but not when they are in Unicode canonical order:

``````%Setup Hebrew text font:
\definefontfeature[f:pointedhebrew][default][
ccmp=yes,
mark=yes,
script=hebr
]
\definefontfamily[hebrew] [rm] [SBL Hebrew] [features=f:pointedhebrew]
%Set the body font:
\setupbodyfont[hebrew]
%Set up right-to-left alignment:
\setupalign[r2l]
\starttext
%Characters after normalization, in Unicode canonical order (bet + segol + dagesh + final nun):
בֶּן

%A word with characters in typographically recommended order (bet + dagesh + segol + final nun):
בֶּן
\stoptext
``````

I typeset this using ConTeXt version 2020.03.10, as released with TeXLive 2020. I got the SBL Hebrew font from https://www.sbl-site.org/educational/BiblicalFonts_SBLHebrew.aspx. According to the font's user manual (see the link above the MWE), the font should be able to combine the marks to form the correct glyph regardless of their order after the consonant, but that doesn't seem to be the case here.

Is there an OpenType feature I need to enable in my `\definefontfeature` call to fix this, or is there some module or option I can include to get ConTeXt to typeset Unicode-normalized Hebrew as if it were ordered in the recommended way, like XeLaTeX does? I see that the uninormalize module is mentioned in XeLaTeX, LuaLaTeX, fontspec, unicode and normalization; can that be used with ConTeXt?

• Does it make a change when you apply the predefined `hebrew` featureset (i.e. `features=hebrew`) to the font which sets a few more options. Apr 24, 2020 at 6:28
• No, replacing `features=f:pointedhebrew` with `features=hebrew` in the MWE above does not change anything. Apr 24, 2020 at 12:19
• It's worth noting that some reordering of the characters may be happening; for instance, the (normalized) sequence shin + sheva + shin dot will be typeset as a shin with a shin dot, but no sheva, while the (typographically recommended) sequence shin + shin dot + sheva will be typeset correctly. But it could also be the result of the shin dot being in different combining class than the sheva or the result of the font having a dedicated shin with shin dot character and a rule for forming the ligature. Apr 26, 2020 at 1:31
• You should post your question on the ConTeXt mailing list where you can get help from Hans Hagen, the developer of ConTeXt. Apr 28, 2020 at 11:31
• Thanks! I have sent an e-mail with my question to the mailing list just now. If I'm able to get my MWE to work using that, I'll share it as an answer here. Do you happen to know what the usual turnaround time is for a question to appear on the mailing list? I sent one a week ago on a different topic, but it still hasn't shown up on the list. Apr 28, 2020 at 12:02

I posted this question on the ConTeXt mailing list (https://www.mail-archive.com/ntg-context@ntg.nl/), and from there I learned that ConTeXt did not have a mechanism implemented for reordering Hebrew points in a way that would work best with the substitution rules implemented in most Hebrew fonts. Hans Hagen graciously offered his help over the course of several days, and he added a reordering mechanism to the standard `hebrew` featureset. Now, the MWE in my question will work as expected if the font declaration is changed to `\definefontfamily[hebrew] [rm] [SBL Hebrew] [features=hebrew]`.