10

With the new harfbuzz font engine in LuaTeX, I look for a solution to do glyph based fallback.

Say I have a CJK font without some latin letters, I would like to select the CJK font and my latin font as a fallback.

Currently I use virtual fonts for that purpose. First I fill a virtual font with each character in the latin font, then I override all the positions with the CJK font.

Now with the harfbuzz engine in LuaTeX, I don't think this approach is needed / possible.

These are the only steps I currently use with harfbuzz in LuaTeX:

local face = hb.new_face(f.filename)
local font = hb.new_font(face)
local buf = hb.Buffer.new()
buf:add_utf8(str)
hb.shape(font, buf, { language = "...", script = "...", direction = "ltr" },"")

What strategy for font fallback or emulating virtual fonts can I do?

  • you can test if the font contains a character using font:get_nominal_glyph(utf8 char) – michal.h21 Dec 11 '19 at 9:22
9

HarfBuzz does not have any built-in capacity for doing this, so you have to do it manually. Your example of having one latin font and one CJK font is quite common, but it actually combines multiple concepts: Different scripts and different fonts. Even if you only have a single font containing all glyphs, you have to split the text into Latin parts and CJK parts first and shape them separately. We will ignore this for now to avoid making this into an answer about Unicode script handling.

So let's assume for now that all characters have the same font or the glyphs that should be shape with CJK script are exactly the glyphs not contained in the Latin font. (This assumption is normally wrong, e.g. because of common punctuation signs, numbers, etc.)

You first need to know which codepoints are covered by each font. Depending on how you loaded your font, you might have created a table for this at load time. For this step, face:collect_unicodes() is your friend: It returns an array of all covered codepoints.

local face1 = hb.new_face(f1.filename)
local unicodes1 = face1:collect_unicodes()
local font1 = hb.new_font(face1)
local face2 = hb.new_face(f2.filename)
local unicodes2 = face2:collect_unicodes()
local font2 = hb.new_font(face2)

It will be easier to use if we have a table which contains true for all covered codepoints, so let's transform it:

local has_unicode1 = {}
for i=1,#unicodes1 do
  has_unicode1[unicodes1[i]] = true
end
-- And the same for font2, omitted here

Now you iterate over your text, determining for each character which font covers it. Here you have to decide what to do with codepoints not covered by any font. We will just use font 1 for them:

local lastfontchange = 1
local lastfont
for offset, codepoint in utf8.codes(str) do
  local currentfont = has_unicode1[codepoint] and 1
                   or has_unicode2[codepoint] and 2
                   or 1
  -- Did the font change?
  if lastfont ~= currentfont then
    if offset ~= 1 then -- We only have to do something if this isn't the first font we are dealing with
      -- Now shape the part between `lastfontchange` and `offset`
      local buf = hb.Buffer.new()
      buf:add_utf8(str:sub(lastfontchange, offset-1))
      if lastfont == 1 then
        hb.shape(font1, buf, { language = "...", script = "...", direction = "ltr" },"")
      else -- lastfont == 2 then
        hb.shape(font2, buf, { language = "...", script = "...", direction = "ltr" },"")
      end
      -- TODO: Do something with the shaped glyphs
    end
    lastfontchange = offset
    lastfont = currentfont
  end
end
-- We still need to shape everything after the last font change (This part probably should be made into it's own function to avoid the repetition):
local buf = hb.Buffer.new()
buf:add_utf8(str:sub(lastfontchange, #str))
if lastfont == 1 then
  hb.shape(font1, buf, { language = "...", script = "...", direction = "ltr" },"")
else -- lastfont == 2 then
  hb.shape(font2, buf, { language = "...", script = "...", direction = "ltr" },"")
end
-- TODO: Do something with the shaped glyphs
  • Interesting approach, I think this will do. I need to know much more about harfbuzz. Thank you! – topskip Dec 11 '19 at 9:34
  • 1
    Though this is the most common technique for determining when to use a fallback font, it has the disadvantage of not allowing HarfBuzz to use canonically equivalent characters supported by the font when the others are missing. A different approach is to shape with the first font, get all ranges when the glyph index returned by HarfBuzz is 0 and shape them with the second font and so on. – Khaled Hosny Dec 11 '19 at 13:07

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.