Deciding Which Glyphs to Use in Font Encoding

Question

I wanted to use one of my favorite fonts, Crimson, while still maintaining full support from the microtype package. Since XeTeX and LuaTeX are not fully supported by the microtype package (as of version 2.5, beta 08, "adjustment of interword spacing and of kerning only works with pdfTeX (≥ 1.40)"), I was forced to convert the Crimson font files from the OTF format to the Type 1 format used by (pdf)TeX. In the process of using otftotfm to convert the font files to Type1 format, I encountered some concerns that I would like to address.

• When I converted the OTF files to the Type 1 format using T1 encoding, otftotfm displayed the following warning.

$ sudo otftotfm -e ec -fcalt -fdlig -ffrac -fkern -fonum -fordn -fpnum -fsinf -fsmcp -fsubs -fsups -fzero --vendor sebastiankosch Crimson-Roman.otf T1--Crimson--Roman

...
otftotfm: warning: not enough room in encoding, ignoring 8 glyphs
otftotfm: (This encoding doesn’t have room for all the glyphs used by the
otftotfm: font, so these have been left out:
otftotfm:   T_h f_f_b f_f_h f_f_j f_f_k f_h f_j f_k.
otftotfm: To select specific glyphs, add them to the input encoding.)
...

I know that pdfTeX limits the number of glyphs in a given font to 256, and that the purpose of the font encoding is to determine the character to use in each of the 256 positions. But I would like to use the ligatures that otftotfm left out. How can I include these additional ligatures in the font without any side effects? If this is not possible, which glyphs are safe to move to another font without affecting things like hyphenation? How can I accomplish this?

• If I want to include old-style numerals and other symbols (e.g. Cyrillic alphabet), do I have to create a separate font for this purpose?

• Why is there an 8-bit, 256 glyph limit for a given font in pdfTeX? This limit seems silly and totally arbitrary --- why does this limitation still exist?

Answer 1

The limit of 256 slots per font derives from the TFM format and there's nothing one can do about this. Remember that TeX was finished in the eighties and font technology at the time only used two byte encoded fonts.

The development of pdftex predates the OpenType fonts; the developers chose to remain consistent with TeX's scheme and the TFM font format.

When TeX receives the instruction “print a character”, it takes the corresponding slot from the current font. So, if you want that a ligature is taken from another font you have to explicitly change the current font. There's no way to tell TeX “if you find ffk change font to ‘Crimson ligatures’ and typeset the f_f_k ligature from that font”.

Thus, either you free the output encoding removing less useful characters, or you give up some ligatures. I'm aware this is a big limitation, but with pdftex there's nothing to do. Notice also that pdftex won't hyphenate a word where there's a font change past it. This limitation is raised in LuaTeX.

In my opinion, the best choice is to switch to Lua(La)TeX, if you really need fancy ligatures, font features such as old style digits and different sets of characters. The 2.8 beta version of microtype should work with LuaLaTeX almost fully with respect to its capabilities with pdftex. There are limitations with XeLaTeX, instead.

Answer 2

Note that old style digits are not a problem. You just create multiple type 1 fonts corresponding to the different sets of digits you might wish to use. So e.g. A and a go into all of these fonts. Old style digits go into one, lining into another and so on. Whatever you need.

Ligatures are more problematic. If the LY1 encoding covers the languages you wish to typeset, that will allow more ligatures to be included as it has free slots. Otherwise, you have to start freeing up slots from the T1 encoding in order to make room for the ligatures. The same issue occurs with e.g. swash and alternate characters. The T1 encoding has no free slots so the only way to get ligatures or alternate characters beyond ff, fi, ffi, fl and ffl is by freeing slots.

The tools may have improved over the last couple of years but when I was doing this a lot, I never found the automated generation to produce satisfactory results for anything beyond the basics. There are just too many decisions you need to make - and too many quirks in many fonts - for an automated process to generate output which supports the fancier features. However, perhaps I gave up on those tools too easily! Doing it by hand is certainly a time-consuming and fiddly job...