I have made a new encoding for "modern" greek based on ISO 8859-7.

Now I'd like to create hyphenation patterns for it, to use with pdftex (that is, 8 bit based).

If I understand correctly, I can do this with hyph-utf8 by creating a .dat file mapping my encoding to unicode and then processing the unicode hyphenation patterns for monotonic greek with it.

Here's an excerpt of a .dat file:

0x9A    U+04D8      CYRSCHWA
0x9B    U+040A      CYRNJE
0x9C    U+0401      CYRYO
0x9D    U+2116      textnumero
0x9E    U+00A4      textcurrency
0x9F    U+00A7      textsection
0xA0    U+0491  1   cyrgup
0xA1    U+0493  1   cyrghcrs
0xA2    U+0452  1   cyrdje
0xA3    U+045B  1   cyrtshe

Most of the columns are rather self-explaining, but I'd still love to see some documentation before I start experimenting ;-)

In particular, it will save some copy-pasting work if I could just ignore everything but the first two columns (what is the significance of the glyph name for the hyphenation pattern, after all?)


Thanks to Patrick Gundlach for suggesting the README in the hyph-utf8 CTAN source directory. On conversions, this file has to say,

- source/generic/hyph-utf8/data/encodings/*.dat
- tex/generic/hyph-utf8/conversions/conv-utf8-*.dat

Auto-generates conversions from UTF-8 to some particular encoding.

So now, I know what to do with the .dat file. An important step forward :-)

But I still need to know what exactly the contents of the file mean (as I have to make a new one for my own encoding).


After analysing this a bit more the only information I need is what the "1" in the third column is for.

Looking at the ruby code, it has the effect that for instance the following

0xE0    U+00E0  1   agrave

(from ec.dat) will generate in the converter conv-utf8-ec.tex (which will take the unicode hyphenation patterns and map them to T1 fontencoding) the line

\lccode"E0="E0 % à - U+00E0 - agrave

So, hopefully this can be answered now by anybody who knows hyphenation patterns at all: What types of characters is such a line needed for?

  • The contents of README on ctan.org/tex-archive/language/hyph-utf8/source/generic/… seems to be what you look for, doesn't it? And for the glyph names: the hyphenation patterns are font encoding dependent AFAIK, and the font encodings are font dependent because each font (Type1 to be exact) can have different glyph names for the same glyph. So there is no universal EC encoding or TS1 or .. because the names are only matched by a subset of the available fonts. – topskip Apr 12 '12 at 19:53
  • @PatrickGundlach Thank you very much for this comment. I have everything apart from the hyphenation patterns. The glyph names go into the .enc file, and I already successfully made one for my new encoding. I also fail to find any trace in the hyph-utf8 docs that it is doing anything with glyphs. But maybe I'm overlooking something. One possible explanation might be that the .dat files themselves are again auto-generated from some other source like enc+cmap and the glyph names are only in there for comparison and checking whether everything went right. – Stephan Lehmke Apr 12 '12 at 21:38
  • 1
    In order to be fully considered for hyphenation, a word must consist of characters having non zero \lccode. Since agrave is lowercase, its \lccode is just set to its character code. – egreg Apr 13 '12 at 10:09
  • @egreg Ok, so this would mean all "lowercase real letters" need a "1"? Why not the uppercase ones? – Stephan Lehmke Apr 13 '12 at 10:12
  • @StephanLehmke I'm not really into the problem, but I believe that the correspondence between uppercase and lowercase characters is set elsewhere and so not needed when hyphenation is considered. – egreg Apr 13 '12 at 10:19

In order to be fully considered for hyphenation, a word must consist of characters having non zero \lccode. Since agrave is lowercase, its \lccode is just set to its character code.

This is the purpose of the line

\lccode"E0="E0 % à - U+00E0 - agrave

that is not really needed for the \lowercase procedure, because characters with zero \lccode are left untouched by it.

There will be (somewhere else) a line

\lccode"C0="E0 % À - U+00C0 - Agrave

that sets the correspondence for \lowercase, but also defines a nonzero \lccode for À.

  • Great! I put "1"s to all lowercase characters after making sure they are in the same positions as for T1 encoding. With this, generating a loader for the UTF8 monotonic greek hyphenation patterns worked and I'm getting correct hyphenation! – Stephan Lehmke Apr 18 '12 at 15:34

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.