# Hyphenation patterns / lccode / “.dat” format of hyph-utf8

I have made a new encoding for "modern" greek based on ISO 8859-7.

Now I'd like to create hyphenation patterns for it, to use with `pdftex` (that is, 8 bit based).

If I understand correctly, I can do this with hyph-utf8 by creating a `.dat` file mapping my encoding to unicode and then processing the unicode hyphenation patterns for monotonic greek with it.

Here's an excerpt of a `.dat` file:

``````0x9A    U+04D8      CYRSCHWA
0x9B    U+040A      CYRNJE
0x9C    U+0401      CYRYO
0x9D    U+2116      textnumero
0x9E    U+00A4      textcurrency
0x9F    U+00A7      textsection
0xA0    U+0491  1   cyrgup
0xA1    U+0493  1   cyrghcrs
0xA2    U+0452  1   cyrdje
0xA3    U+045B  1   cyrtshe
``````

Most of the columns are rather self-explaining, but I'd still love to see some documentation before I start experimenting ;-)

In particular, it will save some copy-pasting work if I could just ignore everything but the first two columns (what is the significance of the glyph name for the hyphenation pattern, after all?)

## Edit

Thanks to Patrick Gundlach for suggesting the README in the hyph-utf8 CTAN source directory. On conversions, this file has to say,

``````generate-converters.rb
======================
INPUT:
- source/generic/hyph-utf8/data/encodings/*.dat
OUTPUT:
- tex/generic/hyph-utf8/conversions/conv-utf8-*.dat

Auto-generates conversions from UTF-8 to some particular encoding.
``````

So now, I know what to do with the `.dat` file. An important step forward :-)

But I still need to know what exactly the contents of the file mean (as I have to make a new one for my own encoding).

## Update

After analysing this a bit more the only information I need is what the "1" in the third column is for.

Looking at the ruby code, it has the effect that for instance the following

``````0xE0    U+00E0  1   agrave
``````

(from `ec.dat`) will generate in the converter `conv-utf8-ec.tex` (which will take the unicode hyphenation patterns and map them to T1 fontencoding) the line

``````\lccode"E0="E0 % à - U+00E0 - agrave
``````

So, hopefully this can be answered now by anybody who knows hyphenation patterns at all: What types of characters is such a line needed for?

-
The contents of README on ctan.org/tex-archive/language/hyph-utf8/source/generic/… seems to be what you look for, doesn't it? And for the glyph names: the hyphenation patterns are font encoding dependent AFAIK, and the font encodings are font dependent because each font (Type1 to be exact) can have different glyph names for the same glyph. So there is no universal EC encoding or TS1 or .. because the names are only matched by a subset of the available fonts. –  topskip Apr 12 '12 at 19:53
@PatrickGundlach Thank you very much for this comment. I have everything apart from the hyphenation patterns. The glyph names go into the `.enc` file, and I already successfully made one for my new encoding. I also fail to find any trace in the `hyph-utf8` docs that it is doing anything with glyphs. But maybe I'm overlooking something. One possible explanation might be that the `.dat` files themselves are again auto-generated from some other source like enc+cmap and the glyph names are only in there for comparison and checking whether everything went right. –  Stephan Lehmke Apr 12 '12 at 21:38
In order to be fully considered for hyphenation, a word must consist of characters having non zero `\lccode`. Since `agrave` is lowercase, its `\lccode` is just set to its character code. –  egreg Apr 13 '12 at 10:09
@egreg Ok, so this would mean all "lowercase real letters" need a "1"? Why not the uppercase ones? –  Stephan Lehmke Apr 13 '12 at 10:12
@StephanLehmke I'm not really into the problem, but I believe that the correspondence between uppercase and lowercase characters is set elsewhere and so not needed when hyphenation is considered. –  egreg Apr 13 '12 at 10:19
show 1 more comment

In order to be fully considered for hyphenation, a word must consist of characters having non zero `\lccode`. Since `agrave` is lowercase, its `\lccode` is just set to its character code.

This is the purpose of the line

``````\lccode"E0="E0 % à - U+00E0 - agrave
``````

that is not really needed for the `\lowercase` procedure, because characters with zero `\lccode` are left untouched by it.

There will be (somewhere else) a line

``````\lccode"C0="E0 % À - U+00C0 - Agrave
``````

that sets the correspondence for `\lowercase`, but also defines a nonzero `\lccode` for `À`.

-
Great! I put "1"s to all lowercase characters after making sure they are in the same positions as for T1 encoding. With this, generating a loader for the UTF8 monotonic greek hyphenation patterns worked and I'm getting correct hyphenation! –  Stephan Lehmke Apr 18 '12 at 15:34