# ucharclasses and Greek

I find that the package ucharclasses cannot correctly identify all the Unicode Greek.

e.g. if I use

 \usepackage{fontspec} \usepackage[Greek]{ucharclasses} \setTransitionsForGreek{\fontspec{Palatino Linotype}}{\fontfamily{lmodern}\selectfont} 

and then in the document entered

 Πέτρος ἀπόστολος Ἰησοῦ Χριστοῦ ἐκλεκτοῖς παρεπιδήμοις διασπορᾶς Πόντου, Γαλατίας, Καππαδοκίας, Ἀσίας καὶ Βιθυνίας 

The output text would be

Πέτρος ἀ Ἰῦ Χριστοῦ ἐῖ παρεπιδήμοις διασπορᾶ Πόντου, Γαλατίας, Καππαδοκίας, Ἀ καὶ Βιθυνίας

i.e. some missing characters.

Even more weird, from the documentation I found out the Greek actually called Coptic, GreekAndCoptic and GreekExtended, but when I called them out individually, more characters will be rendered (but still missing some).

I tried not using ucharclasses but use \fontspec{Palatino Linotype} directly on those text and it works perfectly.

Any idea why is it? Thanks!

# Edit 1

Thanks. I used font family, and I also did a test to show what I mean:

 \usepackage{lmodern} \usepackage{fontspec} \usepackage[Greek]{ucharclasses} \newfontfamily\mylatin{Palatino Linotype} \newfontfamily\mygreek{Palatino Linotype} \setTransitionsForGreek{@EnterGreek@ \mygreek}{\mylatin @ExitGreek@} 

The output will be this:

 @EnterGreek@Πέτρος@ExitGreek@@EnterGreek@ἀ@ExitGreek@πόστολος@ExitGreek@ @EnterGreek@ Ἰ@ExitGreek@ησο@EnterGreek@ ῦ@ExitGreek@ @EnterGreek@ Χριστο@EnterGreek@ ῦ@ExitGreek@ @EnterGreek@ ἐ@ExitGreek@κλεκτο@EnterGreek@ ῖ@ExitGreek@ς@ExitGreek@ @EnterGreek@ παρεπιδήμοις@ExitGreek@ @EnterGreek@ διασπορ@EnterGreek@ ᾶ@ExitGreek@ς@ExitGreek@ @EnterGreek@ Πόντου, @EnterGreek@ Γαλατίας, @EnterGreek@ Καππαδοκίας, @En- terGreek@ Ἀ@ExitGreek@σίας@ExitGreek@ @EnterGreek@ κα@EnterGreek@ ὶ@ExitGreek@ @EnterGreek@ Βιθυνίας@ExitGreek@ 

Comaring the input:

 Πέτρος ἀπόστολος Ἰησοῦ Χριστοῦ ἐκλεκτοῖς παρεπιδήμοις διασπορᾶς Πόντου, Γαλατίας, Καππαδοκίας, Ἀσίας καὶ Βιθυνίας 

Focusing on the 2nd word ἀπόστολος for example, it "exits Greek" after ἀ and treated πόστολος as Latin.

So I actually think it is a bug in ucharclasses.

# Edit 2

I'm not sure if it is related to the limitation, rather than a bug, of ucharclasses. From the documentation:

However, be aware that this only “just works” for Unicode blocks. If you are working with typographically overlapping languages, such as combining English and Vietnamese in one document, things get a lot more complex if you want one font for English and another for Vietnamese. Both of these languagese use Latin blocks, so it is inherently impossible to tell which language is intended based on which Unicode block a character in a word belongs to.

However it didn't mention anything about Greek. Any idea?

# Edit 3

MWE like this?

 \documentclass[a4paper]{article} \usepackage{fontspec} \usepackage[Greek]{ucharclasses} \newfontfamily\mygreek{Palatino Linotype} \setTransitionsForGreek{@Begin@\mygreek}{\fontfamily{lmodern}\selectfont @End@} \begin{document} Πέτρος ἀπόστολος Ἰησοῦ Χριστοῦ ἐκλεκτοῖς παρεπιδήμοις διασπορᾶς Πόντου, Γαλατίας, Καππαδοκίας, Ἀσίας καὶ Βιθυνίας \end{document}  The tags @Begin@ and @End@ are to show the bug I'm talking about.

# Edit 4

Thanks for the answer that solve the problem. I didn't included Latin and that is the root of all the problem.

So the correct code looks like this:

## The Right Way

 \documentclass[a4paper]{article} \usepackage{fontspec} \usepackage[Latin, Greek]{ucharclasses} \newfontfamily\mygreek{CMU Serif} \setDefaultTransitions{\fontfamily{lmodern}\selectfont}{} \setTransitionsForGreek{\mygreek}{} \begin{document} \fontfamily{lmodern}\selectfont \section{Testing Greek Πέτρος ἀπόστολος Ἰησοῦ Χριστοῦ ἐκλεκτοῖς παρεπιδήμοις διασπορᾶς Πόντου, Γαλατίας, Καππαδοκίας, Ἀσίας καὶ Βιθυνίας} \begin{itemize} \item Πέτρος ἀπόστολος Ἰησοῦ Χριστοῦ ἐκλεκτοῖς παρεπιδήμοις διασπορᾶς Πόντου, Γαλατίας, Καππαδοκίας, Ἀσίας καὶ Βιθυνίας \end{itemize} testing \end{document} 

But I got another problem: the Latin in the section title, Testing Greek, is not bold.

If I use my old code, that gives the bug of not producing the right Greek:

## The Old Way

 \documentclass[a4paper]{article} \usepackage{fontspec} \usepackage[Greek]{ucharclasses} \newfontfamily\mygreek{CMU Serif} % \setDefaultTransitions{\fontfamily{lmodern}\selectfont}{} \setTransitionsForGreek{\mygreek}{\fontfamily{lmodern}\selectfont} \begin{document} \fontfamily{lmodern}\selectfont \section{Testing Greek Πέτρος ἀπόστολος Ἰησοῦ Χριστοῦ ἐκλεκτοῖς παρεπιδήμοις διασπορᾶς Πόντου, Γαλατίας, Καππαδοκίας, Ἀσίας καὶ Βιθυνίας} \begin{itemize} \item Πέτρος ἀπόστολος Ἰησοῦ Χριστοῦ ἐκλεκτοῖς παρεπιδήμοις διασπορᾶς Πόντου, Γαλατίας, Καππαδοκίας, Ἀσίας καὶ Βιθυνίας \end{itemize} testing \end{document} 

Then although some Greek are missing, the Latin in the section title, Testing Greek, is now bold.

## The Control

A Control test like this:  \documentclass[a4paper]{article} % \usepackage{fontspec} % \usepackage[Greek]{ucharclasses} % \newfontfamily\mygreek{CMU Serif} % \setDefaultTransitions{\fontfamily{lmodern}\selectfont}{} % \setTransitionsForGreek{\mygreek}{\fontfamily{lmodern}\selectfont} \begin{document} \fontfamily{lmodern}\selectfont \section{Testing Greek }% Πέτρος ἀπόστολος Ἰησοῦ Χριστοῦ ἐκλεκτοῖς παρεπιδήμοις διασπορᾶς Πόντου, Γαλατίας, Καππαδοκίας, Ἀσίας καὶ Βιθυνίας} \begin{itemize} \item testing %Πέτρος ἀπόστολος Ἰησοῦ Χριστοῦ ἐκλεκτοῖς παρεπιδήμοις διασπορᾶς Πόντου, Γαλατίας, Καππαδοκίας, Ἀσίας καὶ Βιθυνίας \end{itemize} testing \end{document} 

would gives the correct bold section title.

I also notice other problems in "the Right Way": since when it leaves the Greek Unicode block, it doesn't change back to the default font, then the next item bullet would be in Greek font, etc. i.e. the bullets would have different fonts. In "the Old Way" when I use \setTransitionsForGreek{\mygreek}{\fontfamily{lmodern}\selectfont}, since it always ends with \fontfamily{lmodern}\selectfont, the problem wasn't there.

I would say all these originates from the mystery that the Greek blocks ends somewhere between words, which I still regard it as a bug (See the test in the 1st edit). I have a guess on the possible reason of this mystery. Referring to my original question:

Greek actually called Coptic, GreekAndCoptic and GreekExtended...

May be the breaks are between the Coptic, GreekAndCoptic and GreekExtended.

# Edit 5

I think I kind of find a solution although I don't understand why. If I use "the Right Way", without adding Latin in ucharclasses, the Latin in the section title would be bold as usual.

• Welcome to TeX.SX! You can have a look at our starter guide to familiarize yourself further with our format. – karlkoeller Mar 3 '15 at 8:41
• You might wan to add a real MWE that we can compile and investigate further. – moewe Mar 3 '15 at 8:52
• You shouldn't use \fontspec to begin with, but rather defining a font family in the preamble. The manual should be fixed about that. However, ucharclasses should only be used if just single words that don't need hyphenation are used. In your case, I'd use polyglossia and mark the transitions with the suitable commands or environments. – egreg Mar 3 '15 at 9:13
• Thanks. I did a test and found a better way to present the problem I was talking about. I edited the main text and please see it there (since it is too long for I to type it here in the comment). – Kolen Cheung Mar 5 '15 at 2:36
• Did you try egreg's suggestion and use polyglossia instead? Note that you still haven't provided an MWE so people can't reproduce. – cfr Mar 5 '15 at 4:00

Try the following code:

\documentclass[12pt,a4paper]{article}
\usepackage[Latin,Greek]{ucharclasses}
\usepackage{fontspec}
\newfontfamily\mynormal{Palatino Linotype}
\setDefaultTransitions{\mynormal}{}
\newfontfamily\mygreek{Junicode}
\setTransitionsForGreek{\mygreek}{}
\usepackage{lipsum}
\begin{document}
\lipsum[1]
Πέτρος ἀπόστολος Ἰησοῦ Χριστοῦ ἐκλεκτοῖς παρεπιδήμοις διασπορᾶς Πόντου, Γαλατίας, Καππαδοκίας, Ἀσίας καὶ Βιθυνίας
\end{document}


• Thanks. I am so stupid and spent almost 10 hours searching this and reading the documentations, etc. I can't believe how simple the solution is. The main mistake I made is not to include Latin. I then used the "tags" to test it again, and it looks like: @Begin@Πέτρος@end@ @Begin@ἀ@end@@Begin@πόσ τολος@end... Although it still breaks ἀ@end@@Begin@πόσ τολος in a strange way, but it did have the correct output if the tags are omitted. Now, should I go back to the original questions to tidy things up or leave it as is? And are there anyway to mark the problem solved? Thanks again! – Kolen Cheung Mar 5 '15 at 6:03
• I found another problem using this approach. I'm adding an edit 4 at the end of the question. Thanks. – Kolen Cheung Mar 5 '15 at 7:33
• Turns out I find that if I omitted the Latin in ucharclasses would solve the problem of bold titles. But I don't understand why it's that. @Ludenticus – Kolen Cheung Mar 5 '15 at 8:05
• I found out that if I omit Latin, it would leads to other problem. Now I'm in a dead end. @Ludenticus – Kolen Cheung Mar 5 '15 at 8:40
• And I found out it is related to the fact that I used \setDefaultTransitions{\fontfamily{lmodern}\selectfont}{}. If I use something else like \setDefaultTransitions{CMU Serif}{}, the title would properly bold again. However, it causes me other problems again. Dealing with unicode in LaTeX is really troublesome. @Ludenticus – Kolen Cheung Mar 5 '15 at 8:53

# Solution 1

Finally I find the answer from this post:



And I find out \setDefaultTransitions is not needed if there's no other Unicode exists.

I also find out using Latin in \usepackage[latin, Greek]{ucharclasses} is also not neccessary, if in the 2nd arguement of \setTransitionsForGreek includes whatever I need to put in \setTransitionsForLatin. And I prefer this method since it makes sure whenever it leaves a certain unicode block it switch right back to the default, and things like item bullet will be using the same font.

Now it's almost perfect, but there's still problem handling the transition to Chinese and I don't understand why. May be I should start a new question later.

Lastly, after dozens of hours figuring out how to handle Unicode in LaTeX and still get many mysteries and quirks and still have unsolved problems(The breaking within Greek words, Chinese fonts transition, etc.)... So frustrating

# Solution 2


 \documentclass{article} \usepackage{fontspec} \usepackage[Latin, Greek]{ucharclasses} \setTransitionsForGreek{\fontspec{CMU Serif}}{} \begin{document} \section{Testing Greek Πέτρος ἀπόστολος Ἰησοῦ Χριστοῦ ἐκλεκτοῖς παρεπιδήμοις διασπορᾶς Πόντου, Γαλατίας, Καππαδοκίας, Ἀσίας καὶ Βιθυνίας} \begin{itemize} \item Πέτρος ἀπόστολος Ἰησοῦ Χριστοῦ ἐκλεκτοῖς παρεπιδήμοις διασπορᾶς Πόντου, Γαλατίας, Καππαδοκίας, Ἀσίας καὶ Βιθυνίας \end{itemize} testing \end{document} 

# Solution 3

All the problems orginates from the fact that I want to go back to the default lmodern, and that lmodern doesn't support Greek. I compared lmodern and cm-unicode and found that the differences are pretty minor. If I don't want to deal with the mess mentioned above, and don't mind using a slightly different font than lmodern, then I could just change the default to cm-unicode:

 \documentclass{article} \usepackage{fontspec} \setmainfont{CMU Serif} \setsansfont{CMU Sans Serif} \setmonofont{CMU Typewriter Text} \begin{document} \section{Testing Greek Πέτρος ἀπόστολος Ἰησοῦ Χριστοῦ ἐκλεκτοῖς παρεπιδήμοις διασπορᾶς Πόντου, Γαλατίας, Καππαδοκίας, Ἀσίας καὶ Βιθυνίας} \begin{itemize} \item Πέτρος ἀπόστολος Ἰησοῦ Χριστοῦ ἐκλεκτοῖς παρεπιδήμοις διασπορᾶς Πόντου, Γαλατίας, Καππαδοκίας, Ἀσίας καὶ Βιθυνίας \end{itemize} testing \end{document} 

"Default" means 'minimum required to run the app'; it doesn't mean "the recommended setting". You choose what main font(s) you want to use as a default for your document. Look in the TexMF tree, under fonts. The Latin Modern families of fonts are available as OpenType fonts (under \fonts\opentype\public\lm\). Nowadays, also, Unicode system fonts are available, not just local Tex fonts. For example, Noto Serif contains Latin, Greek and Coptic, and Cyrillic, and that means you don't have to use transition packages to manage these scripts:

(Greek and Coptic scripts go together as a pair, because of their large overlap.)

MWE

\documentclass[12pt]{article}
\usepackage{fontspec}
\setmainfont{Noto Serif}
\newfontface\fgla{Fedorovsk Unicode TT}
\begin{document}
\section{Testing Greek} Πέτρος ἀπόστολος Ἰησοῦ Χριστοῦ ἐκλεκτοῖς παρεπιδήμοις διασπορᾶς Πόντου, Γαλατίας, Καππαδοκίας, Ἀσίας καὶ Βιθυνίας
\begin{itemize}
\item Πέτρος ἀπόστολος Ἰησοῦ Χριστοῦ ἐκλεκτοῖς παρεπιδήμοις διασπορᾶς Πόντου, Γαλατίας, Καππαδοκίας, Ἀσίας καὶ Βιθυνίας
\item Петр, Апостол Иисуса Христа, пришельцам, рассеянным в Понте, Галатии, Каппадокии, Асии и Вифинии, избранным,

from:
1-е послание Петра 1 глава – Библия: https://bible.by/syn/46/1/
\end{itemize}

A Glagolitic font in the Tex distribution:

{\fgla \huge Петр, Апостол Иисуса Христа, пришельцам,}

\end{document}


Question here describes the various ways to get back to the fontspec default settings (including Latin Modern; it used to be CM), and also how to switch font encodings if desired.