# xeCJK messes with punctuation

Neither a question nor really a complain, but I find I cannot use xeCJK in multilingual documents (in the present case French + English + Chinese), simply because it messes with the punctuation in the non-Chinese parts. It will, for instance, change "this… that" to "this…that" and "this, that" to "this , that", which is inconvenient. Too bad because the package is otherwise very nice.

Here's a minimal example showing the problem:

\documentclass {memoir}
\usepackage{polyglossia}
\usepackage{xeCJK}
\setmainlanguage{french}
\usepackage{fontspec}
\setmainfont{TeX Gyre Pagella}
\setCJKmainfont {STSong}
\begin{document}
This… that This, that
\end{document}


The output is the following:

The "this , that" problem appears strictly when I set the main language to French, not if I set it to English. The "this…that" problem appears irrespective of whether the main language is French or English.

And, by the way, forgive me if that's the wrong place to draw attention to this, but the package author's email address doesn't appear in the documentation and I just chanced on this site

-
Welcome to TeX.sx! Please add a minimal working example (MWE) that illustrates your problem. –  Joseph Wright Dec 2 '11 at 9:22
Better to phrase this as a question: It seems that xeCJK does these things... here's a MWE to illustrate the behaviour... how can I fix it? –  qubyte Dec 2 '11 at 9:55
\documentclass {memoir} \usepackage{polyglossia} \usepackage{xeCJK} \setmainlanguage{french} \usepackage{fontspec} \setmainfont{TeX Gyre Pagella} %\setCJKmainfont {STSong} \begin{document} This… that This, that \end{document}  –  François Dec 2 '11 at 10:05
Please forgive the appearance of the code… The "this , that" problem appears strictly when I set the main language to French, not if I set it to English. The "this…that" problem appears irrespective of whether the main language is French or English. –  François Dec 2 '11 at 10:09
It's usually better to post extra code as an edit to the question rather than as comments. I've done this for you. I've also posted a workaround. @LeoLiu is part of the ctex-kit project which I think includes xeCJK. I've notified him in this comment. –  Alan Munn Dec 2 '11 at 11:38

This is not a bug.

xeCJK treats ambigious punctuations as CJK punctuations. The ellipsis is treated as CJK punctuation thus it uses STSong font (华文宋体) and the following spaces are ignored.

You can use \ldots, \dots or \textellipsis to get the proper ellipsis for western languages. (In fact xeCJK patches theses macros specially using \makexeCJKinactive.) Any of these should work well with xeCJK:

This\dots that This, that

This\ldots that This, that

This\textellipsis that This, that


Alan's solution is also advisable, if you use only a small amount of CJK materials. It works, but the drawback is that you must specify a chinese environment manually everywhere you use Chinese. And it also disables some functions of polyglossia.

If you use only a few words of Chinese without long sentences, you don't need xeCJK package. The special progress for CJK punctuations and spacing is overkill. xeCJK is mainly designed for CJK documents with small amounts of other languages.

Similar punctuations includes the quotes “ ” and ‘ ’, you are supposed to use tradational TeX form foo' and foo''.

Warning: xeCJK is NOT supposed to be used together with polyglossia. xeCJK conflicts with polyglossia since they both use \XeTeXinterchartoks heavily, and I can't find a good way to let them work together.

xeCJK has a \makexeCJKinactive, but it does not help here. \makexeCJKinactive simply sets \XeTeXinterchartokenstate=0 and then many of the functions of polyglossia are also disabled. You should choose one of xeCJK and polyglossia.

If you decide to not to use xeCJK, you can set:

\XeTeXlinebreaklocale "zh"
\XeTeXlinebreakskip = 0pt plus 1pt minus 0.1pt


and set the Chinese font manually everytime you use Chinese. It is very suitable for temporaryly typesetting a few Chinese words. And it is very safe. For example:

\documentclass{memoir}
\usepackage{polyglossia}
\XeTeXlinebreaklocale "zh"
\XeTeXlinebreakskip = 0pt plus 1pt minus 0.1pt
\newfontfamily\stsong{STSong}
\newcommand\chinese[1]{{\stsong #1}}
\setmainlanguage{french}
\setmainfont{TeX Gyre Pagella}

\begin{document}
This… that This,
that «Chinese» % automatic spacing after « and before » obtained by polyglossia
\chinese{汉字}. % Chinese as normal text
\end{document}


However, if you have some long Chinese texts, you should use xeCJK for easier font switching and better puncuations. And if you use polyglossia for French, punctuation progress by \french@punctuation will be broken if you are not carefully. If you insist on using xeCJK and polyglossia together, use xeCJK before polyglossia. This might produce less errors, but I didn't make enough test, use it at your own risk.

\documentclass{memoir}
\usepackage{xeCJK}
\setCJKmainfont{STSong}
\usepackage{polyglossia} % It works fine only for simple text, but still dangerous
\setmainlanguage{french}
\setmainfont{TeX Gyre Pagella}

\begin{document}
This\ldots that This,
that «Chinese» % automatic spacing after « and before » obtained by polyglossia

%%% Never use special puctuations handled by polyglossia together with CJK symbols!
%%% This shows a WRONG result, "文" is missing because of the « before it:
«文字»
\end{document}


Currently I am the active maintainer of xeCJK. For any question about xeCJK, you can email to me (leoliu.pku at gmail dot com) directly.

I'm sorry for the incompatibility of xeCJK and polyglossia. Maybe I'll add some commands to disable xeCJK safely in future versions.

-
Thanks, Leo for your detailed answer. So it sounds like my solution is really not to be recommended at all. Do you think it's best that I delete it? –  Alan Munn Dec 2 '11 at 14:24
@Alan: You don't have to. I'll make some small changes on your answer to prevent worst case. –  Leo Liu Dec 2 '11 at 14:34
Thanks, I've also added some more explanation and suggest strongly that people should read your answer before deciding on whether mine is a possible solution. I think it will depend a lot on the other language(s) that you are using as to the severity of the incompatibility. –  Alan Munn Dec 2 '11 at 15:06
It was all too good to be true, then. I was working on a document with long sections in both Chinese and western languages, and thought I could just mix them freely and let XeLaTeX take care of it all for me. I also thought that the Unicode standard provided for distinct CJK punctuation, so that an ellipsis would be encoded differently in the ambit of Chinese text than it would using English, for instance. Too bad. Thanks, Leo. –  François Dec 2 '11 at 15:07

Here at least is a possible workaround, but depending on the particular languages you are using it may not be the best solution, since xeCJK and polyglossia are not really compatible with each other. PLEASE read Leo's answer before deciding to adopt mine as a possible solution. xeCJK provides two commands to turn on and off its spacing/punctuation code. If you decide to do this, you should load polyglossiaafter xeCJK. If your main language is not Chinese, then just use the command at the beginning of the document and create a chinese environment which turns it on selectively.

However, as noted in Leo's answer this will also turn off any special punctuation handling (and possibly other things) by polyglossia too, so it may not work in all cases.

% !TEX TS-program = XeLaTeX

\documentclass {memoir}
\usepackage{xeCJK}
\usepackage{polyglossia} % use it AFTER xeCJK. But still dangerous. (comment by Leo)
\setmainlanguage{french}
\usepackage{fontspec}
\setmainfont{TeX Gyre Pagella} %
\setCJKmainfont {STSong}
\newenvironment{chinese}
{\makexeCJKactive} % punctuation functions from polyglossia are also
%                    disabled if you use this (by Leo)
{\par}
\begin{document}
\makexeCJKinactive
This… that This, that This. That. This; that.

\begin{chinese}
This… that This, that This. That. This; that.
\end{chinese}

This… that This, that This. That. This; that.
\end{document}
`

-
Thanks both for the much needed editing and the solution, @Alan. I have personally resorted to using \newfontfamily\CJK{STSong}, to similar effect. But just as you state, both your solution and mine are mere workarounds and the intended behavior of xeCJK is much nicer. My "question" was actually a sort of bug report that might not have belonged on this site, but I did see some posts by the author of xeCJK, and it seemed a way to reach him. –  François Dec 2 '11 at 13:12