# xdvipdfmx refuses to process multiple diacritics

so i'm trying to typeset a book with chinese characters and pinyin transcription, and xelatex is throwing errors. i managed to narrow down the problems to the doubly-accented ê̄ down here:

\documentclass{book}
\usepackage{polyglossia}
\setdefaultlanguage{english}
\setmainfont[Mapping=tex-text]{Sun-ExtA}
\begin{document}
ê̄
\end{document}


that's a u+00ea LATIN SMALL LETTER E WITH CIRCUMFLEX followed by a u+0304 COMBINING MACRON. now these suckers really don't behave all too well on my osx mountain lion—depending on application they're rendered ugly, don't want to get copy-and-pasted, all kinds of errors. but worst, they make xelatex—more specifically, xdvipdfmx—quit:

otf_cmap>> Creating ToUnicode CMap for "Sun-ExtA/H/65536/0/0"...
No Unicode mapping available: GID=50878, name=(none)(CID:Sun-ExtA/H/65536/0/0[IPIDPM+Sun-ExtA][CIDFontType2]
** ERROR ** Invalid glyph index (gid 50878)

Output file removed.
)
Error 256 (driver return code) generating output;
file dictionary/output-pdf/dictionary.pdf may not be valid.


Now you might be tempted to point at the specific font i used here, and you would be right in a way—when i omit the font specification, i do get a page rendered (with a quite unsightly ê̄). i did check Sun-ExtA for code point availability, and yes, it does contain both codepoints. it doesn't seem to be the only culprit; in the process of narrowing down the location of the failure, i also removed another font (http://www.georgduffner.at/ebgaramond/) which lead to similar errors (with different alleged GIDs).

the document i'm typesetting is quite complex as it contains chinese characters from the Astral Planes (cjk extensions b and c), pinyin, hiragana, katakana, hangul and so on. it is quite difficult to find nice, correct fonts for all these characters that go together well, so i'm not like, well that one doesn't work so let's just replace it with another font. that won't fly.

to go on, i must specifically find out:

• how to narrow down this class of errors in the first place. it took me many hours to superficially understand the cause and find the source. there is painfully little in the thousands of lines of output that goes to the screen and into the log file (hell i wasn't even able to redirect all output—how does tech print to the screen if its neither stdout nor stderr?). the message that i did manage to isolate does not tell any line number or textual context and is slightly crazy—i mean, when EB Garamond was still in xdvipdfmx complained about GIDs that map to a unicode cyrillic block, which i do not use, and the GID in the above message would correspond to 욾, which does not occur in my data either. are those GIDs even supposed to be unicode codepoints? how are you expected to hunt this down?

• next, i need to know what the best practices are to typeset multiple diacritics in xelatex. it doesn't have to be the most general solution, as there are only a couple relevant cases.

edit others seem to believe just writing out combining accents is a solution, but my trials (also with other fonts) just lead to overlapping accents next to the letter.

edit per the recommendations from the discussion linked above, i tried to \usepackage{covington} and \twoacc[\~|\={a}]—that works with standard fonts and with Sun-ExtA, but not with the Ubuntu font; allegedly GID 1267 being the culprit. i consider this a TeX bug—i mean it's just stacking three outlines on top of each other, right?

edit i forgot to mention: i'm on osx 10.8.4 here; the xetex installation is the one from MacTeX-2012. i did check on multiple fonts, as i had heard about problems (there was a single double font, which i resolved).

edit this example makes xdvipdfmx quit with Invalid glyph index (gid 1265):

\documentclass{book}
\usepackage{polyglossia}
\setmainfont{Ubuntu}
\setdefaultlanguage{english}
\begin{document}
x́
\end{document}


the combining accent used here is u+0301; with u+0304 it quits with a GID of 1268; other diacritics either show a missing chr box or throw similar errors.

-
What operating system do you use and what is the version of XeTeX, and do you happen to have multiple installed copies of Sun-ExtA font? (on Linux, you can check with fc-list --format="%{family}:%{file}\n" | grep Sun-ExtA). –  Khaled Hosny Jun 16 '13 at 2:06
pls see above edit –  flow Jun 16 '13 at 9:37
The multiple copies issue shuold not affect Mac OS X users (and it was fixed in 2012 version any way), where do I get Sun-ExtA font? –  Khaled Hosny Jun 16 '13 at 9:45
OK, that font is broken, it has glyph substitutions that point to non-existent glyph ids, so the error message is legitimate and the font has to be fixed. I get no errors with Ubuntu font, please post the actual text used and the error message. –  Khaled Hosny Jun 16 '13 at 14:02
@flow Is this fixed in TL2013 for you? If so, could you let us know so we can come to some resolution on the question (fixed bug in XeTeX?) –  Joseph Wright Aug 18 '13 at 21:09

here is an answer to the question of how to typeset multiple accents: i came across the somewhat elusive accentbx package by A.S. Berdnikov (berd@ianin.spb.su or berdn@ptslab.ioffe.rssi.ru). it's not on CTAN but someone was so kind to put it on github as https://github.com/zellyn/accentbx so go & grab it. A ready-made PDF is also available online.

for some reason i don't quite understand the package fails to load (at least under XeLaTeX 2012) with

! You can't use \char' after \the.
\EU1\i ->\char
"0131\relax
l.506 \edef\@tempa{\the\i
} \chardef\idotless=\@tempa\relax


so i uncommented lines 500 thru 522 that concern the handling of dotless i's and j's. now i can write

\documentclass{book}
\usepackage{polyglossia}
\setmainfont{Ubuntu}
\setdefaultlanguage{english}
\usepackage{/path/to/accentbx}
\def\tA#1{\upaccent{\tiny a}{#1}}
\begin{document}

\tA{x}

\upaccent{\aboxshift{ˉ}}{ü}

\end{document}


to obtain a teeny a on top of an x and a macron on top of an umlaut, which is just great.

having seen and tried more than one solution for this perennial problem, i recommend everyone who needs to stack type every now and then should avail themselves of this package. the accompanying PDF is very thorough in its treatment of the host of problems that type stackers and careful accentuators experience in the world of TeX.

-

Might there be a problem with your font? I can't use Sun-ExtA because it is proprietary, but when I use one of my fonts (Gentium), it works just fine to compile your document with XeLaTeX:

\documentclass{book}
\usepackage{polyglossia}
\setdefaultlanguage{english}
\setmainfont[Mapping=tex-text]{Gentium}
\begin{document}
ê̄ {\char"00EA\char"0304}
\end{document}
`

-
yes, it's indeed the specific font. and as your answer does show, unicode combining accents do not solve the problem at hand... –  flow Jun 21 '13 at 12:21
As I understood your question, the problem was that you believed xelatex couldn't compile a document with diacritics. But it can, as shown above. –  Sverre Jun 21 '13 at 16:16