64

This is a problem I've had consistently with XeLaTeX. Here's a small example:

\documentclass{article}
\usepackage{fontspec}
\setmainfont[Ligatures=TeX,Numbers={OldStyle,Proportional}]{Minion Pro}

\begin{document}
The \textsc{nato} office 1234
\end{document}

The file compiles and displays just fine. But if I try to copy its text from Acrobat to another application, I get garbage characters in place of all small-caps, old-style numbers, and the ligature Th:

e  office 

This also means that if I search the PDF, there are no hits for "The", "NATO", or "1234". And God help me if the file gets indexed by Google.

Now if I switch to LuaLaTeX, I don't have this problem:

The \textsc{nato} office 1234

The nato office 1234

Then if I use uppercase small-caps, I can get exactly what I want:

The \mbox{\addfontfeatures{Letters=UppercaseSmallCaps}NATO} office 1234

The NATO office 1234

But try as I might with XeLaTeX, it still gives the same garbled copy text. The problem seems to be that in the resulting PDF file, LuaTeX generates "correct" CMap entries while XeTeX generates bugged ones which reference the Private Use Area:

LuaTeX                                        XeTeX
--------------------------------------------- ---------------------------------------------

/CIDInit /ProcSet findresource begin          /CIDInit /ProcSet findresource begin
12 dict begin                                 12 dict begin
begincmap                                     begincmap
...                                           ...
1 begincodespacerange                         1 begincodespacerange
<0000> <FFFF>                                 <0000> <FFFF>
endcodespacerange                             endcodespacerange
0 beginbfrange                                5 beginbfchar
endbfrange                                    <0044> <0063>
13 beginbfchar                                <0046> <0065>
<0044> <0063>                                 <0050> <006F>
<0046> <0065>                                 <0107> <E062>
<0050> <006F>                                 <010F> <FB03>
<0107> <00540068>                             endbfchar
<010F> <FB03>                                 1 beginbfrange
<015D> <0031>                                 <015D> <0160> <F731>
<015E> <0032>                                 endbfrange
<015F> <0033>                                 4 beginbfchar
<0160> <0034>                                 <05C9> <E000>
<05C9> <0041>                                 <05D6> <E044>
<05D6> <004E>                                 <05D7> <E049>
<05D7> <004F>                                 <05DC> <E061>
<05DC> <0054>                                 endbfchar
endbfchar                                     endcmap
endcmap                                       CMapName currentdict /CMap defineresource pop
CMapName currentdict /CMap defineresource pop end
end                                           end
end

Indeed, if I just "transplant" all these CMap entries from LuaTeX's PDF into XeTeX's PDF, the problem goes away! So:

  1. Is there a way to make XeTeX emulate the way LuaTeX makes CMap entries?
  2. If not, is there a way to edit the font to remove any references to the Private Use Area which may "tempt" XeTeX to make these incorrect CMap entries?
  3. If none of that is possible, is there a way to edit the CMap entries in PDFs made by XeTeX, automatically and without needing a LuaTeX run?

EDIT: Since Jörg asked, here's how I did the "CMap transplant":

  1. Make a file test.tex with the following contents:

    \documentclass{article}
    \usepackage{fontspec}
    \setmainfont[Ligatures=TeX,Numbers={OldStyle,Proportional}]{Minion Pro}
    \begin{document}
    The \mbox{\addfontfeatures{Letters=UppercaseSmallCaps}NATO} office 1234
    \end{document}
    
  2. Compile it with xelatex and lualatex and uncompress each PDF with pdftk:

    xelatex test
    mv test.pdf test_x.pdf
    lualatex test
    mv test.pdf test_l.pdf
    pdftk test_x.pdf output test_xu.pdf uncompress
    pdftk test_l.pdf output test_lu.pdf uncompress
    
  3. Open each uncompressed PDF file in a text editor and look for the sections above. Delete the section from XeTeX's PDF and replace it with the equivalent section from LuaTeX's PDF (from begincodespacerange to endcmap).

  4. The PDF is corrupted now, but pdftk will fix it:

    pdftk test_xu.pdf output test_xf.pdf
    

Now test_xf.pdf will have the correct copy text. This is a neat proof-of-concept but it's useless for several reasons:

  • You have to compile in XeTeX and LuaTeX. If my files all compiled in LuaTeX then I'd just use LuaTeX and be done with it. That's the ideal solution here anyways.

  • You can't make a "key file" that compiles in LuaTeX and then just put its CMap into every PDF you make using XeTeX, since XeTeX appears to assign random input codes (e.g., <0044>, <0046>, <05C9>) every time the file is changed. Transplanting a CMap which uses a different set of input codes results in no characters being selectable.

  • If you use more than one font, you need to fix the CMap for each one. This applies even to the same typeface in two different optical sizes.

So the only way I can see this work is if someone made a program which had a built-in list of fonts and their problematic PUA references, which would go through every CMap section in a PDF line-by-line, identify the PUA references by their targets, and then change the targets according to the list. But that seems far too much work and at a certain point you just have to migrate to LuaTeX.

20
  • 1
    @Jörg Yes, I've thought about using accsupp but with XeTeX it would require wrapping all numbers and ligatures in a macro. The manual also suggests that it could transpose characters, which would be very bad with numbers. I'd prefer to do everything with cmap magic if possible.
    – Chel
    Mar 26, 2012 at 13:54
  • 1
    This question popped up now and then in the xetex mailing list. And as far as I remember no good solution (apart from using accsupp) has ever been offered. cmap magic seems to be too font dependant to work. Mar 26, 2012 at 15:35
  • 2
    After reading through this thread on the mailing list (mail-archive.com/[email protected]/msg00520.html) it appears that I might have to switch to luatex...
    – Jörg
    Mar 26, 2012 at 19:13
  • 4
    Another good reason to use LuaTeX instead of XeTeX :-)
    – raphink
    May 31, 2012 at 13:48
  • 2
    I can confirm the problem with TL 2012, but if I compile the example with TL 2014 the problem seems to be gone (stumbled upon the question in context of tex.stackexchange.com/q/238260). Is it possible that the problem has been resolved by some changes in XeLaTeX? Apr 14, 2015 at 19:05

1 Answer 1

8

A partial answer to your second question regarding PUA encodings:

You can remove PUA encodings with ttx, a tool that is included in the Adobe Font Development Kit for OpenType (AFDKO).

However, new fonts such as Arno Pro do not have PUA encodings for smallcaps at all, but compiling an document such as yours still results in small caps not being searchable. Hence, the notion that Jonathan Kews states here is unfortunately wrong. The fault are not PUA encodings, but faulty cmaps generated by XeTeX.

Having said that, in your question you state that you can transplant a correct cmap generated by LuaTeX to a faulty map generated by XeTeX. Maybe a workaround would be to create a correct cmap for every possible glyph and then use that for XeTeX. That would be completely font (and font revision) dependent, of course, but it should work for your private purposes.

Unfortunately I cannot try that as I don't know how to "transplant" a cmap. Could you please elaborate how you did that?

Edit: And I just realised that everything works perfectly with Junicode and EB Garamond (when you specify the SC font separately), i.e. something like:

\documentclass{article}

\usepackage{fontspec}
\setmainfont{Junicode}

\begin{document}

{\addfontfeature{Letters=UppercaseSmallCaps}DIE STRAßE IST ZU SCHMAL FÜR AUTOS.

{\addfontfeatures{Numbers=OldStyle}12345}

\end{document}

compiles perfectly with searchable PDF. Maybe I give up on trying to understand what is going on...

1
  • See my edited question for details on the "CMap transplant."
    – Chel
    Jun 8, 2012 at 23:44

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .