\pdfoutline, German sharp-s character

Question

I use the following code to produce bookmarks in a pdf document:

\pdfdest num 1 fit
\pdfoutline goto num 1 count 0 {äöüÄÖÜß}

äöüÄÖÜß
... more text ...

The text prints fine. The pdf bookmark shows umlaut characters but not the 'ß' (s-sharp) character. For 'ß' it displays: 'T1ss'. For the bookmark above 'äöüÄÖÜT1ss' gets displayed.

The documentation for \pdfoutline includes the sentence: 'Note that this is limited to characters in the PDF Document Encoding vector.' Since 'ß' is printed fine in the document, I assume it must be in the 'PDF Document Encoding vector' (and the 6 umlaut characters get printed and are inserted in the Bookmarks perfectly fine). Do I need to review the PDF file structure? Where should I look for the 'PDF Document Encoding vector'? And then, how can I ensure whether or not something gets included there?

I'm using both MikTeX 2.3 and MikTeX 2.9 on a Windows7 32 bit OS. Same result. T1 encoding has been used here for 15+ years, but interestingly, it was only last week that a colleague reported this Problem for the first time.

General document outline:

\pdfoutput=1
\documentclass[german]{article}
\usepackage[latin1]{inputenc}
\usepackage[T1]{fontenc} 

\usepackage{babel}
\usepackage{times}

\begin{document} 
\pdfdest num 1 fit%
\pdfoutline goto num 1 count 0 {1 äöüÄÖÜß}

1 äöüÄÖÜß - etc. etc.

\eject

filler

\eject

\pdfdest num 2 fit%
\pdfoutline goto num 2 count 0 {2 äöüÄÖÜ{\ss}}  % same result

2 äöüÄÖÜß - etc. etc.

\eject

filler

\end{document} 

Answer 1

PDF bookmarks can be encoded in PDFDocEncoding (see PDF specification) or in encoding UTF-16BE with byte order mark (BOM).

Package hyperref(/bookmark) takes care of the encoding issues and supports many LaTeX commands inside bookmark titles.

If you want to reinvent the wheel, then package stringenc can help. And some characters should be escaped to get a valid PDF string.

The following example provides macro \OutlineTitleToPDFString that takes a plain text string with the current input encoding of package inputenc. First it tries to convert the string to PDFDocEncoding. If there are characters that does not fit, then UTF16-BE is used. BTW, this is basically the algorithm that is used by hyperref option pdfencoding=auto. The result is stored in macro \OutlineTitleResult.

\documentclass[ngerman]{article}
\usepackage[latin1]{inputenc}
\usepackage[T1]{fontenc} 

\usepackage{babel}
\usepackage{times}

\usepackage{stringenc}
\newcommand*{\OutlineTitleToPDFString}[1]{%
  \StringEncodingConvertTest{% try PDFDocEncoding first
    \OutlineTitleString % macro that stores the result
  }{%
    \detokenize{#1}% bookmark title in #1, plainly given (not inside macro)
  }{%
    \inputencodingname % encode from
  }{%
    pdfdoc% encode to
  }{% success
  }{% does not fit in PDFDocEncoding, try Unicode
    \StringEncodingConvert{%
      \OutlineTitleString % macro that stores the result
    }{%
      \detokenize{#1}% bookmark title
    }{%
      \inputencodingname % encode from
    }{%
      utf16be% UTF-16BE
    }{%
      \edef\OutlineTitleString{\OutlineBOM\OutlineTitleString}%
    }{%
      \errmessage{Converting bookmark title failed!}%
    }%
  }%
  \xdef\OutlineTitleResult{%
    % escape characters to get valid PDF string
    \pdfescapestring{\OutlineTitleString}%
  }%
}
\begingroup
  \catcode255=12 %
  \catcode254=12 %
  \xdef\OutlineBOM{^^fe^^ff}%
\endgroup

\begin{document} 
\pdfdest num 1 fit%
\OutlineTitleToPDFString{1 äöüÄÖÜß}%
\pdfoutline goto num 1 count 0 {\OutlineTitleResult}%

1 äöüÄÖÜß - etc. etc.

\end{document}

TeX markup is not supported at all in PDF bookmarks. It can be supported to some degree, take a look at hyperref's \pdfstringdef (about 6000 lines of code including pd1enc.def, puenc.def, psdextra.def).

Answers to the questions

• You are lucky that the characters in äöüÄÖÜ are working at all. First package inputenc maps them to LICRs (LaTeX Internal Character Representation), in this case: \"a, \"o, ...

  Then they are mapped via encoding T1 of package fontenc to characters, whose slots accidentally match with the slots in PDFDocEncoding. As you can see with ß, this does not hold for any character. PDFDocEncoding is neither Latin1 nor T1.

• Of course, you need to review the PDF file structure in great detail, if you want to implement all at low level.

• PDFDocEncoding is documented in the PDF specification, "Annex D: Character Sets and Encodings".

• Bookmarks can only be encoded in PDFDocEncoding or Unicode as UTF-16BE with BOM. LaTeX font encodings are irrelevant.