How do I encode foreign characters so that they are searchable in the resultant PDF file?

Question

This may be a PDF problem rather than a LaTeX problem, but as it involves pdfLaTeX output I'm hoping I won't get my wrist slapped too hard.

I'm creating a bibliography, and inevitably some of the authors' names have accented or other foreign characters in them. I'd like to be able to search the PDF file for these authors, but I'm having problems with some of the characters. As an example, I've created a simple .tex file as follows:

\documentclass{article}
\usepackage[utf8]{inputenc}
\begin{document}
Some foreign characters: öøäéüæåñ
\end{document}

I can search successfully in the resultant PDF file for ö, ä, é, ü, å and ñ, either using those specific characters or an unaccented o, a, e, u or n. However, the PDF file will not find the ø or æ characters whether I copy and paste them into the search box from another programme, or whether I use Alt+155 or Alt+145. The problem is compounded by the fact that some journals use a simple 'o' for ø or 'ae' for æ, so I'd like to get a search 'hit' for ø using either ø or o, and similarly a search hit for æ using either æ or ae. Is any of this remotely possible, or am I trying to be too clever?

Answer 1

Can you try:

\usepackage[T1]{fontenc}

Full test document:

\documentclass{article}
\usepackage[utf8]{inputenc}
\usepackage[T1]{fontenc}
\begin{document}
Some foreign characters: öøäéüæåñ
\end{document}

Answer 2

Although I can't give a working solution, I want to explain where the problem stems from.

Like Crissov conjectured in his comment, this has a lot to do with the reader application you are using. In my case Okular would not even find öaéuåñ inside of words. As example I used:

\documentclass{article}
\usepackage{fontspec}
\setmainfont[Ligatures=TeX]{STIXGeneral}
\XeTeXinputnormalization=1 %or 2
\begin{document}
Töst Tøst Täst Tést Tüst Tæst Tåst Testeñ
\end{document}

With \XeTeXinputnormalization=1 I can't find the special characters at all, when searching for regular ones. If I set \XeTeXinputnormalization=2, at least I'll find To in Töst. Still I will find neither Tost nor To in Tøst.

The reason for this behaviour is, that Okular is doing a quite exact comparison and, more importantly, there is a major difference between the characters ö and ø (which causes your problem):

ö (U+00F6) has a decomposition into a base character o (U+006F) and an accent ̈ (U+0308) associated to it. While you can write o̷ (U+006F U+0337) and it will usually displayed as ø, it is not associated to ø (U+00F8) and when converting between unicode normal forms it will not be converted. For æ it is the same as for ø. All the other characters you mentioned do have an associated decomposition. (See unicode character decomposition mapping for reference.)

So it seems your reader application does the sensible thing, to consider not only the actual character, but also the base character of its decomposition. For the other ø, æ etc. it is simply too much to expect from it.

You could probably convert all ø to images and only lay an invisible o behind it, like it is done for scanned pdfs with OCR. But, apart from that being an ugly solution, you would not be able to find the word, if you do type the actual ø then...