Searchable text and math, encoding

Question

Im running into a problem with text and math font encoding.

First off, the goal at the end of the day is to have a .pdf and .tex which can be searched and copied. The document is compiled in XeTex, and written (the .tex) entirely with unicode characters. So ω instead of \omega.

The most significant issue I am running into is fonts. After some searching, I found that the Latin Modern Roman typeset doesnt actually have very many characters in it. I cannot find a more extensive version other than latinmodern-math.otf, which has its own problems. It is important to be able to type every character outside of a math environment.

After exploring with the unicode-math package (which uses latinmodern-math.otf) I found that it destroys the output .pdf because of the math font indexes it uses. For example, all italic characters have a unique hex value instead of the same as their upright character (the later being the standard). This makes searching the .pdf for all bold, italic, and Fraktur impossible.

Similarly, without the unicode-math package, (because typing ω in a math setting does nothing) using \omega creates a ! character, which also makes the document unsearchable. (I have yet to figure out why it doesn't show up as an ! in the .pdf).

Example, without the unicode-math package: $H_{2}(j Ω \Omega ω \omega)$ will produce H2(jΩ!). Both the Ω and ω were omitted from the end result.

Example, with the unicode-math package: $H_{2}(jΩω)$ will produce 𝐻􀏵(𝑗Ω𝜔)

What I am looking for, is to be able to type, for example:

Here is some text, Ω, ω, $H_{2}(jΩω)$

And have that output the characters:

Here is some text, Ω, ω, H2(jΩω)

Then of course format the math section like normal. (not shown)

Does anyone know how to solve this problem, and create both text and math that turn standard unicode input into standard unicode output? One that can be properly searched and copied?

Answer 1

Here's my experiment:

\documentclass{article}
\usepackage{unicode-math}
\setmainfont{XITS}
\setmathfont{XITS Math}
\begin{document}
Here is some text, Ω, ω, $H_{2}(jΩω)$
\end{document}

Here's the output:

And here's what I get after copying from the PDF, pasting into http://rishida.net/tools/conversion/ and use the “Unicode U+hex notation”

Here is some text, U+03A9, U+03C9, U+1D43B2(U+1D457U+03A9 U+1D714)

So the math H (U+1D43B), the math j (U+1D457), Omega (U+03A9) and math omega (U+1D714) are correctly recognized.

If I use the math-style=ISO option to unicode-math, the uppercase Omega in math mode is printed as U+1D6FA.

If \omega or \Omega are used for the input, the result doesn't change.