# Can Luatex (or latex) detect presence of non-latin1 character in string?

Using Luatex Unicode, I would like to detect whether or not a string contains any characters outside the Latin-1 set. If it does, then I would like to fork. Something like this:

\def\myTestString{foo\textemdash bar}%
\ifIsOnlyLatinOne{\myTestString}{\doThis}{\doThat}%


Ideally, \myTestString would be expanded, if it contains any TeX character codes (no big macros). I would also like it to inspect pasted Unicode characters. In the above case, since \textemdash is outside Latin-1, the result would be \doThat. The same should apply if I pasted the emdash from a character map, instead of writing the TeX code.

I do not refer to encoding. Everything is UTF8. I only refer to the choice of characters.

Rationale: A part of my document must be limited to Latin-1 characters, whether or not Luatex can process Unicode. I want to remind myself by throwing a message (I know how to do that part).

• What about \newcommand{\notx}{x}? And, unfortunately, \textemdash is far from being a simple macro. – egreg Jul 1 '15 at 22:32
• @egreg: Not sure what your \newcommand structure would do. Is it the case that the rendered string would fail to match itself, outside of Latin-1? I will try it and report back (don't have TeX here, right now). Currently, the only way I can think of that makes sense to me (a newbie) is to loop through the character set and remove the Latin-1 characters. Then, if the remainder is not empty, there must be some non-Latin1 characters there. Also, I did not realize that \textemdash was more complicated than simply substituting a few bytes (depending on encoding). – RobtA Jul 1 '15 at 23:30

You can iterate over all the character nodes and check for a non-ascii character

\documentclass{article}

\usepackage{fontspec}
\begin{document}

\def\myTestString{fo―o\textemdash bar}%

\setbox0\hbox{\myTestString}

\if\directlua{%
res="TF"
for n in node.traverse(h) do
if (n.id==37) then
print("\string\n==" .. n.char)
if (n.char > 255) then
res="TT"
end
end
end
tex.print(res)
}%
non latin-1
\else
all latin-1
\fi

\end{document}


Note if you don't load fontspec here you will be defaulting to OT1 encoding and \textemdash will not expand to a character above 255.

• Thanks. Using fontspec. I will have a look at that when I get on my TeX computer instead of this one. The general idea of iterating seemed like the only way to do it. I also saw egreg's succinct comment. Will report back and mark answer if appropriate. But I have a question: I saw the tex.print line, I do not want to print anything into the document (at that point), but rather fork the logic in the preamble. If the only way to do it is by printing into the document, then maybe I can re-assess other parts of my code. – RobtA Jul 1 '15 at 23:38
• @RobtA you can remove all the printing, that's just so I had half an idea what I was doing:-) – David Carlisle Jul 1 '15 at 23:39
• Gotcha. Will report back tomorrow. – RobtA Jul 1 '15 at 23:40
• Works (removed the print). How it will be used: My input string must be limited to Latin-1 character set (with UTF8 encoding). But it will be copied and pasted from a different application that might have inserted smart quotes and apostrophes, and so forth (those are in ANSI but not Latin-1). I need to flag that, because it is hard to see on screen. – RobtA Jul 2 '15 at 13:03
• Note: The code works with latin1, not just ascii. Output should actually be "non latin1" and "all latin1" rather than ascii. Mere typo, I assume. – RobtA Jul 2 '15 at 15:34