# lost my smilies: it is worth the effort to make Unicode work?

I made some LyX Beamer slides with smiley faces and I found they do not survive the transition to utf-8.

I've fiddled with this quite a bit. Right now, it appears that, at least for miscellaneous symbols like smilies and frownies, I don't need Unicode entries at all. Entering them as Unicode is a frustrating, confusing experience.

Am I wrong?

Here is my simple use case. The only content is 4 symbols, no Unicode

\documentclass[english]{article}
\usepackage[T1]{fontenc}
\usepackage[latin9]{inputenc}
\usepackage{wasysym}
\usepackage{babel}
\begin{document}
\frownie \smiley \blacksmiley \sun{}
\end{document}


That compiles, looks fine. I type in "\smiley", that symbol is drawn from wasysm.

Then I changed the language encoding to utf8 and insert Unicode symbols

\documentclass[english]{article}
\usepackage[T1]{fontenc}
\usepackage[utf8]{inputenc}
\usepackage{wasysym}
\usepackage{babel}
\begin{document}
☹☺☻☼
\end{document}


The document does not compile, the error messages are

Package inputenc Error: Unicode char ☹ (U+2639)
Package inputenc Error: Unicode char ☺ (U+263A)
Package inputenc Error: Unicode char ☻ (U+263B)
Package inputenc Error: Unicode char ☼ (U+263C)


A quick fix seems to be ignore Unicode input, go back to \smiley. This does compile:

\documentclass[english]{article}
\usepackage[T1]{fontenc}
\usepackage[utf8]{inputenc}
\usepackage{wasysym}
\usepackage{babel}
\begin{document}
\frownie \smiley \blacksmiley \sun{}
\end{document}


It is not a forward looking Unicode solution.

However, all of the Unicode solutions seem to be elaborate ways to get the same work done. They create a mapping from input symbol ☺ to \smiley in the wasysym package. Here are examples.

1. DeclareUnicodeCharacter. If I enter a Unicode character in the document, say ☺, I can use Emacs to find out what the Unicode character number is (C-u C-x =), it says the smilie is 263A.

This file compiles:

\documentclass[english]{article}
\usepackage[T1]{fontenc}
\usepackage[utf8]{inputenc}
\usepackage{wasysym}
\usepackage{babel}
\DeclareUnicodeCharacter{263A}{\smiley}
\begin{document}
☺
\end{document}


So, in return for the "benefit" of being able to type in ☺ in LaTeX, then I get the additional job of telling LaTeX where to find that symbol. Isn't it simpler to just type \smiley in the text?

1. Use newunicodechar package.

This is easier because it works without specifying the Unicode character number. newunicodechar takes the symbol as the first parameter and the waysym LaTeX markup in the second.

Here is a working example

\documentclass[english]{article}
\usepackage[T1]{fontenc}
\usepackage[utf8]{inputenc}
\usepackage{wasysym}
\usepackage{babel}
\usepackage{newunicodechar}
\newunicodechar{☺}{\smiley}
\begin{document}
☺
\end{document}


Again, the Unicode "victory" is that I can enter "☺" rather than "\smiley", but so far as I can tell, that's the only benefit.

1. Change compiler to xelatex, revise the document font settings.

I did some testing and it does work. However, I'm concerned about other features that might not work correctly. (Remembering when I changed from latex to pdflatex and pstricks quit working.)

• it's up to you whether you want to use unicode input or not, for one-off smilies it doesn't make much difference, if you want to type greek or arabic or something then typing every character by its english name isn't that friendly. Also it's not as hard as you make out to get the numbers, you don't need to ask emacs, the character and the unicode number are shown in the error message that you quote in your question. – David Carlisle Sep 25 '18 at 19:08
• Thanks. It only seemed difficult because I was learning, I suppose. – pauljohn32 Sep 25 '18 at 22:41

The first error message for the first document that you posted is, as you show,

! Package inputenc Error: Unicode character ☹ (U+2639)
(inputenc)                not set up for use with LaTeX.
See the inputenc package documentation for explanation.
Type  H <return>  for immediate help.
...

l.7 ☹
☺☻☼
?


If you type h at the prompt as it suggests you get a further hint:

 h
You may provide a definition with
\DeclareUnicodeCharacter


So the Unicode number is in the error message you don't have to look it up separately and

\DeclareUnicodeCharacter {2639}{\frownie}


does what you want.

For symbols such as ☹ it is, as you say, easier for most people to type the ASCII based TeX name \frownie but the same mechanism also lets you type χαμόγελο or スマイル or whatever (google translate to blame for any errors in translation:-) and if you are writing in such a language, entering each letter via a command based on its English name would be somewhat tiresome.

But by design in inputenc all such characters always go via an ASCII based internal form before then being further processed to the typeset output, so it is entirely a choice of input form. Whether it is worth the effort depends on your circumstances and whether it is easier to type ☹ or \frownie or χαμόγελο or something like \textchi\textalpha\textmu\'\textomicron\textgamma\textepsilon\textlambda\textomicron

• Thanks. Can I ask one clarifying issue? When people write Unicode for characters of other languages, I now see value of Unicode. Are those authors required to provide DeclareUnicodeCharacter or newunicodechar for each and every Unicode symbol that they introduce? If that is required for them, then I now understand why xetex is such a valuable improvement. – pauljohn32 Sep 26 '18 at 20:35
• @pauljohn32 someone has set up something. usually there is by now a package that has done it for you, the core latex input release (only in last two or three years) covers the all the accented characters you can do with T1 encoded fonts, so that covers most (not all) latin based scriptsand packages for Greek etc exist, for massive character sets like Chinese the cjk packages do something essentially similar , so yes there are tens of thousands of lines of macros for something that in xetex or luatex is passing characters data "straight through" to the font back end. – David Carlisle Sep 26 '18 at 20:52