Why use both fontenc and inputenc in the same document? [duplicate]

Question

What's the point of using both fontenc and inputenc in the same preamble? Why would anyone want to specify an input encoding (with inputenc) and an output encoding (with fontenc) that don't coincide? And if they do coincide, why use both?

This question is not a duplicate of this one. In the answer given to that question two scenarios are described:

Scenario 1

1. When you type ä in an editor set up for Latin-1, the machine stores character number 228.

2. When TeX reads the file it finds the character number 228 and the macros of inputenc transform this into \"a.

3. Now fontenc comes into action; the command \" has an associated table of the known accented characters the font has available, and ä is among these, so the sequence \"a is transformed into the command "print character 228" in the current (T1-encoded) font.

In this case the two coincide.

Scenario 2

for instance, of ß:

1. The machine stores character number 223

2. The macros of inputenc change this into \ss

3. fontenc transforms this into "print character 255" (where T1 encoded fonts have a ß character).

In the first scenario what's the point in specifying both inputenc and fontenc if they coincide (e.g. they both associate the character number 228 with the string \"a.

In the second scenario, why would you want to convert the character number 223 to the character number 225? Why can't you stick to 223?

Answer 1

font encoding and input are quite unrelated. By changing the font encoding you can map the same input to different output:

\documentclass{article}

\usepackage[LSF,LGR,T3,T1]{fontenc}

\begin{document}

K Q N

\fontencoding{LSF}\selectfont 

K Q N 

\fontencoding{LGR}\selectfont

K Q N 

\fontencoding{T3}\selectfont

K Q N

\end{document}

Why not only one encoding?

pdflatex can handle only fonts with 256 chars, so even if you use a large input encoding like utf8 you can't use it on the font side.

And while with lualatex you can have "unicode" on both side and so have much less need for special font encodings: unicode doesn't cover all possible symbols. E.g. from the symbols needed to typeset chess (figures, boards and commentary symbols) only around the half are actually in unicode.

Answer 2

OK I'll try to take a different slant than the answers in the duplicate question.

A document (let's say it is saved in UTF-8)

\documentclass{article}

%\usepackage[utf8]{inputenc}
%\usepackage[T1]{fontenc}

\begin{document}

Söméţhìñĝ

\end{document}

This produces

Because tex does not decode the UTF-8 input and all the accented letters make multi-byte sequences using characters above 127 and the default font encoding (OT1) has no characters in those slots so the log says

Missing character: There is no � in font cmr10!
Missing character: There is no � in font cmr10!
Missing character: There is no � in font cmr10!
Missing character: There is no � in font cmr10!
Missing character: There is no � in font cmr10!
Missing character: There is no � in font cmr10!
Missing character: There is no � in font cmr10!
Missing character: There is no � in font cmr10!
Missing character: There is no � in font cmr10!
Missing character: There is no � in font cmr10!
Missing character: There is no � in font cmr10!
Missing character: There is no � in font cmr10!

Note the number of missing character relates to the number of bytes in the the internal UTF8 representation.

If we use inputenc to declare the encoding

you get

which looks OK but if you have a long text you will find that (a) hyphenation is not working and (b) you can not search for this text in the resulting pdf as the font encoding OT1 has no accented letters the ö is rendered as a normal o with an accent " positioned over it.

If now we declare that we want to use a different font encoding say T1 and uncomment that line then you get

which looks slightly different if you look closely but most importantly it (mostly) uses pre-constructed glyphs in the font so is searchable and hyphenation works.

Note however you can still input (thousands) of characters in UTF-8 input that do not relate to characters in the declared font encoding (which only has 256 slots) so depending on the instructions used latex will either need to "construct" something as it does for OT1 or give an error that the character is not supported or as Ulrike showed you can use different font encodings in the same document for the same input encoding to cover different ranges.

Going back to the original document save it as latin1 (iso-8859-1) encoding

\documentclass{article}

%\usepackage[latin1]{inputenc}
%\usepackage[T1]{fontenc}

\begin{document}

SöméThìñG

\end{document}

Note I had to drop two characters that can not be encoded in this input encoding even though I could typeset them with the 7-bit OT1 tex encoding above.

Declaring the latin1 input encoding to latex gets us

But as before this is constructed characters so no searching.

If you declare the input encoding to be latin1 and the font encoding to be T1 (which is not the same in general) then

Answer 3

Very simple.

1. No inputenc and no fontenc

Let's assume first you don't load inputenc.

If you don't load fontenc, the fonts are chosen following the OT1 encoding; upon finding ß in the input there are two main scenarios:

1. the file uses a single byte encoding (such as Latin-1)
2. the file uses UTF-8

In case 1, you'll get the message Missing character ^^df; in case 2 the message will tell you that two characters are missing, namely ^^c3 and ^^df.

2. With inputenc and no fontenc

If you load inputenc with the option corresponding to the encoding of the file, ß will be mapped to \ss. The OT1 encoding definitions will choose the right character to output. On the other hand, hyphenation of words with accented characters will usually be wrong or massively incomplete. For French and German it would be a stopper.

3. No inputenc, but load fontenc

Suppose you load \usepackage[T1]{fontenc} and no inputenc. Again, two scenarios like before. In the case of Latin-1, ß will print ý; in the case of UTF-8, ß will print Ãý.

4. With both inputenc and fontenc

ß will print ß

Conclusion

Which among the four options is good?

Answer 4

Files, both for input and for fonts, have a limited set of slots available which can be addressed. Classically (TeX82), both were 7-bit so allowed 128 separate slots in any one file. TeX90 is 8-bit so extends this to 256. So fonts encoded for TeX use may have either 128 or 256 separate glyphs available (ignoring empty slots) in any one file. To use more than 256 symbols we therefore need to pick more than one font file: Ulrike's answer demonstrates this nicely.

In the input file, again if we start from a 7- or 8-bit situation we need to be able to make best use of the slots available. There are a range of 8-bit input encodings which have been used over time, often related to the language(s) they are used for. However, these encodings have been developed separately from the font encodings used to place glyphs in output. As such, there are two stages we need to carry out:

• Read the input and covert from the input encoding to some fixed internal representation (this is called the LICR in LaTeX).
• Select the correct output glyph for each LICR point, taking account of the fact that there can be more LICR slots than font slots (LICR uses what we can think of as 'descriptions' of slots, so is not limited to a fixed number of positions).

As the two stages are separate and essentially independent, the approach of having two separate LaTeX packages is appropriate. Note that for stability reasons, LaTeX2e with pdfTeX always starts from the assumption of ASCII input encoding and a 7-bit output font encoding.

Notably, the encodings which have been widely used are stable: no changes in input or output positions can be made. This is essential if documents/data are to continue to be usable. In particular, the original Knuth 'OT1' encoding is somewhat 'interesting', whilst T1 does not entirely align with the Latin-1 input encoding structure.

Some confusion may arise as with Unicode the situation is somewhat less acute. Unicode specifies a large number of available slots with a fixed set of meanings, and these map directly between input and output for a the large number of cases that have been codified. As such, for a Unicode TeX engine used with a Unicode font, the need for inputenc/fontenc is removed (largely). However, even with this large range of standardised positions there are still cases that are not covered: one still needs mechanisms to deal with 'non-standard' or 'new' meanings. (Unicode is also 'stable' in the sense that some less-than-ideal decisions cannot be reversed as the encoding must remain usable into the future.)

---

A separate but important point here is that although we have Unicode engines nowadays, pdfTeX (8-bit) is very stable and many workflows are perfectly acceptable using it. This means that whilst for some cases (say mixing Cyrillic and English) one could use a Unicode engine and not need separate encodings, there are many reasons that users prefer to stick to tried-and-tested methods. (Font availability in particular is not a certainty for Unicode work: no font covers the entire range and many very good fonts are available only in classical TeX format.)

Answer 5

If I understand your final question correctly, you're asking why we couldn't use the same "numbers" everywhere, for both input encoding and font encoding.

The short answer is that the two sets to which we want to give "numbers" are different:

• A font is a collection of glyphs (shapes): it describes appearance. It's up to the font designer to put any shape at any position, though they'll usually be sensible and try to follow certain conventions (among many) where possible. With fontenc, you tell TeX where to look up certain shapes it wants to find.

• Your input is a collection of characters, seen by TeX as a stream of bytes. With inputenc, you tell TeX which (sequences of) bytes in your input correspond to which characters.

Examples of input characters: The lowercase letter A (looks like a), the space character, the newline character, the Greek letter pi (looks like π), the Devanagari letter ka (looks like क), the Angstrom sign (looks like Å), the letter A with a dotted circle (looks like Å).

Examples of shapes: Anything you can draw.

How do they differ? Let me count the ways…

A font contains shapes, not characters.

• A font may contain multiple shapes for a character (e.g. different ways of writing lowercase a).
• A font may contain the same shape for multiple characters (e.g. Å and Å).
• A font may contain shapes for non-characters (e.g. "the bowl to the left of the vertical bar in the Devanagari letter क").
• A font may omit shapes for a character (e.g. the newline character). And of course any font will omit most characters and support only a very limited range.

In short: there's no unique "number" that can be given to both characters and shapes, because the two sets are different and cannot be mapped to each other.

However, there are limited cases where font encodings and input encodings may agree:

• if you use (say) only characters A–Z, a–z, 0–9 (and a few more), then indeed the input convention can agree to put the letter A at position 65 (this is called ASCII), and the font convention can agree to put its shape A at position 65 as well. This already happens by default and you don't have to specify anything.

• With Unicode (which currently has over 120,000 characters), quite a large number of characters have been given numbers, and fonts that contain only shapes for those characters can agree to use the same numbers. This would mean that a font designed for the Brahmi script (and which contained shapes precisely for individual characters) would have to put its characters in positions 69632 to 69743. Such large "position numbers" work only with Unicode-aware engines XeTeX and LuaTeX. Instead if the font simply used positions (say) 10 to 122, it could potentially work even without those engines.

In all other (and typical) cases, fonts and characters simply cannot be given the same numbers. Or simply happen not to have been given the same numbers. If you always insisted that they had to use the "same" encoding (over whatever limited domain such a statement is meaningful), you would be able to use only certain fonts with certain input encodings: for instance if you changed the way you input ä, then in your hypothetical world you may suddenly find yourself needing to use a different font. This would be silly.