1

UTF-8 representations of Unicode characters may take up to four octets to define.

If an octet is from "00 to "7F, then it is a single character. If it is from "C0 to "DF, then it is the first octet of a two-octet character, with "E0 to "EF and "F0 to "F7 for the first octets of three- and four-octet characters respectively. If it is from "80 to "BF, then it is a continuation of a character started by one of the three types of "first octets" above. (Sources: https://stackoverflow.com/questions/5290182/how-many-bytes-does-one-unicode-character-take, https://www.wikiwand.com/en/UTF-8)

However, in "Providing some UTF-8 support via inputenc" (https://texdoc.org/serve/utf8ienc.pdf/0), it is mentioned instead that the ranges for the first octets of two-, three-, four-octet, and continuation characters are "C2 to "DF, "E0 to "EF, "F0 to "F3, and "80 to "B9 respectively.

What is the reason for the discrepancies? Namely, where are "C0, "C1, "F4 to "F7, and "BA to "BF?

1 Answer 1

3

You cannot create a UTF8 code which begins by C0 or C1. Lowest Unicode which needs two bytes is 80, i.e 1000 0000 in binary. (It is € character, but this is not important now). You have only 6 bits for saving it to the second byte, because second byte have to be prefixed by 10. So, we divide the binary number to 2+6: 10 000000. First part must be saved to the first byte prefixed by 110, i.e. 110 00010 and the second part is saved to the second byte prefixed by 10, i.e. 10 000000. The resulting UTF8 code is 11000010 10000000, i.e. C2 80.

4
  • Thank you, that serves to explain the lower limit of "C2 on the first octet of two-octet characters. The other two mismatching bounds don't make sense yet though: U+10FFFF is "F4 "8F "BF "BF, which goes above both "F0 to "F3 and "80 to "B9.
    – texnically
    Oct 22, 2022 at 13:30
  • 1
    sorry it's a typo in the doc, I'll fix. The code is right. @texnically Oct 22, 2022 at 17:13
  • That clears it up. Thank you @DavidCarlisle!
    – texnically
    Oct 22, 2022 at 18:17
  • @texnically github.com/latex3/latex2e/commit/… Oct 22, 2022 at 18:27

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .