Does XeLaTeX have something like DeclareUnicodeCharacter but supporting Unicode grapheme clusters (i.e. units of multiple of code points)?

Question

I’d like to use \DeclareUnicodeCharacter to define mappings of Unicode characters, represented in decimal or hexadecimal form, with alternative expressions or graphics that should replace the Unicode characters. For example:

\DeclareUnicodeCharacter{014F}{\u{o}}

While this specific example works fine, Unicode characters cannot just be single code points but also grapheme clusters, i.e. sequences of multiple code points that form a unit and a single visible character. Example:

U+006E U+0303 = ñ (sometimes, there are equivalents like U+00F1)

It seems the command \DeclareUnicodeCharacter comes from the inputenc package and supports values between 0 and 10FFFF only, which is enough for single code points, but may not provide any means for composed grapheme clusters. But when using XeLaTeX, the implementation does not come from inputenc, right?

So with inputenc or with a “native” implementation, is there any way to map grapheme clusters instead of just single code points? For example:

\DeclareUnicodeCharacter{006E0303}{...}
# or
\DeclareUnicodeCharacter{006E,0303}{...}

Edit:

The use case is something like \DeclareUnicodeCharacter or \newunicodechar (perhaps without a complete extra package), but for units of multiple code points instead of just single code points, in order to create custom mappings.

It seems TECkit mappings, referenced in the Mapping attribute of fontspec, may provide the exact functionality (including multiple code points being mapped) (Edit: but only to “plain text”, not to commands, apparently), but is not elegant, not contained in the same text/source file, and requires separate tooling.

There’s also \XeTeXinterchartoks, but this doesn’t really make definitions easy to write, especially for multiple individual grapheme clusters (as opposed to character blocks).

Answer 1

Your question is confusing as \DeclareUnicodeCharacter is not defined for xelatex.

However if I understand the use case it is possible (although I don't really recommend it) to use XeTeX character classes.

this detects just n followed by a combining tilde and replaces the pair by an \fbox construct.

\documentclass{article}

\XeTeXinterchartokenstate=1

\newXeTeXintercharclass\nclass

\XeTeXcharclass `\n \nclass

\XeTeXinterchartoks 0 \nclass = {\ntest}
\XeTeXinterchartoks 4095 \nclass = {\ntest}

\def\ntest#1{\futurelet\next\ntestb}

\def\ntestb{%
 \ifx\next ^^^^0303%
    \fbox{An n-tilde combining pair}%
    \expandafter\eatnext
    \else
      {\XeTeXinterchartokenstate=0 n}%
    \fi}

\def\eatnext#1{}

\begin{document}



noo  ñ   abcñxyz  oon

\end{document}

Answer 2

You need nothing of that kind and it works out of the box.

Here's an example with plain XeTeX, in order to show that nothing at all is needed. In the first call I use n followed by U+0303 COMBINING TILDE, in the second call I use directly ñ.

%%% print the Unicode point of the given string and the result
\def\test#1{%
  \unicodestring#1\relax
  --- #1
}

\font\testfont="[Junicode.ttf]:mapping=tex-text"

\def\unicodestring#1#2{%
  (\the\numexpr`#1\relax)\space
  \ifx#2\relax\else
    (\the\numexpr`#2\relax)\space
  \fi
}

\testfont

\test{ñ}

\test{ñ}

\bye

Answer 3

The mapping is the input that you type. (So you will need an input method. Simplest method is direct input.)

OpenType font files contain the rules for compound glyphs (ligatures). The font shaping engine (e.g., HarfBuzz when using Xelatex) applies the rules.

You do not need to re-invent the wheel.

MWE

\documentclass{article}
\usepackage{xcolor}
\usepackage{fontspec}
\setmainfont{Noto Serif}\newcommand\textnote[1]{{\color{blue}$\leftarrow$ #1}}
\begin{document}
\symbol{110} + \ \symbol{771} = \symbol{110}\symbol{771} : (U+006E U+0303)

\symbol{241} : (U+00F1)

a\symbol{771} b\symbol{771} c\symbol{771} d\symbol{771} e\symbol{771} \textnote{Combining Diacritical Mark}

ᲀᲁᲂᲃᲄᲅᲆᲇᲈ \textnote{Cyrillic Extended-C}

\end{document}

---

Added

For plain xetex, you can use ^^^^ notation if you have no keyboard overlay, OS language choice, character map, regex replace etc.

MWE

%xetex
\font\testfont="[NotoSerif-Regular.ttf]:mapping=tex-text"

\testfont

^^^^006e^^^^0303 using \space ^^^^0302 ^^^^0302 ^^^^0302 ^^^^0302  notation.

\bye