3

Once more time I am (frustratingly losing time: true joke!) trying to use better TeX primitives. The next snippet doesn't work as expected in Knuth's Tex. The lower case version simply swallow the character.

\catcode"C3=13 % catcode for first bit
\catcode"A7=11 % and second ones for
\catcode"87=11 % unicode ç Ç

\def^^c3#1{%
  \begingroup
  \def\arg{#1}%
  \def\upper{^^87}% work only in original TeX engine
  \def\lower{^^A7}% not work in both cases
%   
%
  \ifx \upper\arg \c C%
  \else
  \ifx \lower\arg \c c%
  \fi\fi
  \endgroup
}

caçhaca e Çassi.

ca\c chaca e \c Cassi. %expected macro expansion
\end

Using LuaTeX engine the things get worse: the ç and Ç are completely ignored.

I would like to understand each of these unexpected results.

6
  • 5
    LuaTeX is a native UTF-8 engine, you are using bytes ...
    – Joseph Wright
    Dec 8, 2023 at 9:40
  • So when was the syntax lost in LuaTex or bad applied by me? I can't get at first glance what LuaTex's UFT-8 supports really means. Dec 8, 2023 at 9:49
  • 2
    why do you have the internal, definitions of \def\arg etc then compare with \if not \ifx ????? having \begingroup and definitions will destroy any kerns or ligatures in the font specified for ç You could look how latex defines these working for multiple encodings and engines and without breaking font-specified kerns Dec 8, 2023 at 14:27
  • 1
    @DavidCarlisle I assume this is purely academic, and if we are dealing with bytes then \if is fine ...
    – Joseph Wright
    Dec 8, 2023 at 16:53
  • 1
    @JosephWright well \ifx would be a whole lot safer than \if if two tokens end up in \arg but conversely if \if is used you could remove all the internal \def and the \begingroup so not break kerning, so it's not really academic, so much as a pointer to a more usable definition Dec 8, 2023 at 16:55

2 Answers 2

6

The issue when running with Knuth's TeX or another 8-bit TeX (e.g. pdfTeX) is that \def\lower{^^A7} stores literally ^^A7 in \lower: you want \def\lower{^^a7}. It will then work: you have a stored byte as expected.

In LuaTeX (or XeTeX), input is not read as bytes but as Unicode codepoints: this means that ç and Ç are single tokens which have their own catcodes.

3
  • So, writing \catcode`\Ç=13 \defÇ{\c C} (and variations) resolves the problem in LuaTex. Now I've understood the meaning of saying "LuaTex has UTF-8 support". Token is not defined by byte, but by character. Dec 8, 2023 at 12:50
  • @DanielBandeira Well I'd just pick a font which has Ç as a character, but yes that would work
    – Joseph Wright
    Dec 8, 2023 at 12:51
  • Tell me, please, how to easily make a font reachable by the engine. Dec 15, 2023 at 10:43
1

You can use \def\lower{^^a7} in LuaTeX (but note the lowercase a):

This is LuaTeX, Version 1.17.0 (TeX Live 2023)
 restricted system commands enabled.
**\relax

*\def\lower{^^a7}\show\lower
> \lower=macro:
->§.
<*> \def\lower{^^a7}\show\lower

But the main problem is that LuaTeX never sees C3A7 when the input is ç, because it doesn't read byte-by-byte, but codepoint-by-codepoint. If the file is UTF-8 encoded, then upon seeing C3A7, LuaTeX transforms it into U+00E7 and this is what's passed to the eyes for tokenization. So your active ^^c3 character is never seen.

Well, you can use the legacy methods, but I see no reason to.

\documentclass{article}
\usepackage[utf8]{luainputenc}

\catcode"C3=13 % catcode for first bit
\catcode"A7=11 % and second ones for
\catcode"87=11 % unicode ç Ç

\def^^c3#1{%
  \begingroup
  \def\arg{#1}%
  \def\upper{^^87}% work only in original TeX engine
  \def\lower{^^a7}% not work in both cases
%   
%
  \ifx \upper\arg \c C%
  \else
  \ifx \lower\arg \c c%
  \fi\fi
  \endgroup
}

\begin{document}

ç Ç

^^c3^^a7 ^^c3^^87

\end{document}

enter image description here

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .