"Chi error":

! Text line contains an invalid character.
l.18   \testengine ^^?
                      ^^?!\relax % That's Chi, a 2-byte UTF-8 sequence'

The error is triggered by

18  \testengine χ!\relax % That's Chi, a 2-byte UTF-8 sequence

in dehypht-x-2019-04-04.tex (windows 10 TexLive 2019 distribution).

Does eTeX need to be taught UTF-8 before it can handle this file? How to do this?

I just had the problem that the etex.src sometimes contained tabs instead of spaces.

Do I have another file problem with dehypht-x-2019-04-04.tex or is it an eTeX setup problem?

I generated eTeX with web2js Convert TeX's pascal to javascript .

 fmt_file = "*etex.src"
 library.setInput("\n" + fmt_file + " \\dump\n\n", 

As you can see in the following log the "Chi" problem starts in the dehypht-x-2019-04-04.tex file:

This is e-TeX, Version 3.14159265-2.6 (INITEX)
**entering extended mode
(etex.src (plain.tex Preloading the plain format: codes, registers,
parameters, fonts, more fonts, macros, math definitions, output routines,
hyphenation (hyphen.tex [skipping from \patterns to end-of-file...]))
(etexdefs.lib Skipping module "grouptypes"; Loading module "interactionmodes";
Skipping module "nodetypes"; Skipping module "iftypes";) (language.def
(hyphen.tex) (dehypht-x-2019-04-04.tex
! Text line contains an invalid character.
l.18   \testengine ^^?
                      ^^?!\relax % That's Chi, a 2-byte UTF-8 sequence
? ! Text line contains an invalid character.
l.18   \testengine ^^?^^?
                         !\relax % That's Chi, a 2-byte UTF-8 sequence
? Runaway argument?
\relax \ifx \secondarg \empty \message {dehyph-exptl: using a \ETC.
! Paragraph ended before \testengine was complete.
<to be read again>
\relax \ifx \secondarg \empty \message {UTF-8 Hyphenation patt\ETC.
! File ended while scanning use of \testengine.
<inserted text>
\addlanguage ...uselanguage {#1}\input #2
                                          \if *#3*\else \input #3 \fi...
l.73 ...ntgreek}{loadhyph-grc.tex}{}{1}{1}

? (ibyhyph.tex Greek hyphenation patterns for Ibycus encoding, v3.0
! TeX capacity exceeded, sorry [pattern memory=8000].
l.615 a)2n1a'gku
  • Without -ini that can't work, but it should give a different error to the one you see. Something is clearly up with the IniTeX catcodes of your binary. Can you do etex -ini "\showthe\catcode`^^?"? – Joseph Wright Jul 26 '19 at 14:54
  • Yes, that's for find the files, but eTeX should still report the full path – Joseph Wright Jul 26 '19 at 14:55
  • 1
    Well somehow it's in IniTeX mode (\patterns is allowed). You won't see the issue with the US patterns as they are not trying to do an engine test, but the other ones are (to support UTF-8 with pTeX in particular). Like I said, my guess is that the catcode set up is wrong: we need to see what \showthe\catcode`^^? gives. – Joseph Wright Jul 26 '19 at 15:03
  • Let us continue this discussion in chat. – Joseph Wright Jul 26 '19 at 15:06

This answer is based on the ideas, hints and suggestions of David Carlisle, Joseph Wright and Marcel Krüger. They have solved the problem and I am just writing it down here.

It's an character encoding problem. tex.web reads one character per byte. However, χ (Chi) consists of 2 bytes: \xcf\x87 i.e. unicode code point U+03C7.

How to teach tex.web to read UTF-8 input:


%add the bigger codepage:

for i:=0 to @'37 do xchr[i]:=' ';
for i:=@'177 to @'377 do xchr[i]:=' ';
for i:=0 to @'37 do xchr[i]:=chr(i);
for i:=@'177 to @'377 do xchr[i]:=chr(i);

Here you can find a more detailed instruction from Marcel Krüger how to use this 7 lines in a sys-file: How to prepare a chg file, tie and tangle: tex.web tex.ch etex.ch and etex.sys to a new etex.web?

With this modification you can generate a system which can process χ.




This is TeX, Version 3.14159265 (preloaded format=plain 1776.7.4)
No pages of output.
Transcript written on chi_test.log.
|improve this answer|||||
  • @ShreevatsaR What exactly does \def\zz#1#2{[#1][#2]} means? I would expand this macro zzas follows: \immediate\write20{[Chi_byte_1][Chi_byte2]..} i.e. you just have to know here that chi is a container for 2 bytes which can be adressed bei #1 and #2?. You might as well think the first dot of .. is #2. – CarpeDiemKopi Aug 3 '19 at 12:14

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.