# tex4ht corrupts “--” inside CDATA in the HTML file

I am using an outside javascript package (syntaxHighlighter) for code formatting when I compile to HTML that requires one to put code inside <![CDATA[ ... code .. ]]> as explained in the above link

This causes no problem, except that tex4ht for some reason changes -- (two dashes) to something strange which causes syntaxHighlighter to fail rendering it, since it does not show on the screen, which is wrong.

Here is a MWE, and I show the command used and the final HTML, one can see clearly that -- has been changed. This should not happen. Anything inside <![CDATA[ ... code .. ]]> should not be touched by tex4ht as this is plain verbatim code and should be left as is. It is means to be used by the Javascript library.

\documentclass[11pt]{scrartcl}%
\IfFileExists{luatex85.sty}{\usepackage{luatex85}}{}

\ifdefined\HCode% detect tex4ht
\usepackage[utf8]{luainputenc}
\usepackage[T1]{fontenc}
\else
\usepackage{fontspec}
\fi

\begin{document}
\ifdefined\HCode
{\ScriptEnv{html}{\ifvmode\IgnorePar\fi\EndP\NoFonts\hfill\break}{\EndNoFonts\endgraf}
\begin{html}
<DIV><style scoped>.syntaxhighlighter {font-size: normalsize!important;}</style>
<script type="syntaxhighlighter"class="brush:bash; toolbar: false;" >
<![CDATA[
make4ht --lua -u -c my.cfg -e main.mk4 foo.tex "htm,pic-align,notoc*"
]]>
</script>
</DIV>
\end{html}}
\else
test
\fi

\end{document}


Compiled using make4ht --lua foo7_1.tex and this is how the HTML looks like

<?xml version="1.0" encoding="iso-8859-1" ?>
<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Transitional//EN"
"http://www.w3.org/TR/xhtml1/DTD/xhtml1-transitional.dtd">
<!--http://www.w3.org/TR/xhtml1/DTD/xhtml1-transitional.dtd-->
<html xmlns="http://www.w3.org/1999/xhtml"
>
<meta http-equiv="Content-Type" content="text/html; charset=iso-8859-1" />
<meta name="generator" content="TeX4ht (http://www.tug.org/tex4ht/)" />
<meta name="originator" content="TeX4ht (http://www.tug.org/tex4ht/)" />
<!-- xhtml,html -->
<meta name="src" content="foo7_1.tex" />
>
<DIV><style scoped>.syntaxhighlighter {font-size: normalsize!important;}</style>
<script type="syntaxhighlighter"class="brush:bash; toolbar: false;" >
<![CDATA[
make4ht lua -u -c my.cfg -e main.mk4 foo.tex "htm,pic-align,notoc*"
]]>
</script>
</DIV>

</body></html>


Notice the -- are not displayed any more. They are changed to some non-displayable code. Here is screen shot

What do I need to change to make tex4ht not change code inside my HTML like this? Please note that when you compile the above to HTML on your computer, nothing will display, since one needs the javascript package installed. But by viewing the source, one can see the problem.

A hex dump of the HTML shows that -- was changed to hex 15

Which from the ascii table it is NAK (negative acknowledge)

Update

This problem happens for any data, even in <PRE> ... </PRE> and has nothing to do with CDATA. It seems tex4ht simply does not like to see -- when in lualatex mode. Here is a new MWE

\documentclass[11pt]{article}%
\IfFileExists{luatex85.sty}{\usepackage{luatex85}}{}

\ifdefined\HCode% detect tex4ht
\usepackage[utf8]{luainputenc}
\usepackage[T1]{fontenc}
\else
\usepackage{fontspec}
\fi

\begin{document}
\ifdefined\HCode
{\ScriptEnv{html}{\ifvmode\IgnorePar\fi\EndP\NoFonts\hfill\break}{\EndNoFonts\endgraf}
\begin{html}
<PRE>
-- this is a test
</PRE>
\end{html}}
\else
test
\fi

\end{document}


Same problem. Here is the HTML generated after make4ht --lua foo.tex

If I do not NOT compile using lualatex mode, then it works! And I get -- show up in HTML. i.e. if I compile the above using make4ht foo.tex, here is the HTML

Now the -- are there. So this is a problem with lualatex mode with tex4ht.

• unrelated to tex but that <!CDATA is an HTML:/Javascript syntax error (although the browser typically might be forgiving and not report it) – David Carlisle Jun 30 '16 at 6:47
• @DavidCarlisle I am sorry, I am not following you. Where is the syntax error? – Nasser Jun 30 '16 at 11:42
• <script><![CDATA[ is a javascript syntax error in html. (It would be valid in xhtml, but I doubt you are serving your pages as XML rather than text/html) – David Carlisle Jun 30 '16 at 12:46
• Extra info: vi shows the corrupted character as ^U which is ascii control code 21 (0x15) in case that's of any use to anyone. I don't know why it's happening. I did wonder if it was trying to interpret -- as an en-dash but --- is converted to ^U- in the html file. – Nicola Talbot Jun 30 '16 at 13:08
• @NicolaTalbot surprisingly enough vi is right:-) (21 is endash in T1 encoding) the issue is the normal verbatim active character ligarture supression isn't woring in luatex as called by maek4ht – David Carlisle Jun 30 '16 at 13:44

Hyphen character is made active by \ScriptEnv command, which means that it calls a command when it is encountered in the environment defined with it. The ScriptEnv definition is little bit cryptic:

 \bgroup
\catcode\-=13
\catcode$$=1 \catcode$$=2  \catcode\/=0
\catcode\{=12   \catcode\}=12  \catcode\\=12
/gdef/ScriptEnv#1(%
/expandafter/let/csname :#1:/endcsname=/empty
/edef/:temp(/def/expandafter/noexpand/csname a:#1/endcsname
####1/expandafter/noexpand/csname end/endcsname{#1}%
(####1/noexpand/:EndVerbatim/noexpand/end(#1)))/:temp
%   /expandafter/def/csname
%      b:#1/endcsname##1\end{#1}(##1/:EndVerbatim/end(#1))%
/expandafter/def/csname
b:#1/endcsname(/bgroup /catcode/-=13
/def/:temp####1\end{#1}(/egroup
/def-(/string-/relax)
####1/:EndVerbatim%
/expandafter/def/csname end#1/endcsname()%
/end(#1))%
/:temp)%
/edef/:temp(/noexpand/DefScript:
/expandafter/noexpand/csname #1/endcsname
(/expandafter/noexpand/csname a:#1/endcsname)%
(/expandafter/noexpand/csname b:#1/endcsname))/:temp
)
/egroup


the important line is:

/def-(/string-/relax)


it seems that under LuaTeX, two consecutive hyphens forms endash in this case, which is not translated to unicode character for some reason, but remains in 8-bit encoding. I don't really understand it, but the solution is to alter the \ScriptEnv definition to suppress the ligature forming:

\bgroup
\gdef\:scriptenv:breakhyphen{\hbox{}}
\catcode\-=13
\catcode$$=1 \catcode$$=2  \catcode\/=0
\catcode\{=12   \catcode\}=12  \catcode\\=12
/gdef/ScriptEnv#1(%
/expandafter/let/csname :#1:/endcsname=/empty
/edef/:temp(/def/expandafter/noexpand/csname a:#1/endcsname
####1/expandafter/noexpand/csname end/endcsname{#1}%
(####1/noexpand/:EndVerbatim/noexpand/end(#1)))/:temp
%   /expandafter/def/csname
%      b:#1/endcsname##1\end{#1}(##1/:EndVerbatim/end(#1))%
/expandafter/def/csname
b:#1/endcsname(/bgroup /catcode/-=13
/def/:temp####1\end{#1}(/egroup
/def-(/string-/:scriptenv:breakhyphen/relax)
####1/:EndVerbatim%
/expandafter/def/csname end#1/endcsname()%
/end(#1))%
/:temp)%
/edef/:temp(/noexpand/DefScript:
/expandafter/noexpand/csname #1/endcsname
(/expandafter/noexpand/csname a:#1/endcsname)%
(/expandafter/noexpand/csname b:#1/endcsname))/:temp
)
/egroup


correct the definition in local copy of latex.4ht. I will update the tex4ht sources.

• thank you. I changed latex.4ht as you have and now the -- show up! I have no idea how you understand the above code and what it all means. It is really scary looking. Thanks again. – Nasser Jun 30 '16 at 16:16
• @Nasser I don't really understand it, I only know that \def- influences hyphens :) – michal.h21 Jun 30 '16 at 17:20