2

cwebmac.tex version 3.67 --- July 2006

The following Lua code converts string from UTF-8 to UTF-16BE with tex.write, which prints all with catcode 12, except space, which is what is needed. We add the code to the top of cwebmac.tex:

% from navigator.tex
\catcode`\%=12
\directlua{
  local function to8 (...)
    local arg, str = {...}, ""
    for i, num in ipairs(arg) do
      tex.write(string.char(92) .. string.format("%03o", num))
    end
  end
  function toutf16(str)
    tex.write(string.char(92) .. "376" .. string.char(92) .. "377")
    for c in string.utfvalues(str) do
      if c < 0x10000 then
        to8(c / 256, c % 256)
      else
        c = c - 0x10000
        local a, b = c / 1024 + 0xD800, c % 1024 + 0xDC00
        to8(a / 256, a % 256, b / 256, b % 256)
      end
    end
  end
}
\catcode`\%=14

Also, use new toutf16 function by editing cwebmac.tex like this:

--- /usr/share/texlive/texmf-dist/tex/plain/cweb/cwebmac.tex
+++ /usr/local/cwebmac.tex
@@ -474,7 +474,7 @@
     \pdfoutline goto name {NOS} count -\secno {\outsecname}
     \def\X##1:##2\X{\Xpdf##1:##2\X \firstsecno##1.%
       {\toksF={}\makeoutlinetoks##2\outlinedone\outlinedone}%
-      \pdfoutline goto num \the\toksA \expandafter{\the\toksE}}
+      \pdfoutline goto num \the\toksA {\directlua{toutf16("\luaescapestring{\the\toksE}")}}}
   \else\ifpdf
     \special{pdf: outline -1 << /Title (\outsecname)
       /Dest [ @thispage /FitH @ypos ] >>}
@@ -491,7 +491,7 @@
   \expandafter\xdef\csname#1\endcsname{\the\countA}}
 \def\writebookmarkline#1#2#3#4#5{{%
   \let\(=\let \let\)=\let \let\[=\let \let\]=\let \let\/=\let
-  \pdfoutline goto num #3 count -\expnumber{chunk#2.#3} {#5}}}
+  \pdfoutline goto num #3 count -\expnumber{chunk#2.#3} {\directlua{toutf16("\luaescapestring{#5}")}}}}
 \def\con{\par\vfill\eject % finish the section names
 % \ifodd\pageno\else\titletrue\null\vfill\eject\fi % for duplex printers
   \rightskip 0pt \hyphenpenalty 50 \tolerance 200

For the most part, bookmarks are correct (BTW, what for is that \expandafter in the first change?). But in this outline

enter image description here

there are extra backslashes before ( and ). Why do they appear? Did I use the correct approach for encoding PDF outlines?

By using the above lua block in separate file with the following code, we show that toutf16 function works correctly (i.e., it does not produce the backslashes):

% ...
\pdfdest num 1 fit
\pdfoutline goto num 1 count 0 {\directlua{toutf16("\luaescapestring{Если файл дан, пробовать открыть *(++argv); continue если неудача}")}}

\bye

In contrast, when we do everything via \special{pdf:tounicode UTF8-UCS2} in dvipdfmx by adding this to the top of cwebmac.tex, everything works smoothly:

\let\oldshipout\shipout
\def\shipout{\global\let\shipout\oldshipout\afterassignment\myboat\setbox255=}
\def\myboat{\aftergroup\myship}
\def\myship{\setbox255=\vbox{\special{pdf:tounicode UTF8-UCS2}\unvbox255}\shipout\box255}
\let\pdf+

EDIT for more info and better solution see this thread.

  • Let me guess: the pdf uses a different font format than the dvi viewer? – Martin Schröder Jan 9 '16 at 22:06
  • @MartinSchröder: if you try to compile any cweb file (ASCII-only) with pdftex, you will see what I'm talking about... DVI viewer is irrelevant here, I'm using the same pdf viewer in both cases - I convert dvi to pdf with dvipdfmx. – Igor Liferenko Jan 10 '16 at 3:32
2

toutf16 function should be changed to this:

\directlua{
  function toutf16(str)
    tex.write(string.char(92) .. "376" .. string.char(92) .. "377")
    for c in string.utfvalues(str) do
      if c < 0x10000 then
        tex.write(string.char(92) .. string.format("%03o", c / 256) ..
                  string.char(92) .. string.format("%03o", c % 256))
      else
        c = c - 0x10000
        local c1 = c / 1024 + 0xD800
        local c2 = c % 1024 + 0xDC00
        tex.write(string.char(92) .. string.format("%03o", c1 / 256) ..
                  string.char(92) .. string.format("%03o", c1 % 256) ..
                  string.char(92) .. string.format("%03o", c2 / 256) ..
                  string.char(92) .. string.format("%03o", c2 % 256))
      end
    end
  end
}

Also, cwebmac.tex must instead be changed like this:

...
\pdfoutline goto name {NOS} count -\secno {\directlua{toutf16("\outsecname")}}
...
\pdfoutline goto num \the\toksA {\directlua{toutf16(string.gsub([[\the\toksE]],string.char(92),""))}}}
...
\pdfoutline goto num #3 count -\expnumber{chunk#2.#3} {\directlua{toutf16(string.gsub([[#5]],string.char(92),""))}}}}
...

Besides, there are two bugs in cwebmac.tex, which show up when it is used in pdf mode. Following are the fixes for both of them.

1) this adds extra space after each section number:

--- /usr/share/texlive/texmf-dist/tex/plain/cweb/cwebmac.tex
+++ cwebmac.tex
@@ -278,7 +278,7 @@
   \sfcode`;=1500 \pretolerance 200 \hyphenpenalty 50 \exhyphenpenalty 50
   \noindent{\let\*=\lapstar\bf\secstar.\quad}%
   \ifpdftex\smash{\raise\baselineskip\hbox to0pt{%
-     \let\*=\empty\pdfdest num \secstar fith}}
+     \let\*=\empty\pdfdest num \secstar fith}}%
   \else\ifpdf\smash{\raise\baselineskip\hbox to0pt{%
      \let\*=\empty\special{%
        pdf: dest (\romannumeral\secstar) [ @thispage /FitH @ypos ]}}}\fi\fi}

2) this removes all the bookmarks, if \nosecs is used:

--- /usr/share/texlive/texmf-dist/tex/plain/cweb/cwebmac.tex
+++ cwebmac.tex
@@ -457,6 +457,7 @@
 \def\fin{\par\vfill\eject % this is done when we are ending the index
   \ifpagesaved\null\vfill\eject\fi % output a null index column
   \if L\lr\else\null\vfill\eject\fi % finish the current page
+  \ifpdftex \makebookmarks \fi
   \parfillskip 0pt plus 1fil
   \def\grouptitle{NAMES OF THE SECTIONS}
   \let\topsecno=\nullsec
@@ -470,7 +471,7 @@
   \def\Us{\note{Used in sections}} % crossref for uses of a section
   \def\I{\par\hangindent 2em}\let\*=*
   \ifacro \def\outsecname{Names of the sections} \let\Xpdf\X
-  \ifpdftex \makebookmarks \pdfdest name {NOS} fitb
+  \ifpdftex \pdfdest name {NOS} fitb
     \pdfoutline goto name {NOS} count -\secno {\outsecname}
     \def\X##1:##2\X{\Xpdf##1:##2\X \firstsecno##1.%
       {\toksF={}\makeoutlinetoks##2\outlinedone\outlinedone}%

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.