4

I asked a very similar question several years ago on the tex4ht listserv and received a very helpful answer from Michal Hoftich. He described certain limitations for working with Unicode and tex4ht that I suspect may have been ironed out over the past few years of development. And so I am in essence asking the same question again (though some of the details are different) in hopes of learning whether there are simpler or more automated ways of converting my XeLaTeX document to .docx format.

setup

source file

First I produce a file that concatenates head-simplified.tex with the contents of main.tex and then tail.tex, where the three files are as follows:

head-simplified.tex:

%!TEX TS-program = xelatex
%!TEX encoding = UTF-8 Unicode

\documentclass[12pt]{memoir}
\usepackage{ifmtarg}
\usepackage{calc}

%Greek
\newcommand{\greek}[1]{#1}
\newcommand{\gk}[1]{#1}

%Arabic
\newcommand{\ar}[1]{#1}
\newenvironment{arab}{}{}

% Bibliography etc
\usepackage[american]{babel} 
\usepackage{csquotes}
\usepackage[notes,
        alldates=       comp,
        backend=        biber,
        bibwarn=        false,
        cmsdate=        both, % reprints: print `origdate` as well as `year`
        compresspages=  true,
        doi=            false,
        eprint=     false,
        ibidtracker=        true,
        inheritshorthand,       % feature added for me by D.Fussner
        isbn=           false,
        mincrossrefs =  2,
        shorthandibid,
        strict=     true,
        url=            true,
        usetranslator=  true,
        uniquename= true
    ]{biblatex-chicago}
\bibliography{word.converter.alexhistory-ent.bib}

\usepackage{textcomp} % For angle brackets `\textlangle` and `\textrangle`

\begin{document}

%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
% code which C.V. Radhakrishnan added to make the unicode carry
% over smoothly to the word world.
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
\makeatletter
\def\hshchr{\expandafter\@gobble\string\#}
\def\ampchr{\expandafter\@gobble\string\&}
\def\entity#1{\HCode{\ampchr\hshchr#1;}}
\makeatother

main.tex:

\newcommand{%
\ص%
}[1]{$\rangle$#1$\langle$}


This is a test file using a macro whose name uses Arabic Unicode characters:
\ar{%
العربية%
}.
Then we can try it as a block quote:
\begin{quote}
\begin{arab}
% Text with Unicode macro:
أن يبد\ص{أ}
% Text without Unicode macro:
%أن يبدأ
\end{arab}
\end{quote}
If that doesn't work, here is some non-Arabic Unicode: \emph{kitābat al-ʿarabiyyah bi-l-ḥurūf al-lātīniyyah}, \gk{καὶ τὸ στοιχείοις Ἑλληνικοῖς γράφειν}...

tail.tex:

\end{document}

conversion

Then I use a script called texbody2word that automates the instructions that I owe to C.V. Radhakrishnan and Michal Hoftich and others who have helped make this conversion possible:

#!/bin/bash

#This takes a tex file with only the body
#(no header or `\end{document}`) and compiles
#it as a PDF

#filename without extension `.tex`
filename=$(basename "$1")
filename="${filename%.*}"
converterdir="/Users/alexandre/Dropbox/scripts/texbody2word-files"

#produce standalone tex file
cat $converterdir/head-tail/head-simplified.tex $filename.tex $converterdir/head-tail/tail.tex > $filename.out.tex

#convert unicode to ascii codes in tex file and bib file
perl $converterdir/utf2ent.pl $filename.out.tex > $filename.out-ent.tex
perl $converterdir/utf2ent.pl ~/Dropbox/bib-dbs/alexhistory.bib > word.converter.alexhistory-ent.bib

#apply tex4ht to tex file; then biber; then tex4ht again 
mk4ht oolatex $filename.out-ent "xhtml, superscript, charset=utf-8"  -utf8
biber $filename.out-ent
mk4ht oolatex $filename.out-ent "xhtml, superscript, charset=utf-8"  -utf8

#convert resulting odt file to docx
#(LibreOffice must NOT be open for this to work)
soffice --headless --invisible --convert-to docx $filename.out-ent.odt

result

When I run it on my system (using MacTeX 2018, packages updated on January 13, 2018, using TeX Live Utility), the output is a .docx file that mixes good output with gobbledygook like "ntity1589Δ1" (followed by two "images" that do not render and contain the words "The picture can't be displayed") caused by the fact that my setup cannot understand macros that use Unicode in the macro name because it converts all Unicode characters in the original TeX file to \entity{....}.

After the total gobbledygook, the rest of the file looks something like this:

This is a test file using a macro whose name uses Arabic Unicode characters: العربية. Then we can try it as a block quote:

أن يبد
entity1589أ

If that doesn’t work, here is some non-Arabic Unicode: kitābat al-ʿarabiyyah bi-l-ḥurūf al-lātīniyyah, καὶ τὸ στοιχείοις Ἑλληνικοῖς γράφειν...

As you can see, that is pretty good, except for the failure to deal with the Unicode-named macro.

If I comment out the part of the TeX file that defines the macro using a Unicode character, then the initial gobbledygook disappears, and only this last part, beginning with "This is a test file," remains.

question

My question is this: Is there a way to keep using my Unicode macro-names and just change the workflow in order to produce a .docx file?

1 Answer 1

2

Yes, the process can be simplified considerably. tex4ht supports full Unicode with XeTeX and LuaTeX engines, so it is not necessary to use Perl scripts for the pre-processing. Another change to your workflow is that the recommended way to produce ODT document from tex4ht is to use make4ht -f odt script, which fixes lot of issues that can happen during the conversion.

I would also modify your macro, it is better to use text commands for < and > than math:

\makeatother\newcommand{%
 \ﺹ%
 }[1]{\textless #1\textgreater}

But I think the output is wrong anyway. Probably because of font shaping in the output, because it is represented differently in the XML than in the rendered document.

One thing with XeTeX support in tex4ht is that the support for non-Latin scripts must be explicitly loaded using the \xeuniuseblock command. It can be used in the configuration file, like myconfig.cfg:

\Preamble{xhtml}
\xeuniuseblock{Greek}
\xeuniuseblock{Arabic}
\makeatletter
\ConfigureEnv{arab}{\@rltrue}{\@rlfalse}{}{}
\makeatother
\begin{document}
\EndPreamble 

The names to be used should be OpenType script names (these are used as value for script key in Fontspec font selection commands).

The configuration for arab environment enables RTL direction for the enclosed text. This will produce right aligned text with correct text direction for Arabic.

The last bit is the Biber support. It can be requested in the make4ht build file. It can be even required conditionally, in order to save the compilation time. Save the following file as main.out.mk4:

Make:add("biber", "biber ${input}")
if mode=="draft" then
Make:htlatex {}
else
Make:htlatex {}
Make:biber {}
Make:htlatex {}
Make:htlatex {}
end

The file can be compiled using the following command:

make4ht -ux -f odt  -c myconfig.cfg main.out.tex

The subsequent compilations may use the draft mode, which use only one LaTeX compilation and doesn't use Biber, which results in huge saving of the compilation time:

make4ht -uxm draft -f odt  -c myconfig.cfg main.out.tex

The result in LibreOffice:

enter image description here

You can see that the macro output is really strange.

7
  • Thank you, this is great! Now about fixing the formatting, I wonder if it has to do with right-to-left issues. When I swap \textless and \textgreater, it's almost right, except that the angle bracket on the right should appear in the Arabic font (which presumably would flip it the correct way) at the end of the line on the left. Also, Arabic block quotes would ideally be R-to-L by default in the output. Jan 16, 2019 at 20:35
  • Adding more Arabic in the block quote after the macro fixes the formatting problem (once I had swapped < and >), so it suggests that the real lingering problem is how an Arabic paragraph is formatted in the output, particularly punctuation at the end (i.e., left-most extreme) of a line. Jan 16, 2019 at 20:40
  • @AlexRoberts I think I've found the solution for the direction issues. See the updated answer. It is just necessary to place the quote environment inside arab, so it has an effect.
    – michal.h21
    Jan 16, 2019 at 20:44
  • 1
    @AlexRoberts I've also fixed a bug in the configuration for the quote environment in tex4ht sources, so the direction stuff will work after tex4ht update in TL, which should happen in few days.
    – michal.h21
    Jan 16, 2019 at 20:48
  • I can't seem to reproduce what you're getting with the new lines in myconfig.cfg. It still has the old strange output, even when I put the quote environment inside the arab environment. (Or did you mean that it will only work after the TL update?) Also, is there a way that does not involve swapping the nesting of the quote and arab environments? My (huge) document has many instances of arab-environments inside of quote-environments, usually because in a single quote I quote the Arabic text followed by my English translation. Jan 16, 2019 at 21:01

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .