# How to interpret message "Invalid UTF-8 stream" when trying to convert a tex-file to a rtf-file using pandoc?

I want to convert a .tex file into something that is possible to open in MS Word (.doc, .docx, .rtf, ...) by means of the pandoc command line software. A minimum .tex file is attached below. I use the following command in a standard command window when trying to convert :

pandoc -o MyFile.tex MyFile.rtf

This produces the following error message:

What does "Invalid UTF-8 stream" mean, and what can I do about it? I am using MikTeX 2.9 and TexWorks.

My latex file MyFile.tex is like this:

% !TeX program    = pdflatex
% !TeX encoding   = ISO-8859-1
% !TeX spellcheck = nb_NO

\documentclass[10pt,a4paper,twosided]{article}

\usepackage[T1]{fontenc}
\usepackage{amsmath}
\usepackage{amsthm}
\usepackage{amssymb}
%\usepackage[dvips]{graphicx}
\usepackage{graphicx}

%\usepackage[french,english]{babel}
\usepackage[english]{babel}
%\usepackage[latin1]{inputenc}

\usepackage[usenames,dvipsnames]{color}
\usepackage{subfigure}
\usepackage{natbib}
\usepackage{dcolumn}

\usepackage{bm}
\usepackage{url}

\usepackage{alltt}
\usepackage{xspace}
\usepackage{enumerate}
\usepackage{textcomp}
\usepackage{etoolbox}
\usepackage{longtable}
\usepackage{pdflscape}
%\usepackage[absolute,showboxes]{textpos}
\usepackage[absolute]{textpos}

\usepackage{fancyhdr}
\usepackage{array}
\usepackage{tabularx}
\usepackage{xfrac}

\usepackage{fmtcount}
\usepackage{mfirstuc}
\usepackage[width=0.95\textwidth,font={bf,small}]{caption}[2011/08/06]

\usepackage{booktabs}
\usepackage[table]{xcolor}

\usepackage{tikz}
\usepackage{pgfplots}
\pgfplotsset{plot coordinates/math parser=false}

\usepackage{filecontents}
\usepackage[pagewise,displaymath,mathlines]{lineno}
\usepackage{layout}
\usepackage[margin=1in]{geometry}
\usepackage{changepage}
\usepackage{placeins}
\usepackage{paralist}
\usepackage{refcount}

\usepackage[pdftex]{hyperref}

\hypersetup{pdfauthor = Espen Donali}

\hypersetup{pdfdisplaydoctitle=true}
\hypersetup{bookmarks=true}
\hypersetup{pdfpagemode=UseOutlines}
\hypersetup{pdfpagelayout=TwoColumnRight}

\hypersetup{pdftoolbar=true}
\hypersetup{dvips}
\hypersetup{hyperfigures}
\hypersetup{pdfstartview=FitV}

\hypersetup{backref=page}
\hypersetup{pagebackref=true}

\hypersetup{allcolors=blue}
\hypersetup{citecolor=blue}
\hypersetup{urlcolor=blue}
\hypersetup{anchorcolor=red}
\hypersetup{runcolor=red}
\hypersetup{hyperfootnotes=true}

\begin{document}

my test of pandoc latex to word

\end{document}

enter code here


You have told your editor that the file is stored as latin1 encoding

% !TeX encoding   = ISO-8859-1


But pandoc thinks it is reading a UTF-8 encoded file so can not decode it. These encodings are the same for ascii letters but differ for anything else.

Pandoc may have an option to tell it the file encoding or if not, you should get your editor to save the file as utf-8.

• Thanks :) By changing "ISO-8859-1" to "UTF-8" in my minimum example pandoc functioned ! BUT, when doing the same on my real file, from which the minimum example were made from, I get this error: pandoc: Cannot decode byte '\x72' : Data.Text.Encoding.decodeUtf8: Invalid UTF-8 stream. Is there some mismatch between UTF-8 and any of the packages I have used? Feb 12, 2013 at 12:54
• Follow up question: It seems like pandoc needs all latex code to be contained within the sinlge .tex file being processed. But I have numerous occurences of \inpu{} in my masterfile. Are there any functionality in latex letting me construct a new masterfile where all the input files are copied into the masterfile instead of being referred to with the \input{} command ? Feb 12, 2013 at 13:25
• Have look at Martin Scharrers answer to this question: tex.stackexchange.com/questions/21838/…
– DG'
Nov 26, 2013 at 14:07
• @DavidCarlisle I want to pdf hea-www.harvard.edu/~fine/Tech/vi.html document using pandoc -o vi.pdf vi.html that produce pandoc: Cannot decode byte '\xd7': Data.Text.Encoding.Fusion.streamUtf8: Invalid UTF-8 stream error. But, as it is html file how do I know the encoding? may be ascii. Feb 12, 2018 at 2:29

Pandoc uses the UTF-8 character encoding for both input and output. If your local character encoding is not UTF-8, you should pipe input and output through iconv:

iconv -t utf-8 input.txt | pandoc | iconv -f utf-8

• I get iconv: template.html:411:54: cannot convert Mar 21, 2017 at 5:10
• when using vi/vim to view the .tex file use :set fileencoding=utf-8 and save ist. to show the current fileencoding in vim just type :set fileencoding oder :set fileencoding? Additionally maybe look whats inside :set fileencodings, too Aug 11, 2017 at 9:46
• @incandescentman I figured how to avoid the "iconv: cannot convert" error from the command line. I just tried iconv -f ISO-8859-1 -t UTF-8 template.html and it worked fine. It seems iconv is also not correctly reading the input encoding, so we have to specify it manually. Aug 18, 2018 at 6:02

# Check JabRef encoding

In my case, I bumped into a similar error when converting Pandoc Markdown to XHTML. The culprit were a set of BibTeX citations which JabRef had encoded by default in ISO8859_1.

This default JabRef behaviour can be changed once and for all by setting Default encoding: to UTF8 in JabRef's Options > Preferences > General menu.

• Good thought, but I don't know enough about how pandoc is handling references. I just used iconv on the .bbl and .aux file too but that didn't seem to do it. May 15, 2017 at 19:27