2

I am currently using csvsimple to include some external data in a document processed with pdfLaTeX. The entire project uses UTF-8 as encoding. The original data is maintained in an Excel 2010 file, and since Excel does not support exporting to UTF-8 CSV directly, I have to go through a rather cumbersome process:

  • export to CSV
  • open with Notepad
  • save from notepad, changing the encoding

This process adds a BOM to the CSV file - 0xef 0xbb 0xbf. On my Windows desktop, this does not seem to be a problem, but the CI build on a Linux box breaks down with Missing \endcsname inserted.. Is there a way to tell csvsimple to ignore the BOM or do I have to edit it out before the compilation starts?

Related: Google docs to TeX and pdf shell script produces a blank first page, and gives me a “! LaTeX Error: Missing \begin{document}.” error

Improve this question
9
  • 1
    Several editors don't add a BOM, which seems to be characteristic with Notepad. There are utilities that change the encoding without the need of opening an editor (and don't add the BOM) like Charco but there are surely several others. I'm quite surprised that a well known application doesn't allow exporting as UTF-8 in 2014 (29 years after its first release).
    – egreg
    Oct 31, 2014 at 9:13
  • @egreg You're right, but I was equally surprised that a well known (text) file processing software is unable to handle BOMs in the input :-)
    – vwegert
    Oct 31, 2014 at 9:19
  • TeX predates UTF-8 by several years. Maybe this could be a feature request for the next release of TeX Live.
    – egreg
    Oct 31, 2014 at 9:23
  • Is using another editor than Notepad an option? Notepad++ e.g. allows to save without BOM.
    – Thomas F. Sturm
    Oct 31, 2014 at 9:36
  • 1
    Do you use Notepad or Notepad++? With Notepad++ you can configure the encoding of the file. See menu encoding, switch to utfß.
    – Mensch
    Oct 31, 2014 at 9:54

1 Answer 1

Reset to default
4

As you input the file you could try to simply declare the BOM:

\documentclass[]{book}
\usepackage[utf8]{inputenc}
\DeclareUnicodeCharacter{FEFF}{}
\begin{document}
\input{test-with-bom}
\end{document}
Improve this answer
3
  • Would this answer have solved the following question?: tex.stackexchange.com/q/284916/90087 If yes, it would be nice to have it posted as an answer.
    – A Feldman
    Apr 16, 2016 at 17:38
  • @AFeldman: No, in your case the \DeclareUnicodeCharacter would be too late.
    – Ulrike Fischer
    Apr 16, 2016 at 17:41
  • Thank you, appreciated. I was thinking that it would be preferable to have an answer that did not depend on sed and an external shell script in dealing with the google doc BOM.
    – A Feldman
    Apr 16, 2016 at 17:42

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .