I am currently using csvsimple to include some external data in a document processed with pdfLaTeX. The entire project uses UTF-8 as encoding. The original data is maintained in an Excel 2010 file, and since Excel does not support exporting to UTF-8 CSV directly, I have to go through a rather cumbersome process:

  • export to CSV
  • open with Notepad
  • save from notepad, changing the encoding

This process adds a BOM to the CSV file - 0xef 0xbb 0xbf. On my Windows desktop, this does not seem to be a problem, but the CI build on a Linux box breaks down with Missing \endcsname inserted.. Is there a way to tell csvsimple to ignore the BOM or do I have to edit it out before the compilation starts?

Related: Google docs to TeX and pdf shell script produces a blank first page, and gives me a “! LaTeX Error: Missing \begin{document}.” error

  • 1
    Several editors don't add a BOM, which seems to be characteristic with Notepad. There are utilities that change the encoding without the need of opening an editor (and don't add the BOM) like Charco but there are surely several others. I'm quite surprised that a well known application doesn't allow exporting as UTF-8 in 2014 (29 years after its first release). – egreg Oct 31 '14 at 9:13
  • @egreg You're right, but I was equally surprised that a well known (text) file processing software is unable to handle BOMs in the input :-) – vwegert Oct 31 '14 at 9:19
  • TeX predates UTF-8 by several years. Maybe this could be a feature request for the next release of TeX Live. – egreg Oct 31 '14 at 9:23
  • Is using another editor than Notepad an option? Notepad++ e.g. allows to save without BOM. – Thomas F. Sturm Oct 31 '14 at 9:36
  • 1
    Do you use Notepad or Notepad++? With Notepad++ you can configure the encoding of the file. See menu encoding, switch to utfß. – Mensch Oct 31 '14 at 9:54

As you input the file you could try to simply declare the BOM:

  • Would this answer have solved the following question?: tex.stackexchange.com/q/284916/90087 If yes, it would be nice to have it posted as an answer. – A Feldman Apr 16 '16 at 17:38
  • @AFeldman: No, in your case the \DeclareUnicodeCharacter would be too late. – Ulrike Fischer Apr 16 '16 at 17:41
  • Thank you, appreciated. I was thinking that it would be preferable to have an answer that did not depend on sed and an external shell script in dealing with the google doc BOM. – A Feldman Apr 16 '16 at 17:42

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.