Strange csvsimple output when the line starts with an accented character

Question

The code example below produces an undesirable result, where an Omega appears on the 5th line and all data from the 6th line appears glued to the last cell in the 5th line.

Does anyone knows what might be causing this misbehavior?

\documentclass[10pt,a4paper]{report}
\usepackage[utf8]{inputenc}
\usepackage{amsmath}
\usepackage{amsfonts}
\usepackage{amssymb}
\usepackage{csvsimple}
\usepackage{booktabs}

\begin{filecontents*}{corpusabg.csv}
word,frequency,phonetic transcription,phone length,syllabic transcription,syllable length,average duration
de,125749,de,2,CV,1,-1
que,116882,ke,2,CV,1,-1
a,102779,a,1,V,1,-1
o,91246,o,1,V,1,-1
e,87868,e,1,V,1,-1
é,61550,3,1,V,1,-1
eu,46558,eW,2,VG,1,-1
do,46538,do,2,CV,1,-1
\end{filecontents*}

\begin{document}

\newcommand\myhead[1]{\parbox[t]{4em}{\centering\bfseries#1\par\kern1mm}}

\csvreader[no head,column count=6,tabular=rrrrrr,
  table head=\toprule,
  late after first line=\\\midrule,
  table foot=\bottomrule
  ]%
{corpusabg.csv}{}{%
  \csviffirstrow{\myhead{\csvcoli} & \myhead{\csvcolii} & \myhead{\csvcoliii}
    & \myhead{\csvcoliv} & \myhead{\csvcolv} & \myhead{\csvcolvi}
    }{\csvlinetotablerow}
}

\end{document}

Answer 1

The problem is this bit of code in csvsimple.sty:

\if\csv@par\csvline\relax%
  \else%
  \csv@escanline{\csvline}%
  % check and decide
  \csv@opt@checkcolumncount%
\fi%

here \csvline is the line which was just read and \csv@par is \par. When the read line is é,61550,3,1,V,1,-1, the test is:

\if\csv@par é,61550,3,1,V,1,-1\relax%
  \else%
  \csv@escanline{\csvline}%
  % check and decide
  \csv@opt@checkcolumncount%
\fi%

This will (with pdfTeX, of course) expand é, which is \u8:é, which then further expands into some gibberish. After a complete expansion of this, the resulting test is basically:

\if\csv@par\unhbox <strange bytes>é,61550,3,1,V,1,-1\relax%
  \else%
  \csv@escanline{\csvline}%
  % check and decide
  \csv@opt@checkcolumncount%
\fi%

which returns true because \csv@par and \unhbox are the same for \if. Then the code executed is <strange bytes>é,61550,3,1,V,1,-1\relax and you saw what that makes.

The easiest solution is to wrap the é in braces:

{é},61550,3,1,V,1,-1

so that the test compares \csv@par with { which will be false and the code will do the right thing.

If for some reason you can't/don't want to change the CSV file, then you can try this patch:

\def\csv@def@par{\par}
\makeatletter
\patchcmd\csvloop
  {\if\csv@par}
  {\ifx\csv@def@par}
  {}{\ERROR! Failed to patch}

It will replace the problematic test with one that will test for \par without expanding things. BEWARE! I don't know if the purpose of that test it to check for \par or something, so this might break more things than fix. Use at your own risk!

\documentclass[10pt,a4paper]{report}
\usepackage[utf8]{inputenc}
\usepackage{amsmath}
\usepackage{amsfonts}
\usepackage{amssymb}
\usepackage{csvsimple}
\usepackage{booktabs}

\def\csv@def@par{\par}
\makeatletter
\patchcmd\csvloop
  {\if\csv@par}
  {\ifx\csv@def@par}
  {}{\ERROR! Failed to patch}

\usepackage{filecontents}
\begin{filecontents*}{corpusabg.csv}
word,frequency,phonetic transcription,phone length,syllabic transcription,syllable length,average duration
de,125749,de,2,CV,1,-1
que,116882,ke,2,CV,1,-1
a,102779,a,1,V,1,-1
o,91246,o,1,V,1,-1
e,87868,e,1,V,1,-1
é,61550,3,1,V,1,-1
eu,46558,eW,2,VG,1,-1
do,46538,do,2,CV,1,-1
\end{filecontents*}

\begin{document}

\newcommand\myhead[1]{\parbox[t]{4em}{\centering\bfseries#1\par\kern1mm}}

\csvreader[no head,column count=6,tabular=rrrrrr,
  table head=\toprule,
  late after first line=\\\midrule,
  table foot=\bottomrule
  ]%
{corpusabg.csv}{}{%
  \csviffirstrow{\myhead{\csvcoli} & \myhead{\csvcolii} & \myhead{\csvcoliii}
    & \myhead{\csvcoliv} & \myhead{\csvcolv} & \myhead{\csvcolvi}
    }{\csvlinetotablerow}
}

\end{document}