12

Is there a LaTeX package, which would import data from csv and honour the quotation escaping?

To be exact I mean one using the rules 1. - 7. of http://tools.ietf.org/html/rfc4180#section-2

Neither pgfplotstable nor simplecsv seem to work - they both incorrectly parse the quoted comma as field separator.

column1, column2
value,"A quoted, value"
value2,A value
value3,"a ""value"""

Line 1 (of data) results in a pgfplotstable error: Table 'data.csv' appears to have too many column. Header, lines 2 and 3 render fine.

Desired output is:

column1  |  column2
---------+---------------------
value    |  A quoted, value
value2   |  A value
value3   |  a "value"

The only solution I found is manually enclosing the fields with {} brackets, but it's obviously a hassle.

EDIT: in the original file only the fields that contain commas and quotes are quoted

EDIT2: I understand it might be complicated to change the quotations to proper '' `` pairs; I'm not interested in this.

  • 1
    Welcome to TeX.SX! You can have a look on our our starter page to familiarize yourself further with our format. – Claudio Fiandrino Jul 8 '13 at 12:38
  • would it be OK if we ignore the " and join the resulting columns? I mean are there always commas present inside the quotes? – percusse Jul 8 '13 at 12:58
  • You can use the datatool package. It parses the data correctly – Carsten Thiel Jul 8 '13 at 13:47
  • Had the same CSV problem; ended up writing a custom exporter for tab-separated values instead - worked right away with pgfplotstable. – Joel Purra Sep 12 '14 at 14:12
  • Good question. I also thought I could get away with csvsimple, but, alas, quotes were not supported, and I had a strange issue with special characters like underscore turning into an elevated bullet point, but only in default font. Anyway, I have the luxury of generating the latex, so I ended up doing a CSV parser in another language and writing straight to latex longtable. – icy Sep 25 '14 at 13:49
6

It seems that it is not implemented but can be done with some labor. I've modified the existing macros rather too quickly so Christian should know if I'm breaking anything here. The new setting can be used via col sep=double quotes. But it doesn't support another column with quotes as it is an ambiguous character for opening and closing.

\documentclass{standalone}
\usepackage{pgfplotstable}



\makeatletter
\pgfkeys{/pgfplots/table/col sep/double quotes/.code = {\def\pgfplotstableread@COLSEP@CASE{7}}}

{%
    \catcode`\ =10
    \catcode`\;=12
    \catcode`\:=12
    \gdef\pgfplotstableread@impl@prepare@DO{%
        \ifcase\pgfplotstableread@COLSEP@CASE\relax
            % col sep=space:
            \catcode`\ =10
            \long\def\pgfplotstableread@impl@DO@##1{\pgfplotstableread@impl@ITERATE##1 \pgfplotstable@EOI}%
        \or
            % col sep=comma:
            \let\pgfplotstableread@impl@ITERATE@NEXT=\pgfplotstableread@impl@ITERATE@NEXT@COMMA
            \long\def\pgfplotstableread@impl@DO@##1{\pgfplotstableread@impl@ITERATE##1,\pgfplotstable@EOI}%
        \or
            % col sep=semicolon:
            \catcode`\;=12
            \let\pgfplotstableread@impl@ITERATE@NEXT=\pgfplotstableread@impl@ITERATE@NEXT@SEMICOLON
            \long\def\pgfplotstableread@impl@DO@##1{\pgfplotstableread@impl@ITERATE##1;\pgfplotstable@EOI}%
        \or
            % col sep=colon:
            \catcode`\:=12
            \let\pgfplotstableread@impl@ITERATE@NEXT=\pgfplotstableread@impl@ITERATE@NEXT@COLON
            \long\def\pgfplotstableread@impl@DO@##1{\pgfplotstableread@impl@ITERATE##1:\pgfplotstable@EOI}%
        \or
            % col sep=brace:
            % allow multi line cells:
            \endlinechar=\pgfplotstableread@oldendlinechar\relax
            \let\pgfplotstableread@impl@ITERATE@NEXT=\pgfplotstableread@impl@ITERATE@NEXT@BRACE
            \long\def\pgfplotstableread@impl@DO@##1{\pgfplotstableread@impl@ITERATE##1\pgfplotstable@EOI}%
        \or
            % col sep=tab:
            \catcode`\^^I=12
            \let\pgfplotstableread@impl@ITERATE@NEXT=\pgfplotstableread@impl@ITERATE@NEXT@TAB
            \long\edef\pgfplotstableread@impl@DO@##1{\noexpand\pgfplotstableread@impl@ITERATE##1\pgfplotstableread@tab\noexpand\pgfplotstable@EOI}%
        \or
            % col sep=&:
            \let\pgfplotstableread@impl@ITERATE@NEXT=\pgfplotstableread@impl@ITERATE@NEXT@AMPERSAND
            \long\def\pgfplotstableread@impl@DO@##1{\pgfplotstableread@impl@ITERATE##1&\pgfplotstable@EOI}%
        \or
            % col sep="
            \let\pgfplotstableread@impl@ITERATE@NEXT=\pgfplotstableread@impl@ITERATE@NEXT@DOUBLEQUOTES
            \long\def\pgfplotstableread@impl@DO@##1{\pgfplotstableread@impl@ITERATE##1\pgfplotstable@EOI}%
        \fi
    }%
}%

\long\def\pgfplotstableread@impl@ITERATE@NEXT@DOUBLEQUOTES#1"{%
    \pgfplotstableread@impl@ITERATE@NEXT@{#1}%
    \pgfplotstableread@impl@ITERATE
}%

\makeatother


\begin{document}

\pgfplotstableread[col sep=double quotes]{
A,"the comma , and some    space,"
B," followed by another awkward."
}\mytable


\pgfplotstabletypeset[string type]{\mytable}

\end{document}

enter image description here

I didn't remove the commas because I am not sure if you need them or not. You can also introduce the quotes again but if that's going to be typeset it's better to use csquotes package or at least `` ... " pairs to render the opening quotes properly.

  • 1
    OK, I guess I'll just change the input file with sed or something then. I'd rather run another command after generating data, than having to deal with 2 screens of tex magic – Jakub Bochenski Jul 8 '13 at 14:30
  • @Jakub: writing parsers is hard in any language and CSV is a really tricky format(e.g. look at the CSV parser written in Java or Ruby). A complete parser in awk will be equally awkward. – Aditya Jul 8 '13 at 14:43
  • @JakubBochenski Your edit is a completely different problem since you need to keep track of the number of " pairs. And the pairs are ambigous because opening and closing quotes are identical. The standard uses double quotes similar to what TeX uses braces for and it's a weird choice in my opinion. – percusse Jul 8 '13 at 15:02
  • @Aditya - I just pointed out that in practice I prefer to go with a simpler solution. I'd rather wrap a few lines in my output than use something I don't (want to spend time to) understand – Jakub Bochenski Jul 8 '13 at 16:25
  • 1
    @JakubBochenski: I didn't mean to say that your example was awkward, but the CSV has awkward grammar. See secretgeek.net/csv_trouble.asp – Aditya Jul 8 '13 at 19:21

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.