# Comment character in datatool AND ignore empty lines?

Here is a minimal example on using the datatool package:

\documentclass[a4paper,10pt]{minimal}
\usepackage[utf8]{inputenc} \usepackage{datatool}
\begin{document}

{\DTLsettabseparator%
\DTLloadrawdb[noheader,keys={T,R}]{data}{data.csv}}% Brackets necessary in conjunction with some other packages that don't like what DLTsettabseparator does

\begin{tabular}{c|c}
\DTLforeach{data}{\first=T,\last=R}{ $\first\ K$ & $\last\ k\Omega$ \\ }
\end{tabular}

\end{document}


data.csv:

# This is 'data.csv'
# I'd like comments like this to be ignored by datatool

# And empty lines as well!

# Now comes the data
# Format:
# T[K]    R[kΩ]
1.4 12.79
8.1 8.81
11.4 8.24
17.3 6.63
25.7 5.27
31.2 3.58
39.0 3.03
44.3 2.38
54.7 1.72
61.2 1.21
69.7 0.987
81.0 0.718
88.5 0.626
99.6 0.491


Is there a way to suppress output of lines that start with a # and also suppress the output of empty lines? Or even define a comment character?

Edit: Added a sample .csv file.

-
We could do with a minimal example .csv file –  Joseph Wright Oct 19 '11 at 18:33
I checked the RFC4180 which defines the csv format and unfortunately neither comments nor empty lines are supported (the csv ABNF grammar can give you more information on this). In this case, the format you want to use extends the original csv definition, so it's probably beyond the scope of the datatool package. IMHO the solution might be implementing a custom parser with both comments and empty lines rules defined. :) –  Paulo Cereda Oct 19 '11 at 20:03
I've lodged a feature request with Nicola Talbot on this. –  Joseph Wright Oct 20 '11 at 17:00

I'm not sure if you can get datatool to handle this, but pgfplotstable, which has similar functionality, ignores comment lines and empty lines out of the box. The following code will typeset a simple table of your data:

\documentclass{article}
\usepackage{pgfplotstable}
\usepackage{filecontents}
\begin{filecontents}{testdata.csv}
# This is 'data.csv'
# I'd like comments like this to be ignored by datatool

# And empty lines as well!

# Now comes the data
# Format:
# T[K]    R[kΩ]
1.4 12.79
8.1 8.81
11.4 8.24
17.3 6.63
25.7 5.27
31.2 3.58
39.0 3.03
44.3 2.38
54.7 1.72
61.2 1.21
69.7 0.987
81.0 0.718
88.5 0.626
99.6 0.491
\end{filecontents}

\begin{document}
\pgfplotstabletypeset{\testdata}
\end{document}


I thought you might be interested in what else you can do with this data in LaTeX. Below is an example where I use pgfplotstable to typeset the results in a prettier way, using the lines provided by booktabs. I also plot the data using pgfplots, and fit an exponential function using gnuplot. The fit parameters are displayed on the plot. All of this is done using a single TeX file and a single compilation run. If your data changes, the table, graph and fit parameters will all automatically reflect the changes:

\documentclass{article}
\usepackage{pgfplots}   % For plotting
\usepgfplotslibrary{units}  % For typesetting units in plot
\usepackage{pgfplotstable} % For typesetting tables from data
\usepackage{siunitx} % For typesetting units
\usepackage{booktabs} % Nicer horziontal lines and corresponding spacing
\usepackage{filecontents} % To include the data file in this .tex file
\begin{filecontents}{testdata.csv}
# This is 'data.csv'
# I'd like comments like this to be ignored by datatool

# And empty lines as well!

# Now comes the data
# Format:
# T[K]    R[kΩ]
1.4 12.79
8.1 8.81
11.4 8.24
17.3 6.63
25.7 5.27
31.2 3.58
39.0 3.03
44.3 2.38
54.7 1.72
61.2 1.21
69.7 0.987
81.0 0.718
88.5 0.626
99.6 0.491
\end{filecontents}

\begin{document}
\pgfplotstabletypeset[
fixed zerofill, % Print numbers in fixed format, add zeros if necessary
dec sep align, % Align numbers at the decimal point
columns/0/.style={ % Options for the first column
precision=1, % One decimal
column name={
\shortstack{$T$\\\si{\kelvin}} % Define column name. Use \shortstack to get two lines
}
},
columns/1/.style={
column name={
\shortstack{$R$\\\si{\kilo\ohm}}
}
},
every head row/.style={ % Set style for title row
before row=\toprule, % Rules from booktabs package
after row=\midrule
},
every last row/.style={ % Set style for final row
after row=\bottomrule
}
]{\testdata}
\hspace{1cm}
\begin{tikzpicture}[baseline] % "baseline" for correct vertical alignment with the table
\begin{axis}[
anchor=center, % For correct vertical alignment with the table
ymin=0, enlarge y limits=upper, % Fix lower y limit at zero, increase upper limit a bit
xmin=0,xmax=100, % Fix lower and upper limits for x
axis lines*=left, % Axis lines without arrow tips at the left and bottom
unit code/.code 2 args={\si{#1#2}}, % Use siunitx for typesetting units
xlabel=$T$, x unit=\kelvin, % Set labels and units
ylabel=$R$, y unit=\kilo\ohm, ylabel style={rotate=-90}] % Typeset y label horizontally
\addplot [black, only marks] table \testdata; % Add the data points from the table
\addplot+[raw gnuplot, draw=red, no markers, smooth] gnuplot { % Issue gnuplot command, plot as smoothed line
f(x)=a*exp(b*x); % Define function to fit
a=15; % Initial values
b=-1;
fit f(x) 'testdata.csv' using 1:2 via a,b; % Perform fit
plot [x=0:100] f(x); % calculate points for curve
set print "parameters.dat"; % Open a file to save the parameters into
print a,b; % Write the parameters to file
};
\pgfplotsextra{ % Execute code at end of plot, when everything else has been run
\pgfplotstableread{parameters.dat}\parameters % Open the file Gnuplot wrote
\pgfplotstablegetelem{0}{0}\of\parameters \pgfmathsetmacro\paramA{\pgfplotsretval} % Get first element, save into \paramA
\pgfplotstablegetelem{0}{1}\of\parameters \pgfmathsetmacro\paramB{\pgfplotsretval}
\node at (axis cs:5,10) [ % Position a node at the logical coordinate (4,10)
pin=5:$\pgfmathprintnumber{\paramA} \cdot \mathrm{e}^{\pgfmathprintnumber[fixed,precision=3]{\paramB}\cdot T}$] {}; % Typeset equation using the pin argument.
}
\end{axis}
\end{tikzpicture}
\end{document}

-
Wow. I thank you kindly for the work you put into this! I really appreciate the beatufull answer. As plotting goes I've pretty much settled with gnuplot. Not going to switch as I'm very comfortable with it. I just wanted a way typeset tables from my gnuplot data files automatically, i.e. without copying the data into the latex file manually. –  con-f-use Oct 20 '11 at 8:18