# pgfplotstable's create col/expr does not invoke fpu on constants

I must admit that I am frequently stumbling over the limited numerical precision issues with pgfplots(table). I am aware that \pgfset{fpu} activates the IEEE double precision unit. Moreover, the latest manual (v1.15) states:

This FPU is also part of PgfplotsTable, and it is activated by default for create col/expr ...

With that being said, please help me understand my questions about the following MWE. Take note of the three different expressions for create col/expr={abs(...)}.

\documentclass{article}
\usepackage{pgfplotstable}

\begin{filecontents}{table.csv}
values
392.50271975
392.5024343
392.5016196
392.5005834
392.4992593
\end{filecontents}

\pgfplotstableset{
create on use/error/.style={
create col/expr={
abs(785/2 )%-\thisrow{values})
%abs(392.5-\thisrow{values})
}
},
}

\begin{document}
\pgfplotstabletypeset[
columns={error},
columns/error/.style={
column name={error},
fixed,
precision=8,
},
]\myerrortable
\end{document}

1. abs(785/2 ) = 392.499: Why are only three digits printed? This does not look like the fpu is used.
2. abs(785/2 -\thisrow{values}) = 0.00372, ...: As you see the values table contains more precision digits and they are partially used. The error is always in the third digit resulting from the limited precision result of 785/2. So, why is 785/2 limited to 3 digits, and the values to 5?
3. abs(392. - \thisrow{values}) = 0.00272, ...: If the correct result for 785/2 is used directly, the output is now correct for the given 5 precision digits. But why only 5 digits? The values column contains more and I requested 8. I thought fpu gives full double precision?
• I have not accepted Marijn's answer, yet, as I would really like to see an answer to my questions above. Is this a bug in pgfplots(table) or am I doing/understanding something fundamentally wrong? Why are the calculations taking place at two different precision levels? Sep 25, 2018 at 10:59

As a workaround you can use the expl3 FPU (see, e.g., How to use the TikZ fpu library?).

MWE:

\documentclass{article}
\usepackage{pgfplotstable}
\pgfplotsset{compat=1.15}
\usepackage{expl3}
\ExplSyntaxOn
\cs_new_eq:NN \fpeval \fp_eval:n
\ExplSyntaxOff
\begin{filecontents}{table.csv}
values
392.50271975
392.5024343
392.5016196
392.5005834
392.4992593
\end{filecontents}

\pgfplotstableset{
create on use/error/.style={
create col/expr={
\fpeval{abs(785/2-\thisrow{values})}
}
},
}

\begin{document}
\pgfplotstabletypeset[
columns={values,error},
columns/values/.style={column name={value},fixed,precision=8,},
columns/error/.style={column name={error},fixed,precision=8,},
]\myerrortable
\end{document}


Result:

• Thank you, Marijn, for that workaround! This is indeed very useful to get out of my current misery and keep my workflow using pgfplots. Sep 23, 2018 at 20:29