4

I must admit that I am frequently stumbling over the limited numerical precision issues with pgfplots(table). I am aware that \pgfset{fpu} activates the IEEE double precision unit. Moreover, the latest manual (v1.15) states:

This FPU is also part of PgfplotsTable, and it is activated by default for create col/expr ...

With that being said, please help me understand my questions about the following MWE. Take note of the three different expressions for create col/expr={abs(...)}.

\documentclass{article}
\usepackage{pgfplotstable}

\begin{filecontents}{table.csv}
values
392.50271975
392.5024343
392.5016196
392.5005834
392.4992593
\end{filecontents}

\pgfplotstableread[col sep=comma]{table.csv}\myerrortable

\pgfplotstableset{
    create on use/error/.style={
        create col/expr={
            abs(785/2 )%-\thisrow{values})
            %abs(392.5-\thisrow{values})
        }
    },
}


\begin{document}
\pgfplotstabletypeset[
    columns={error},
    columns/error/.style={
        column name={error},
        fixed,
        precision=8,
    },
    ]\myerrortable
\end{document}
  1. abs(785/2 ) = 392.499: Why are only three digits printed? This does not look like the fpu is used.
  2. abs(785/2 -\thisrow{values}) = 0.00372, ...: As you see the values table contains more precision digits and they are partially used. The error is always in the third digit resulting from the limited precision result of 785/2. So, why is 785/2 limited to 3 digits, and the values to 5?
  3. abs(392. - \thisrow{values}) = 0.00272, ...: If the correct result for 785/2 is used directly, the output is now correct for the given 5 precision digits. But why only 5 digits? The values column contains more and I requested 8. I thought fpu gives full double precision?
1
  • I have not accepted Marijn's answer, yet, as I would really like to see an answer to my questions above. Is this a bug in pgfplots(table) or am I doing/understanding something fundamentally wrong? Why are the calculations taking place at two different precision levels? Sep 25 '18 at 10:59
3

As a workaround you can use the expl3 FPU (see, e.g., How to use the TikZ fpu library?).

MWE:

\documentclass{article}
\usepackage{pgfplotstable}
\pgfplotsset{compat=1.15}
\usepackage{expl3}
\ExplSyntaxOn
\cs_new_eq:NN \fpeval \fp_eval:n
\ExplSyntaxOff
\begin{filecontents}{table.csv}
values
392.50271975
392.5024343
392.5016196
392.5005834
392.4992593
\end{filecontents}

\pgfplotstableread[col sep=comma]{table.csv}\myerrortable
\pgfplotstableset{
    create on use/error/.style={
        create col/expr={
            \fpeval{abs(785/2-\thisrow{values})}
        }
    },
}

\begin{document}
\pgfplotstabletypeset[
    columns={values,error},
    columns/values/.style={column name={value},fixed,precision=8,},
    columns/error/.style={column name={error},fixed,precision=8,},
    ]\myerrortable
\end{document}

Result:

enter image description here

1
  • Thank you, Marijn, for that workaround! This is indeed very useful to get out of my current misery and keep my workflow using pgfplots. Sep 23 '18 at 20:29

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.