# pgfplots - insufficient precision

I am having the following problem: Using pgfplots to plot some convergence graphs, I have to load a text file and do some computations. Most of the time everythings works just fine. But in some situations, the 4-6 significant digits for pgfplots are insufficient.

How can I modify the following code to get my desired graphs? I have read about using gnuplot, but I could not find a similar example for my particular usecase. And the pgfplots manual does say anything about loading a table and doing computaions in gnuplot ot the shell in Sections 4.2.6 or 4.2.7. Is there another way to achieve this? I do have a lot of graphs to generate, so a semi-automatic solution as with the pgfplots code below would be preferable.

\begin{tikzpicture}
\begin{semilogyaxis}
\addplot table[x expr={\thisrowno{0}^0.2}, y expr={sqrt(abs(0.80336293803-\thisrowno{}))}] {data/convRates.txt};
\end{semilogyaxis}
\end{tikzpicture}


convRates.txt

1       0.6123724356960
155     0.8002396840450
1115        0.8032455664310
4519        0.8033592830220
13391       0.8033626913420
32579       0.8033629179460
69187       0.8033629362110
133007      0.8033629378560
236951      0.8033629380130
397483      0.8033629380290


Thanks for any help.

• Welcome to TeX - LaTeX! Have you tried using the floating point library? – Andrew Swann Apr 21 '16 at 15:27
• Hallo, reading the manual, I thought \pgfkeys{/pgf/fpu=true}was activated by default using the \addplot tablecommand. However, setting this explicitely in the preamble I know get a lot of errors: “[...] floating point unit got an ill-formed floating point number near [...]” So I guess that fpu was not set to true ... – erniepb Apr 21 '16 at 15:47
• @erniepb These errors are due to the fact that you try to use comma as decimal separator which is not really supported. Please always use the period instead! – Henri Menke Apr 21 '16 at 15:48
• @Henri Menke: In fact I am using the dot for the decimal separator, but copied the selected columns from libreoffice with a comma. Fixed this in the question. – erniepb Apr 21 '16 at 16:30
• @Henri Menke: Despite your (very nice) answer. Wha could be the reason for the many errors when I set \pgfkeys{/pgf/fpu=true}. It is definitely not the comma-separator as I am actually using the period-separator. – erniepb Apr 21 '16 at 16:40

Instead of using pgfplots to evaluate the expression you could use l3fp which guarantees full 16-digit floating point precision.

Also, always use period as the decimal separator.

\documentclass{article}
\usepackage{pgfplots,xparse}
\ExplSyntaxOn
\DeclareExpandableDocumentCommand \eval { m } { \fp_eval:n { #1 } }
\ExplSyntaxOff
\begin{document}
\begin{tikzpicture}
\begin{semilogyaxis}
x expr={\eval{(\thisrowno{0})^0.2}},
y expr={\eval{sqrt(abs(0.80336293803-\thisrowno{1}))}}
] {
1      0.6123724356960
155    0.8002396840450
1115   0.8032455664310
4519   0.8033592830220
13391  0.8033626913420
32579  0.8033629179460
69187  0.8033629362110
133007 0.8033629378560
236951 0.8033629380130
397483 0.8033629380290
};
\end{semilogyaxis}
\end{tikzpicture}
\end{document}


• @erniepb So sorry, see updated answer. – Henri Menke Apr 21 '16 at 16:11