Take the 2-minute tour ×
TeX - LaTeX Stack Exchange is a question and answer site for users of TeX, LaTeX, ConTeXt, and related typesetting systems. It's 100% free, no registration required.

I'd like to do some simple computation in my table and it looks like I'm seeing some round-off errors.

Does anybody know the precision that pgfplotstable uses internally? Is there any work around besides that following really ugly hack, that I actually don't know, why it's working at all?

Here's my concrete MWE:

\documentclass{article}
\usepackage{pgfplotstable}
\begin{document}
  \pgfplotstableread[col sep = comma]{
    a,         b,
    640000,    3200,  
    1920000,   3200,
    3200000,   3200,
    6400000,   3200,
    12800000,  3200,
  } \data
  \begin{table}
    \pgfplotstableset{create on use/c/.style={create col/expr={2*\thisrow{a}+5*\thisrow{b}}}}
   \pgfplotstableset{create on use/d/.style={create col/expr={(2*(\thisrow{a}*256)+5*(\thisrow{b}*256))/256}}}
   \pgfplotstabletypeset[columns={a,b,c,d},
     columns/a/.style={column name={a}, fixed, precision=0, column type={r|}, },
     columns/b/.style={column name={b}, fixed, precision=0, column type={r|}, },
     columns/c/.style={column name={formular}, fixed, precision=0, column type={r|}, },
     columns/d/.style={column name={Hack}, fixed, precision=0, column type={r|}, },
   ]{\data}
 \end{table}
\end{document}

Thanks a lot, Juhui

share|improve this question
add comment

2 Answers 2

This is not really an answer, but a workaround that might be feasible for you: If you can use lualatex instead of pdflatex, you can use a \directlua statement to do the calculations:

\documentclass{article}
\usepackage{pgfplotstable}
\begin{document}
  \pgfplotstableread[col sep = comma]{
    a,         b,
    640000,    3200,  
    1920000,   3200,
    3200000,   3200,
    6400000,   3200,
    12800000,  3200,
  } \data


  \begin{table}
    \pgfplotstableset{create on use/c/.style={create col/expr={\directlua{tex.sprint(2*\thisrow{a}+5*\thisrow{b})}}}}
   \pgfplotstableset{create on use/d/.style={create col/expr={(2*(\thisrow{a}*256)+5*(\thisrow{b}*256))/256}}}
   \pgfplotstabletypeset[columns={a,b,c,d},
     columns/a/.style={column name={a}, fixed, precision=0, column type={r|}, },
     columns/b/.style={column name={b}, fixed, precision=0, column type={r|}, },
     columns/c/.style={column name={Lua}, fixed, precision=0, column type={r|}, },
     columns/d/.style={column name={Hack}, fixed, precision=0, column type={r|}, },
   ]{\data}
 \end{table}
\end{document}
share|improve this answer
    
Hi Jake, thanks for your fast reply, but unfortunately lualatex is not a feasible solution. Sorry, thanks anyway. –  Juhui Sep 19 '12 at 13:21
add comment

This is a (standard) rounding issue. You would probably see the same if you would compute these numbers in a C program with floats.

A solution is either to use a different math parser which has higher precision (like Jake's answer) or to round the result.

Since you cannot easily increase the precision of the routines in pgfplotstable, you can round the result... which is feasible for this case. The solution would be to use fixed relative: it rounds to the first <precision> nonzero digits. If you use 5 or 6 as precision, you have the precision of the math parser. That is quite reasonably IMHO.

\documentclass{article}
\usepackage{pgfplotstable}
\begin{document}
  \pgfplotstableread[col sep = comma]{
    a,         b,
    640000,    3200,  
    1920000,   3200,
    3200000,   3200,
    6400000,   3200,
    12800000,  3200,
  } \data
  \begin{table}
    \pgfplotstableset{create on use/c/.style={create col/expr={2*\thisrow{a}+5*\thisrow{b}}}}
   \pgfplotstableset{create on use/d/.style={create col/expr={(2*(\thisrow{a}*256)+5*(\thisrow{b}*256))/256}}}
   \pgfplotstabletypeset[columns={a,b,c,c,d},
     columns/a/.style={column name={a}, fixed, precision=0, column type={r|}, },
     columns/b/.style={column name={b}, fixed, precision=0, column type={r|}, },
     columns/c/.style={column name={formular}, fixed, precision=0, column type={r|}, },
     display columns/3/.style={
        column name={formular*}, 
        fixed relative, precision=5, 
        column type={r|},
     },
     columns/d/.style={column name={Hack}, fixed, precision=0, column type={r|}, },
   ]{\data}
 \end{table}
\end{document}

enter image description here

Note that there is also a relative style in the number printer. That would be even better here -- but I designed it in a stupid way and a fix will only be available in the next version (currently, it is useful for numbers less than 1).

share|improve this answer
add comment

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.