2

I'm trying to plot a set of 10000 points and I'm getting the TeX capacity exceeded, sorry [main memory size=5000000]. error.
I'm using externalization.

Here is my MWE.

\documentclass{report}

\usepackage[utf8]{inputenc}
\usepackage{amsmath}
\usepackage{amsfonts}
\usepackage{amssymb}
\usepackage{tikz}
\usepackage{pgfplots}
\pgfplotsset{compat=1.14}
\usepgfplotslibrary{external}
\tikzexternalize

\usepackage[arrowmos,european]{circuitikz}
\usepackage{siunitx}
\usepackage{float}
\usepackage{hyperref}
\hypersetup{
    colorlinks,
    citecolor=black,
    filecolor=black,
    linkcolor=black,
    urlcolor=blue
}   
\usepackage{geometry}
\geometry{
    a4paper,
    total={170mm,257mm},
    left=20mm,
    top=20mm,
}

\newcommand*{\subb}[1]{_{\mathrm{#1}}}
\newcommand*{\supp}[1]{^{\mahtrm{#1}}}

\begin{document}
    \begin{figure}[H]
        \begin{tikzpicture}
            \begin{axis}[xlabel=a,ylabel=b,width=\linewidth]% coordinates
            \foreach \i in {1,...,10}
            \addplot table[x index=0, y index=\i, col sep = comma] {data.csv};
            \end{axis}
        \end{tikzpicture}
    \end{figure}
\end{document}

And data.csv file is just a random set of 10000 points (later will be replaced with actual data).

Is there any solution? Or does TeX simply cannot handle that much data?

Edit


Using a set of only 200 points the output is as expected: enter image description here

  • Does it work with only some points? Could you please prodive an example data set with some points for compilation and testing? – Bobyandbob Nov 8 '17 at 16:11
  • It does work with 200 points. See edit. – jagjordi Nov 8 '17 at 16:13
  • 2
    externalize isn't going to help if you only have one tikzpicture as basically that tries to run each tikzpicture as a separate docuemnt and include them, but the generated separate document is going to be no simpler than the original docuemnt in this case. Can the human reader see 10000 points? Can you not sample the data using some external software prior to plotting? – David Carlisle Nov 8 '17 at 16:13
  • 2
    Try lualatex? lualatex dynamically allocates memory and probably won't throw out of memory errors. – Hood Chatham Nov 8 '17 at 16:22
  • Yes @DavidCarlisle, this is what I'm going to do. Sample 500 or 1000 points. – jagjordi Nov 8 '17 at 16:35
1

This is not an answer, more an extended comment
An answer (use lualatex) was already given in the comment by Hood Chatham.

(But I think it's interesting enough to merit this.)

For plots with a lot of data points, marks have a huge influence on used memory. I tested this for a few marks with 500 points and 10 plots. Here are the results (as ASCII table):

option              data points     remark
                    500      250
                ----------------
[mark=none]     1170026   947012    approx. 0.87k/data point
[mark=square]   1268175
[mark=triangle] 1485689
[mark=diamond]  1614367
[mark=pentagon] 1743355
[mark=+]        1749165
[mark=x]        1749223
[](1)           2197715  1458330    approx. 2.89k/data point
[mark=star]     2351954
[mark=*]        2621674
[mark=o]        2621674
[mark=otimes]   3136768
[mark=ball](2) >3500000  3433143
----------
(1) default mix of marks (*, square*, otimes*, star, diamond*)
(2) TeX capacity exceeded

So if lualatex can't be used, doing such plots without marks would be a good idea. Or at least the marks should be selected carefully.

The data was generated similar to David Carlisles comment with

c=0; while [ $c -le 500 ] ; do echo "$c,$RANDOM,$RANDOM,$RANDOM,$RANDOM,$RANDOM,$RANDOM,$RANDOM,$RANDOM,$RANDOM,$RANDOM"; let "c +=1"; done > data-500.csv

And the tests were done with

\documentclass[border=1mm]{standalone}
\usepackage{pgfplots}
\pgfplotsset{compat=1.15}
\begin{document}
\begin{tikzpicture}
    \begin{axis}[xlabel=a,ylabel=b,width=\linewidth]% coordinates
    \foreach \i in {1,...,10}
        \addplot+[] table[x index=0, y index=\i, col sep = comma] {data-500.csv};
    \end{axis}
\end{tikzpicture}
\end{document}

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.