1

Good Day,

I'll explain my situation for reference before I dive into the question.

I am writing a fairly lengthy report in LaTeX. The report will probably end up being over 60 pages and have multiple sections and lots of figures/tables.

Most of my data is coming from an external program that prints data files with ~ 10,000 data points in each column. I am currently using PGFplots to downsample in order to prevent an "Out of Memory" error.

My worry is that, when I compile the entire report as one single document (instead of section by section) I will get an "Out of Memory" error because of all the figures and data being processed by PGFplots. In addition to downsampling, I also have some filters I am applying to the data via PGFplots.

I am not sure how to estimate the amount of memory LaTeX will need when the final document is compiled so I do not know how to estimate how much PGFplots can handle downsampling/filter - wise.

If any of this is unclear please let me know. Also, this is my first time posting so please feel free to correct me with any customs/best practice when it comes to posting.

Improve this question
3
  • 1
    have you tried a full document. do you get an out of memory error? If not you don't have anything to worry about. In general I would expect most memory to be reclaimed as each figure is set so would not expect memory usage to build up. That said, 10,000 points seems like an awful lot of points for printing at normal paper sizes, and downsampling the data before passing it to tex doesn't sound like a bad idea.
    – David Carlisle
    Apr 13, 2015 at 16:08
  • Welcome to TeX.SX! Consider, that a lot of points increases screen display time of the document (if that's an issue)
    – user31729
    Apr 13, 2015 at 16:08
  • Thanks for the replies! I did not realize how LaTeX memory worked, or that it recovered it after each process (e.g. creating an image from table data). I thought it might have been a like a bucket that simply gets used up, excuse my ignorance! Also, I have gone ahead and written a little script in Matlab that downsizes my data tables for me there, thank you guys.
    – Nukesub
    Apr 13, 2015 at 17:40

1 Answer 1

Reset to default
1

The memory usage of pgfplots is beyond the default configuration of a tex system, especially for huge data sets.

The pgfplots reference manual contains a section named "Memory and Speed considerations" which is dedicated to the associated problems. Reducing its section to one sentence results in "lualatex is the simplest approach to get around memory limits and, since pgfplots 1.12, to reduce time; image externalization avoids unnecessary recompilations".

Especially the performance has been addressed in pgfplots 1.12, so I suggest to give it a try: parts of the computations have been moved to Lua.

More reading: http://pgfplots.sourceforge.net/pgfplots.pdf

Improve this answer
2
  • Cool, I appreciate that and will read it as soon as I get some free time. Thankfully, I think I have a better idea of how tex memory is used now. As long as one single plot does not exceed the mem usage, then the document should compile fine. It may be obvious, but I am using pdfLatex. Also please correct me if I am wrong on the memory comment, that is just my impression of it before reading through the documents you linked.
    – Nukesub
    Apr 20, 2015 at 1:46
  • Your understanding regarding "as long as one single plot does not exceed..." should be correct. This does not address the problem of computation time, of course. You may want to read about the external lib (see pgfmanual.pdf) or standalone to reduce recompilation time.
    – Christian Feuersänger
    Apr 20, 2015 at 19:08

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .