I'll explain my situation for reference before I dive into the question.
I am writing a fairly lengthy report in LaTeX. The report will probably end up being over 60 pages and have multiple sections and lots of figures/tables.
Most of my data is coming from an external program that prints data files with ~ 10,000 data points in each column. I am currently using PGFplots to downsample in order to prevent an "Out of Memory" error.
My worry is that, when I compile the entire report as one single document (instead of section by section) I will get an "Out of Memory" error because of all the figures and data being processed by PGFplots. In addition to downsampling, I also have some filters I am applying to the data via PGFplots.
I am not sure how to estimate the amount of memory LaTeX will need when the final document is compiled so I do not know how to estimate how much PGFplots can handle downsampling/filter - wise.
If any of this is unclear please let me know. Also, this is my first time posting so please feel free to correct me with any customs/best practice when it comes to posting.