# increasing hard coded memory limits of latex

I am drawing a 3d-surface plot with pgfplots, with input data from some simulation results.

As the shader-option faceted interp results in huge pdf-files, I wanted to switch to faceted and increase the number of data points... But at some point, I had problems with compiling with pdflatex due to the limited main memory size, so I switched to lualatex. Even with lualatex, I got some error like Tex capacitiy exceeded, save size = 100000. So I increased the save size, but there seems to be a hard coded limit in Tex at 250.000! (The compilation uses only around 570MiB of RAM at the maximum, so at this point I still have 30 GiB of free RAM...)

My code looks like this:

\documentclass[class=scrbook, 11pt,convert]{standalone}
\usepackage{pgfplots}
\usepackage{pgfplotstable}
\pgfplotsset{colormap={ownbluered}{[1cm] rgb255(0cm)=(0,0,255); rgb255(30cm)=(0,255,255);  rgb255(60cm)=(0,255,128); rgb255(80cm)=(0,255,0); rgb255(90cm)=(192,255,0);
rgb255(115cm)=(255,255,0); rgb255(140cm)=(255,128,0); rgb255(155cm)=(255,0,0); rgb255(160cm)=(230,0,0)}
}

\pgfplotsset{
nonlinear colormap trafo/.code 2 args={
\pgfkeysalso{
y coord inv trafo/.code={%
\pgfmathparse{##1 < 30 ? (##1-30)*(-#2)/30 : (##1-30)*#1/30}%
},
}%
},
nonlinear colormap around 30/.code 2 args={
\pgfkeysalso{
colorbar style={
nonlinear colormap trafo={#1}{#2},
},
}%
},
}

\begin{document}
\def\infile{linear_directivity_1deg.txt}
\def\filenamedB{\infile_dB.txt}
\def\directivity{6.3}% in dB!
\def\directivitymin{-26}% in dB!
\edef\threedB{45}%
\pgfmathparse{\directivitymin+30}
\edef\minimum{\pgfmathresult}%
\pgfmathparse{\directivitymin/2+30}
\edef\halfminimum{\pgfmathresult}%

\begin{tikzpicture}
\begin{axis}[height=10cm, colorbar, colormap name=ownbluered, z buffer=sort, view={135}{25}, mesh/ordering=y varies, axis equal image,  nonlinear colormap around 30={\directivity}{\directivitymin},
colorbar style={
ytick={\minimum,\halfminimum,30,\threedB,60},
%yticklabel={\pgfmathparse{int(\tick)}\pgfmathresult\,dBi},
%yticklabel={\SI{\pgfmathprintnumber[precision=1]{\tick}}{\dBi}},
yticklabel={\pgfmathprintnumber[precision=1]{\tick}\,dBi},
%y unit = dBi,
yticklabel style={align=right, text width = 4em,},
yshift=-1cm,
height=5cm,
},
axis lines=none, enlargelimits=false,]
\addplot3[mesh/rows=181, surf, point meta=\thisrow{dir},line width = 0.025pt] table [x = x,y = y,z = z,]   {\filenamedB};
\end{axis}
\end{tikzpicture}
\end{document}


The read .txt-file can be found here.

An example plot with not so many points (only 5 degree steps instead of 1 degree):

Is there any possibility to improve my code, so that less save size is needed? Or could I compile parts of the software and change the built-in limits for the limits in tex.mf?

• In any case I think it is always good to use a proper colormap other than jet tex.stackexchange.com/questions/228197/… . For memory increase I had a similar issue and I ended up fixing the view angle and removing the back side of the plot. – percusse Nov 7 '15 at 12:53
• I already use a self-defined colormap, and also want to do a nonlinear mapping of point meta values to the colorbar labels... So far, I did want to avoid any calculations in latex, so I calculated the x,y,z and point meta values before – Faekynn Nov 7 '15 at 12:59
• @percusse How can I remove the back side of the plot? – Faekynn Nov 7 '15 at 13:01

This answer addresses your sub-question "Is there any possibility to improve my code, so that less save size is needed?"

Fortunately, the answer is yes (provided you have at least pgfplots 1.12 and you continue to use lualatex).

I expected the code to make use of the new lua backend, a partial reimplementation of some expensive functions in LUA. If it would have done that, it should have avoided the TeX memory limitations. However, the .log file claims that

Package pgfplots info on input line 50: Using 'lua backend=false' for plot 0 (ty
pe 'mesh'): point meta choice does not support LUA.


which lead me to work around the current point meta choice. I remember that the lua backend is disabled if point meta contains a TeX macro (as these cannot easily be migrated to the LUA script base). As soon as I used point meta=explicit combined with \addplot .... table[..., meta=dir] instead of point meta=\thisrow{dir}, the memory problems were solved and I could compile your example with lualatex. Note that both formulations of the point meta are equivalent for your case, but your formulation is easier to read and to remember.

Here is the complete example with result:

\documentclass[class=scrbook, 11pt,convert]{standalone}
\usepackage{pgfplots}
\usepackage{pgfplotstable}
\pgfplotsset{colormap={ownbluered}{[1cm] rgb255(0cm)=(0,0,255); rgb255(30cm)=(0,255,255);  rgb255(60cm)=(0,255,128); rgb255(80cm)=(0,255,0); rgb255(90cm)=(192,255,0);
rgb255(115cm)=(255,255,0); rgb255(140cm)=(255,128,0); rgb255(155cm)=(255,0,0); rgb255(160cm)=(230,0,0)}
}

\pgfplotsset{
nonlinear colormap trafo/.code 2 args={
\pgfkeysalso{
y coord inv trafo/.code={%
\pgfmathparse{##1 < 30 ? (##1-30)*(-#2)/30 : (##1-30)*#1/30}%
},
}%
},
nonlinear colormap around 30/.code 2 args={
\pgfkeysalso{
colorbar style={
nonlinear colormap trafo={#1}{#2},
},
}%
},
}

\begin{document}
\def\infile{linear_directivity_1deg.txt}
\def\filenamedB{\infile_dB.txt}
\def\directivity{6.3}% in dB!
\def\directivitymin{-26}% in dB!
\edef\threedB{45}%
\pgfmathparse{\directivitymin+30}
\edef\minimum{\pgfmathresult}%
\pgfmathparse{\directivitymin/2+30}
\edef\halfminimum{\pgfmathresult}%

\begin{tikzpicture}
\begin{axis}[height=10cm, colorbar, colormap name=ownbluered, z buffer=sort, view={135}{25}, mesh/ordering=y varies, axis equal image,  nonlinear colormap around 30={\directivity}{\directivitymin},
colorbar style={
ytick={\minimum,\halfminimum,30,\threedB,60},
%yticklabel={\pgfmathparse{int(\tick)}\pgfmathresult\,dBi},
%yticklabel={\SI{\pgfmathprintnumber[precision=1]{\tick}}{\dBi}},
yticklabel={\pgfmathprintnumber[precision=1]{\tick}\,dBi},
%y unit = dBi,
yticklabel style={align=right, text width = 4em,},
yshift=-1cm,
height=5cm,
},
axis lines=none, enlargelimits=false,]
\addplot3[mesh/rows=181, surf, shader=interp, point meta=explicit,line width = 0.025pt] table [x = x,y = y,z = z,meta=dir]   {\filenamedB};
\end{axis}
\end{tikzpicture}
\end{document}


Note that the resulting .pdf file is 2.5 MB big due to the many patches in your example, and it takes a very long while to render.

If you want to reduce its size, you could make use of shader=interp; the resulting file size reduces to 1.1 MB and the rendering time is considerably faster. In this case, the figure looks as follows:

Some remarks of sorts "outlook": Perhaps the lua backend will become smart enough to understand your initial code. And perhaps it will also become smart enough to read input files without requiring the expensive TeX routines. Both is planned and would reduce the insanely long compilation time by orders of magnitudes...

EDIT

the case with shader=faceted resulted in the following memory usage on my system

Here is how much of LuaTeX's memory you used:
25089 strings out of 494748
100000,109195293 words of node,token memory allocated
3090 words of node memory still in use:
82 hlist, 6 vlist, 1 rule, 10 math, 244 glue, 5 kern, 5 penalty, 32 glyph, 72
glue_spec, 1 write, 5 local_par, 204 pdf_literal, 2 pdf_refxform, 10 pdf_colors
tack nodes
avail lists: 2:46,3:211,4:287,5:5,6:47,7:1,9:93,10:3
27874 multiletter control sequences out of 65536+600000
26 fonts using 1033763 bytes
79i,10n,103p,718b,2731s stack positions out of 5000i,500n,10000p,27176575b,100000s
<</home/ludewich/tl2014/texmf-dist/fonts/type1/public/amsfonts/cm/cmmi10.pfb><</


I believe the 2731 is the save size (limit 100000)

• I read about the lua backend in the pgfplots manual, but I just thought it would be enabled by default... Thanks a lot! – Faekynn Nov 7 '15 at 16:50
• I encountered some strange behaviour: If I use shader=interp, everything works fine. If I use faceted, it seems to be ok at first (memory usage goes up to 1 GiB like it is the case for interp), but at some point I get the error message with the save size again... The exact message from the log file ...PGFPlots: reading {linear_directivity_1deg.txt_dB.txt} ! TeX capacity exceeded, sorry [save size=250000]. \pgfmath@stack@push ...def \pgfmath@stack@top {#2} \expandafter \expandafter ...  – Faekynn Nov 7 '15 at 17:11
• regarding "would be enabled by default": yes, that is true. However, it is switched off whenever it encounteres something which has not been rewritten in lua. And that happened here – Christian Feuersänger Nov 7 '15 at 19:13
• regarding the difference between interp and faceted: I cannot explain why this happens on your installation. Perhaps you have some slightly different version? I fear I cannot look into this right now. I will edit my answer to show the remaining capacities for the case faceted on my system. Could you check if you have PGF 3.0.1? Some time ago, I fixed a related save-stack issue which is part of that release. – Christian Feuersänger Nov 7 '15 at 19:22
• I still have PGF 3.0.0. I'll see how I can update it... ;) – Faekynn Nov 8 '15 at 8:10