2

I already experienced in this question that the right choice of assignments for the coordinate values and meta values from a table could increase the compilation speed of pgfplots due to the lua backend a lot. So I just wanted to ask, if someone sees something which could be made faster, mainly in the colormap transformation?

For the compilation of this code, the lua backend is also disabled according to the log, with the message: Package pgfplots info on input line 97: Using 'lua backend=false' for axis: x coord trafo unsupported.. Can this be avoided by changing the point meta code or the code of the colorbar axis transformation, maybe e.g. by using some lua code?

My main problem is the compile time (even though I externalize the images). With an input data file with non-satisfying resolution (.txt file has about 300 kB), the compilation time is a bit over 3 minutes. With the double sampling rate, the input file has about four times the size, and the compile time increases sixteenfold, which is then almost 1h. (4²=16, which agres with the pgfplots manual statement of quadratic complexity for large datasets) Where does this complexity come from? Is it just the amount of data, or would it get better if the input file is split in several files?

Just the coordinate transformation code:

 \pgfplotsset{
  symlog colormap trafo/.code={
    \pgfkeysalso{%
      y coord trafo/.code={%
        \pgfmathparse{##1>#1 ? 1+3*ln(##1<#1?1: (##1/#1))/ln(10*sqrt(10)) : (##1<-#1 ? -1-3*ln(##1>-#1?1: (-##1/#1))/ln(10*sqrt(10))      : ##1/#1)}
      },
      y coord inv trafo/.code={%
        \pgfmathparse{##1>1 ? exp((##1-1)/3*ln(10*sqrt(10)))*#1 : (##1<-1 ? -exp((-##1-1)/3*ln(10*sqrt(10)))*#1  : ##1*#1)}
      },
    }%
  },
  symlog colormap/.code n args={3}{
    \pgfkeysalso{
      colorbar style={
        symlog colormap trafo={#1},
        point meta min={#2},
        point meta max={#3},
      },
      point meta={z>#1 ? 1+3*ln(z<#1?1: (z/#1))/ln(10*sqrt(10)) : (z<-#1 ? -1-3*ln(z>-#1?1: (-z/#1))/ln(10*sqrt(10)) : z/#1)},
    }%
  },
}

The complete example

\documentclass{article}
\usepackage{xfrac}

\def\infile{infile.txt}

\usepackage{pgfplots}
\usepackage{pgfplotstable}
\pgfplotsset{compat=newest}

\pgfplotsset{
  symlog colormap trafo/.code={
    \pgfkeysalso{%
      y coord trafo/.code={%
        \pgfmathparse{##1>#1 ? 1+3*ln(##1<#1?1: (##1/#1))/ln(10*sqrt(10)) : (##1<-#1 ? -1-3*ln(##1>-#1?1: (-##1/#1))/ln(10*sqrt(10))      : ##1/#1)}
      },
      y coord inv trafo/.code={%
        \pgfmathparse{##1>1 ? exp((##1-1)/3*ln(10*sqrt(10)))*#1 : (##1<-1 ? -exp((-##1-1)/3*ln(10*sqrt(10)))*#1  : ##1*#1)}
      },
    }%
  },
  symlog colormap/.code n args={3}{
    \pgfkeysalso{
      colorbar style={
        symlog colormap trafo={#1},
        point meta min={#2},
        point meta max={#3},
      },
      point meta={z>#1 ? 1+3*ln(z<#1?1: (z/#1))/ln(10*sqrt(10)) : (z<-#1 ? -1-3*ln(z>-#1?1: (-z/#1))/ln(10*sqrt(10)) : z/#1)},
    }%
  },
}


\begin{document}
\thispagestyle{empty} 

\begin{tikzpicture}
\begin{axis}[axis equal image,
             xmin=-8,
             z buffer = sort,
             xmax=8,
             ymin=0,
             ymax=10,
             grid=major, 
             axis on top,
             colorbar,
             symlog colormap={142.3}{-4500}{4500},
             colormap/viridis, 
             mesh/ordering=y varies,
             view={0}{90},
             colorbar style={
                             scaled ticks=false,
                             ytick={-4501,-1423,-450,-142.3,0,142.3,450,1423,4501},
                             yticklabel={
                                 \pgfmathparse{\tick/100}\pgfmathprintnumber[fixed relative, precision=2]{\pgfmathresult}
                                        },
                             yticklabel style={
                                               align=right,
                                               text width = 1.875em,
                                               },       
                             xlabel = {$E_z \textrm{ in kV/m}$},   
                             xlabel style = {xshift = 0.35cm, yshift = -0.4625 cm},    
                             },
             tick label style={/pgf/number format/fixed},
             enlargelimits=false,
             xlabel={$x \textrm{ in mm}$}, 
             ylabel={$y \textrm{ in mm}$}, 
             ]

      \pgfplotstableset{skip first n=2}
      \pgfplotstableread{\infile}\datatable
            \addplot3[
                     mesh/rows=65, 
                     surf,  
                     shader = interp,          
                    ] table [
                             x index = 0,
                             y  index= 1,
                             z index = 2;
                            ]   from \datatable {};
\end{axis}
\end{tikzpicture}
\end{document}

example infile (truncated)

x [mm] 0
-----------------------------------------------------------------------------------------------------------------------------------------  
-8 0 51.0614
-7.75 0 82.8847
-7.5 0 63.4588
-7.25 0 53.413
-7 0 45.8015
-6.75 0 38.6945
-6.5 0 31.5657
-6.25 0 24.3476
-6 0 17.1636
-5.75 0 10.2049
-5.5 0 3.6868
-5.25 0 -2.17792
-5 0 -7.18864
-4.75 0 -11.1697
-4.5 0 -13.9803
-4.25 0 -15.5134
-4 0 -15.7146
-3.75 0 -14.5676
-3.5 0 -12.1229
-3.25 0 -8.48218
-3 0 -3.81099
-2.75 0 1.55382
-2.5 0 7.13892
-2.25 0 12.0255
-2 0 14.4733
-1.75 0 10.3511
-1.5 0 -56.9732
-1.25 0 -57.9164
-1 0 -21.3907
-0.75 0 93.6603
-0.5 0 512.283
-0.25 0 1778.09
0 0 1881.37
0.25 0 1778.09
0.5 0 512.281
0.75 0 93.658
1 0 -21.3938
1.25 0 -57.92
1.5 0 -56.9765
1.75 0 10.3483
2 0 14.4687
2.25 0 12.0195
2.5 0 7.13207
2.75 0 1.54623
3 0 -3.81901
3.25 0 -8.49036
3.5 0 -12.131
3.75 0 -14.5755
4 0 -15.7219
4.25 0 -15.5201
4.5 0 -13.9861
4.75 0 -11.1745
5 0 -7.19197
5.25 0 -2.17986
5.5 0 3.68637
5.75 0 10.2064
6 0 17.1669
6.25 0 24.3528
6.5 0 31.5727
6.75 0 38.7035
7 0 45.8125
7.25 0 53.4264
7.5 0 63.4758
7.75 0 82.9077
8 0 51.0745
-8 0.25 48.8507
-7.75 0.25 71.4631
-7.5 0.25 51.8272
-7.25 0.25 42.4871
-7 0.25 36.1465
-6.75 0.25 30.5903
-6.5 0.25 25.1597
-6.25 0.25 19.6837
-6 0.25 14.2034
-5.75 0.25 8.83354
-5.5 0.25 3.71559
-5.25 0.25 -1.00413
-5 0.25 -5.18482
-4.75 0.25 -8.69938
-4.5 0.25 -11.4425
-4.25 0.25 -13.3293
-4 0.25 -14.3084
-3.75 0.25 -14.3513
-3.5 0.25 -13.4718
-3.25 0.25 -11.714
-3 0.25 -9.15892
-2.75 0.25 -6.01235
-2.5 0.25 -2.57951
-2.25 0.25 0.450236
-2 0.25 1.43912
-1.75 0.25 -4.83268
-1.5 0.25 -42.3624
-1.25 0.25 -38.0751
-1 0.25 16.1594
-0.75 0.25 178.537
-0.5 0.25 748.531
-0.25 0.25 2461.65
0 0.25 2601.04
0.25 0.25 2461.65
0.5 0.25 748.53
0.75 0.25 178.535
1 0.25 16.1563
1.25 0.25 -38.0789
1.5 0.25 -42.366
1.75 0.25 -4.83578
2 0.25 1.43538
2.25 0.25 0.445735
2.5 0.25 -2.58454
2.75 0.25 -6.01804
3 0.25 -9.16484
3.25 0.25 -11.7201
3.5 0.25 -13.478
3.75 0.25 -14.3574
4 0.25 -14.314
4.25 0.25 -13.3345
4.5 0.25 -11.4472
4.75 0.25 -8.70337
5 0.25 -5.1878
5.25 0.25 -1.0061
5.5 0.25 3.71481
5.75 0.25 8.83392
6 0.25 14.2051
6.25 0.25 19.6867
6.5 0.25 25.1642
6.75 0.25 30.5962
7 0.25 36.154
7.25 0.25 42.4966
7.5 0.25 51.8396
7.75 0.25 71.4815
8 0.25 48.8619
-8 0.5 48.7844
-7.75 0.5 67.6708
-7.5 0.5 47.0852
-7.25 0.5 37.682
-7 0.5 31.6811
-6.75 0.5 26.659
-6.5 0.5 21.8645
-6.25 0.5 17.0579
-6 0.5 12.2321
-5.75 0.5 7.45987
-5.5 0.5 2.84438
-5.25 0.5 -1.50177
-5 0.5 -5.46699
-4.75 0.5 -8.94756
-4.5 0.5 -11.8542
-4.25 0.5 -14.1114
-4 0.5 -15.6659
-3.75 0.5 -16.4791
-3.5 0.5 -16.5394
-3.25 0.5 -15.8496
-3 0.5 -14.4255
-2.75 0.5 -12.3673
-2.5 0.5 -9.79737
-2.25 0.5 -6.98801
-2 0.5 -4.6393
-1.75 0.5 -4.80819
-1.5 0.5 -11.6809
-1.25 0.5 8.14997
-1 0.5 78.3415
-0.75 0.5 274.749
-0.5 0.5 943.2
-0.25 0.5 2944.76
0 0.5 3107.5
0.25 0.5 2944.76
0.5 0.5 943.199
0.75 0.5 274.747
1 0.5 78.3391
1.25 0.5 8.14696
1.5 0.5 -11.6839
1.75 0.5 -4.8112
2 0.5 -4.64255
2.25 0.5 -6.99184
2.5 0.5 -9.80156
2.75 0.5 -12.3719
3 0.5 -14.4304
3.25 0.5 -15.8547
3.5 0.5 -16.5445
3.75 0.5 -16.4843
4 0.5 -15.671
4.25 0.5 -14.1161
4.5 0.5 -11.8587
4.75 0.5 -8.95129
5 0.5 -5.47007
5.25 0.5 -1.50406
5.5 0.5 2.84302
5.75 0.5 7.45941
6 0.5 12.2327
6.25 0.5 17.0596
6.5 0.5 21.8673
6.75 0.5 26.6631
7 0.5 31.6865
7.25 0.5 37.6891
7.5 0.5 47.095
7.75 0.5 67.6869
8 0.5 48.7944
-8 0.75 49.6667
-7.75 0.75 67.5706
-7.5 0.75 45.782
-7.25 0.75 35.9227
-7 0.75 29.7622
-6.75 0.75 24.7203
-6.5 0.75 19.9793
-6.25 0.75 15.2548
-6 0.75 10.5115
-5.75 0.75 5.80032
-5.5 0.75 1.2061
-5.25 0.75 -3.17377
-5 0.75 -7.24051
-4.75 0.75 -10.9001
-4.5 0.75 -14.0696
-4.25 0.75 -16.6759
-4 0.75 -18.6636
-3.75 0.75 -19.9861
-3.5 0.75 -20.6155
-3.25 0.75 -20.5269
-3 0.75 -19.6867
-2.75 0.75 -18.1148
-2.5 0.75 -15.773
-2.25 0.75 -12.5353
-2 0.75 -8.19302
-1.75 0.75 -1.85639
-1.5 0.75 10.4672
-1.25 0.75 42.7128
-1 0.75 122.477
-0.75 0.75 336.302
-0.5 0.75 1052.96
-0.25 0.75 3198.01
0 0.75 3372.36
0.25 0.75 3198.01
0.5 0.75 1052.96
0.75 0.75 336.301
1 0.75 122.475
1.25 0.75 42.7104
1.5 0.75 10.4648
1.75 0.75 -1.85915
2 0.75 -8.19592
2.25 0.75 -12.5388
2.5 0.75 -15.7768
2.75 0.75 -18.1191
3 0.75 -19.6913
3.25 0.75 -20.5316
3.5 0.75 -20.6203
3.75 0.75 -19.991
4 0.75 -18.6684
4.25 0.75 -16.6806
4.5 0.75 -14.0739
4.75 0.75 -10.904
5 0.75 -7.24377
5.25 0.75 -3.17647
5.5 0.75 1.20418
5.75 0.75 5.79908
6 0.75 10.5113
6.25 0.75 15.2555
6.5 0.75 19.9811
6.75 0.75 24.7232
7 0.75 29.7664
7.25 0.75 35.9284
7.5 0.75 45.7904
7.75 0.75 67.5843
8 0.75 49.6757
-8 1 50.6639
-7.75 1 68.5314
-7.5 1 45.5971
-7.25 1 35.1523
-7 1 28.632
-6.75 1 23.3348
-6.5 1 18.395
-6.25 1 13.498
-6 1 8.59186
-5.75 1 3.71581
-5.5 1 -1.05397
-5.25 1 -5.6273
-5 1 -9.91064
-4.75 1 -13.8143
-4.5 1 -17.2579
-4.25 1 -20.1693
-4 1 -22.492
-3.75 1 -24.1753
-3.5 1 -25.1828
-3.25 1 -25.4734
-3 1 -24.9818
-2.75 1 -23.6741
-2.5 1 -21.4015
-2.25 1 -17.7821
-2 1 -12.1512
-1.75 1 -2.75656
-1.5 1 14.6518
-1.25 1 50.5191
-1 1 131.565
-0.75 1 345.117
-0.5 1 1059.9
-0.25 1 3203.64
0 1 3377.83
0.25 1 3203.65
0.5 1 1059.9
0.75 1 345.115
1 1 131.564
1.25 1 50.5172
1.5 1 14.6496
1.75 1 -2.75902
2 1 -12.1539
2.25 1 -17.7854
2.5 1 -21.405
2.75 1 -23.6781
3 1 -24.9861
3.25 1 -25.478
3.5 1 -25.1876
3.75 1 -24.1801
4 1 -22.4967
4.25 1 -20.174
4.5 1 -17.2624
4.75 1 -13.8185
5 1 -9.91432
5.25 1 -5.63034
5.5 1 -1.05653
5.75 1 3.71403
6 1 8.59102
6.25 1 13.4979
6.5 1 18.3959
6.75 1 23.3367
7 1 28.6351
7.25 1 35.1571
7.5 1 45.6044
7.75 1 68.5438
8 1 50.6719
-8 1.25 51.3174
-7.75 1.25 69.3629
-7.5 1.25 45.4568
-7.25 1.25 34.4408
-7 1.25 27.512
-6.75 1.25 21.8823
-6.5 1.25 16.6537
-6.25 1.25 11.4921
-6 1.25 6.33623
-5.75 1.25 1.21921
-5.5 1.25 -3.78699
-5.25 1.25 -8.59521
-5 1.25 -13.1147
-4.75 1.25 -17.2581
-4.5 1.25 -20.9464
-4.25 1.25 -24.1099
-4 1.25 -26.6934
-3.75 1.25 -28.6475
-3.5 1.25 -29.9355
-3.25 1.25 -30.5137
-3 1.25 -30.3075
-2.75 1.25 -29.2648
-2.5 1.25 -27.1978
-2.25 1.25 -23.647
-2 1.25 -17.8619
-1.75 1.25 -8.08802
-1.5 1.25 9.40737
-1.25 1.25 43.5587
-1 1.25 118.623
-0.75 1.25 315.05
-0.5 1.25 972.254
-0.25 1.25 2944.29
0 1.25 3104.61
0.25 1.25 2944.29
0.5 1.25 972.252
0.75 1.25 315.049
1 1.25 118.622
1.25 1.25 43.5572
1.5 1.25 9.4056
1.75 1.25 -8.09007
2 1.25 -17.8643
2.25 1.25 -23.6499
2.5 1.25 -27.2009
2.75 1.25 -29.2685
3 1.25 -30.3117
3.25 1.25 -30.518
3.5 1.25 -29.9403
3.75 1.25 -28.6524
4 1.25 -26.6983
4.25 1.25 -24.1148
4.5 1.25 -20.9512
4.75 1.25 -17.2625
5 1.25 -13.1189
5.25 1.25 -8.59896
5.5 1.25 -3.79004
5.75 1.25 1.21679
6 1.25 6.33456
6.25 1.25 11.4913
6.5 1.25 16.6538
6.75 1.25 21.8834
7 1.25 27.5144
7.25 1.25 34.4446
7.5 1.25 45.463
7.75 1.25 69.3739
8 1.25 51.3245
-8 1.5 51.2722
-7.75 1.5 69.3806
-7.5 1.5 44.8177
-7.25 1.5 33.3384
-7 1.5 26.0324
-6.75 1.5 20.0701
-6.5 1.5 14.5394
-6.25 1.5 9.0976
-6 1.5 3.68028
-5.75 1.5 -1.68137
-5.5 1.5 -6.91641
-5.25 1.5 -11.9388
-5 1.5 -16.6587
-4.75 1.5 -20.9906
-4.5 1.5 -24.8586
-4.25 1.5 -28.1963
-4 1.5 -30.9547
-3.75 1.5 -33.0928
-3.5 1.5 -34.5847
-3.25 1.5 -35.4024
-3 1.5 -35.4942
-2.75 1.5 -34.8323
-2.5 1.5 -33.2706
-2.25 1.5 -30.4296
-2 1.5 -25.6859
-1.75 1.5 -17.5932
-1.5 1.5 -3.12652
-1.25 1.5 24.96
-1 1.5 86.4887
-0.75 1.5 247.128
-0.5 1.5 783.513
-0.25 1.5 2391.2
0 1.5 2522.36
0.25 1.5 2391.21
0.5 1.5 783.511
0.75 1.5 247.128
1 1.5 86.4884
1.25 1.5 24.959
1.5 1.5 -3.12774
1.75 1.5 -17.5947
2 1.5 -25.6879
2.25 1.5 -30.4323
2.5 1.5 -33.2736
2.75 1.5 -34.8358
3 1.5 -35.4981
3.25 1.5 -35.4066
3.5 1.5 -34.5893
3.75 1.5 -33.0976
4 1.5 -30.9597
4.25 1.5 -28.2014
4.5 1.5 -24.8636
4.75 1.5 -20.9955
5 1.5 -16.6633
5.25 1.5 -11.943
5.5 1.5 -6.92009
5.75 1.5 -1.6845
6 1.5 3.67791
6.25 1.5 9.09602
6.5 1.5 14.5387
6.75 1.5 20.0704
7 1.5 26.0338
7.25 1.5 33.3413
7.5 1.5 44.8228
7.75 1.5 69.39
8 1.5 51.2781
2

If you do not have to compile this part every time, you could create a pdf file and include this via includegraphics ....just an idea...

  • I already do this (more or less automatically). However, for big files it the compilation time seems to be strongly nonlinear dependent on file size... – Faekynn May 19 '16 at 6:19
  • I've added a bit of explanation to the question. The problem is that the compile time is about 1 h, which is quite lon in my opinion... – Faekynn May 19 '16 at 13:13
  • I cannot believe that, 1 hour? really? If I want to compile the code above, I will also have to wait for 1 hour? – Peter Ebelsberger May 21 '16 at 17:21
  • No, of course not. Should be done in less than a minute. The file for 1 hour compilation time has the same structure, but a lot more entries. The large file has about 1.3 MB and contains about 71k samples. – Faekynn May 21 '16 at 17:34
0

I found one way to reduce the time significantly. As it is also mentioned in the pfgplots manual, it is important how the data is loaded. Using the file name to load the table

\addplot3[
          mesh/rows=65, 
          surf,  
          shader = interp,          
         ] table [
                  x index = 0,
                  y  index= 1,
                  z index = 2;
                 ]   {\infile}; % \infile just contains the filename, not a table

instead of the macro-based preloaded table

\pgfplotstableread{\infile}\datatable
\addplot3[
          mesh/rows=65, 
          surf,  
          shader = interp,          
         ] table [
                  x index = 0,
                  y  index= 1,
                  z index = 2;
                 ]   from \datatable {};

reduced compilation time for a 500kB input file with about 26k lines significantly from 363 seconds (6 minutes) to 42 seconds. Of course, due to file sizes, it is better to convert the resulting pdf to png afterwards.

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.