# How to simplify vector graphics containing a huge number of data points

In the genomics field people often produce something called Manhattan plot which consists of points standing for millions of p-values:

It's very slow for display in vector format, some viewers simply cannot deal with it, but a raster image is much easier to open in this case, but of course we love vector graphics infinitely better ;) .

At the bottom of the plot we can see the density of points are so high that we can join them into a filled area, which should reduce the calculation burden immensely.

How can this be done in TeX? As suggested by one of the answers, it should be possible to get a union of nearby points in a vector graphic editor, but it will take a lot of work. Ideally, one would like to see each point as a path, and during the ploting process, the union operation to be executed point by point, thus in the end the whole plot will be simplified to the largest degree.

• can you provide such a set of data? – user2478 Oct 21 '13 at 20:59
• what is the advantage of a vector format here (for the actual plot, not the labels) it seems that the pixels essentially imply a fixed resolution anyway. What is lost if you just use a bitmap image and overlay labels with scalable fonts? – David Carlisle Oct 21 '13 at 21:10
• I would propose preprocessing the data set with a script that removes data points that are completely overdrawn. A complete LaTeX solution seems to be unnecessarily complicated, especially since that would require doing the simplification work every time that you compile the document. The script shouldn't be difficult to write. – DCTLib Oct 21 '13 at 21:18
• @Herbert It's easy to simulated with R. For example: p = -log10(runif(1e6, 0, 1)); pdf('x.pdf'); plot(p, pch=20, cex=.5); dev.off() You will get a pdf file in the current directory. Okular will not open it, Ghostscript viewer gs can open it, but very slowly. – qed Oct 22 '13 at 12:00
• @Ruedi It's not quite easy to preprocess the data either. First you need the size of the point in the plot, then you need to calculate a huge lot of euclidean distances. – qed Oct 22 '13 at 12:03

I have no idea what exactly your R expression really does. However, here is a similiar solution with 50,000 points. Run it with xelatex or with latex -> dvips -> ps2pdf:

\documentclass{article}
\usepackage{pst-plot}

\begin{document}

\begin{pspicture}(10,14)
\psset{xunit=0.5}
\psaxes[Dx=2](23,10)
\psplot[plotpoints=50000,plotstyle=dots,dotscale=0.5]{0.01}{23}
[3 srand]{ x cvi 2 mod 0 eq {0 setgray}{0.5 setgray}ifelse
Rand log neg 2 mul 0.6 sub }
\psline[linecolor=blue](0,5)(23,5)
\psline[linecolor=red](0,8)(23,8)
\end{pspicture}

\end{document}


• This is indeed easier to display even when the number of points are scaled up to 1 million, but still quite slow. – qed Oct 22 '13 at 13:49

My suggestion would be to open your output file in a vector editing program, select the plotted points for each area, and take the intersection of them. Doing so would leave you with a single vector for the perimeter of each plot area.

This assumes the dots in the file are rendered as small overlapping circles rather than as points with some corresponding diameter.

I agree with both David and Ruedi that the image just needs to be simplified once instead of being processed repeatedly upon compilation of the document.

• I would say to do the union instead of intersections. – Sigur Oct 21 '13 at 22:55
• It's going to be hard time for the vector editing program. – qed Oct 22 '13 at 12:01