5

additional info at bottom

I was going to use \colorboxes in verbatim mode, one per letter, but this increased significantly the size of pdf (produced via latex+dvipdfmx but pdflatex is similar, and I did further testing with it).

Drawing a yellow rule appears simpler than drawing a letter glyph, but from naïve testing, the PDF is more economical when it has to do many letters than when it has to do many small rules. Is there a way to tell pdflatex that something will be used over and over again, to store it somewhere, and economically use a pointer to it when it is needed? A bit I guess like what is automatically done for the glyphs picked up from the partially embedded font.

In the sort of MWE below, I see that the pdf file size is not outrageous when I have thousands of copies of the same type of colored box, about 4 bytes to switch to yellow, draw a rectangle, switch back to normal color, is not bad. But it is still quite more than what it takes to code the drawing of a letter X for example. Is there a way to obtain more compact PDF ?

(I tried ps2pdf which is ok here, but fails in my real life case to reduce the file size as obtained from dvipdfmx; in fact the file size goes from 1.1M to 1.5M).

\documentclass{article}
\usepackage{color}
\begin{document}

Hello.

% 28367 bytes for 4000 boxes
% 19844 bytes for 2000 boxes
% 15558 bytes for 1000 boxes
% 11658 bytes for 100 boxes
% 11167 bytes for 1 box
% about 4.4 bytes per colored box.
\newsavebox{\mybox}
\sbox{\mybox}{\colorbox{yellow}{\phantom{X}}}

\count255 4000
\noindent\loop
\usebox{\mybox}
%\colorbox{yellow}{\phantom{X}}
\advance\count255 by -1
\ifnum\count255>0
\repeat

% 11695 bytes for 1000 X
% 12272 bytes for 2000 X
% 12895 bytes for 3000 X
% 13057 bytes for 4000 X
% 13603 bytes for 5000 X
% \sbox{\mybox}{X}
% \count255 5000
% \noindent\loop
% \usebox{\mybox}
% \advance\count255 by -1
% \ifnum\count255>0
% \repeat

\end{document}

I report here on my testing with pdfxform which I was doing just a the time wipet's answer appeared. I used the following test file. This is now on an old TL2010. With otherwise default settings the PDF is a PDF 1.5 file. I got 26907 bytes with the word Hello and 4000 yellow boxes done with \usebox and 23972 bytes using the \pdfxform way.

Then in my real use case, again on TL2010 (and dvipdfmx has evolved since) I get the following order of magnitudes:

  1. for a source not doing any boxes but only letters (in monotype), the dvipdfmx produced file has about 580000 bytes and the pdflatex one has about 760000 bytes (font used in the body of the document is mainly newtxtt which I have added to my TL2010).

  2. when I do all the colored boxes, the dvipdfmx now weighs about 930000 bytes (but at home it was about 1050000 bytes) and the pdflatex one 1122000 bytes.

  3. If I now do the \pdfxform thing I get a file of 1090000 bytes.

Conclusion: there is indeniably some gain, but pdflatex+\pdfxform still produces a bigger file in my case than dvipdfmx. And when I don't use at all the colored boxes, my dvipdfmx file is about 40% smaller...

The increase of file size via adding the boxes is about 350k (for both compilation routes) and using \pdfxform saves me about 30k, which is nice but still frustrating.

\documentclass{article}
\usepackage{color}

\begin{document}

Hello.

\fboxsep0pt

\newsavebox{\mybox}
\sbox{\mybox}{\colorbox{yellow}{\strut\phantom{X}}}

% USE OF A BOX
% gives pdf file of 26907 bytes
% This is a 1.5 PDF file pdfTeX-1.40.11  (TL2010)
\count255 4000
\noindent\loop
\penalty0
\usebox{\mybox}%
\advance\count255 by -1
\ifnum\count255>0
\repeat

% % USE OF \pdfxform
% % gives pdf file of 23972 bytes
% \pdfxform\mybox % attention, empties box register

% \count255 4000
% \noindent\loop
% \penalty0
% \hbox{\pdfrefxform \pdflastxform}%
% \advance\count255 by -1
% \ifnum\count255>0
% \repeat

\end{document}

One last bit of info: in my real use case, the document has currently about 150 pages, and there are between 5 and 10 uses of words per page with their letters having a yellow background, actually I can count them: 32297 boxes! Thus the penalty if of about 10 bytes for each such yellow rectangle, and using \pdfxform doesn't improve it that much.

I have redone my tests on my real use case of a 152 pages document with 32297 boxes. To make it cleaner, I have removed the verbatim letters and done a comparison between having 32297 identical strutboxes the width of a letter, colored or not color (this is xcolor and color yellow). Adding the colored boxes costs:

  1. 195303 bytes when using dvipdfmx,
  2. 232110 bytes when using pdflatex,
  3. 194208 bytes when using pdflatex+\pdfxform.

It thus appears that dvipdfmx and pdflatex+\pdfxform are of the same efficiency and that each colored box costs about 6 bytes. I don't know why when this is done in file doing nothing else, the cost is more of 3.2 byte per colored box.

I have also observed something weird with \pdfxform. With this test file:

\documentclass{article}
%\usepackage{xcolor}    
\begin{document}

Hello.

%\fboxsep0pt

\newsavebox{\mybox}
\sbox{\mybox}
  %{\colorbox{yellow}{\strut\phantom{X}}}
  {\strut\phantom{X}}% NO COLOR!

% USE OF \pdfxform

\pdfxform\mybox 

\count255 32297
\noindent\loop
\penalty0
\hbox{\pdfrefxform \pdflastxform}%
\advance\count255 by -1
\ifnum\count255>0
\repeat

\end{document}

I obtain circa 114000 bytes and about the same if the \colorbox is used. However if I don't use the \pdfxform mechanism, then with colored boxes, the pdf weighs about 140000 bytes but when produced without color but only empty boxes it is then only 16572 bytes.

In my big file, using \pdfxform with colored boxes gives 951744 bytes, and using it with non-colored boxes 951711 bytes. Not using pdfxform however, then there is a big difference between colored (953646 bytes) and non-colored cases (721543 bytes).

8
  • Have you tried compressing of the resulting PDF? tex.stackexchange.com/questions/18987/…
    – Daniel
    Commented Oct 10, 2014 at 10:44
  • Have a look at \pdfxform etc. (section 7.8 of the pdfTeX manual). Commented Oct 10, 2014 at 10:54
  • I am not sure how rectangles ('boxes') are drawn in a PDF, but I am pretty sure that the corner coordinates are stored -- if there a lot of rectangles, I assume, that a lot of coordinates must be used.
    – user31729
    Commented Oct 10, 2014 at 10:59
  • @Daniel I have tried ps2pdf which is about the same as the direct gs invocation in the answer you linked to. I tried the invocation proposed there, and it gives the same result (file size increases from 1082657 bytes to over 1500000 bytes).
    – user4686
    Commented Oct 10, 2014 at 11:02
  • @MartinSchröder I will look at \pdfxform. I guess however this will compromise my preferred dvipdfmx route then ;-)
    – user4686
    Commented Oct 10, 2014 at 11:04

1 Answer 1

8

Here is the solution using \pdfxform:

\setbox0=\hbox{\colorbox{yellow}{\phantom{X}}}
\pdfxform0  \mathchardef\myboxnum=\pdflastxform
\def\mybox{\pdfrefxform\myboxnum}

\count255 4000
\noindent\loop
\mybox\
\advance\count255 by -1
\ifnum\count255>0
\repeat

The uncompressed PDF (\pdfcompresslevel=0) from your MWE has 673757 bytes and the same using \pdfxform has 170255 bytes, compress ratio is 3.95. The one yellow box created by your MWE looks in PDF like:

0 0 1 0 k 0 0 1 0 K
0 0 1 0 k 0 0 1 0 K
q
1 0 0 1 150.281 642.291 cm
0 0 13.45 12.785 re f
Q
0 0 1 0 k 0 0 1 0 K
0 g 0 G
0 0 1 0 k 0 0 1 0 K
0 g 0 G
0 g 0 G
0 g 0 G

It is evident that there is very uneffective color manipualtion (the macro package color.sty is responsible for this).

On the other hand, when \pdfxform is used, then the same yellow box is represented by:

q
1 0 0 1 150.281 642.291 cm
/Fm1 Do
Q

The pointer is used here. But the shift of the current point realized by concatenate matrix (cm) have to be applied for each box. On the other hand the printing of the character from font is more optimized in PDF specification: the current point between characters is shifted "automatically".

Edit: I did third experiment for comparison. To use colors manually without macro support in order to mimimalize the garbage in PDF:

\count255 4000
\noindent\loop
\hbox{\pdfliteral{0 0 1 0 k}\rlap{\vrule height2ex width.7em}\pdfliteral{0 g}\kern.7em}
\advance\count255 by -1
\ifnum\count255>0
\repeat

Now, the uncompressed PDF has 250322 bytes and one yellow box looks like:

1 0 0 1 10.205 0 cm
0 0 1 0 k
q
0 0 6.974 8.579 re f
Q
0 g

The compress ratio from MWE to this third solution is 2.7. Without using \pdfxform. Only uneffective macro package is omitted and colors are set by hand.

1
  • I have tried the \pdfliteral. However for my big document it gives a result not worse than using \colorbox only if I use the \pdfxform mechanism. There is then no gain compared to \colorbox+\pdfxform use. My files have compression: they are PDF 1.5 files, as produced by default by pdflatex on my installation.
    – user4686
    Commented Oct 10, 2014 at 17:42

You must log in to answer this question.