Multiple PDFs with page group included in a single page warning

Question

I updated TeX Live to the Ubuntu Quantal version (2012.20120611-4) and I suddenly got this warning:

PDF inclusion: multiple pdfs with page group included in a single page

This is a minimal example for which I get the warning:

\documentclass{book}
\usepackage{graphicx}
\begin{document}
  \includegraphics{image1}

  \includegraphics{image2}
\end{document}

Both images have been produced by the export PDF feature of Inkscape and contain simple line drawings (no fancy stuff).

I have been looking on the Internet, but only found others with this problem and did not found any solutions:

• In the Latex user group they did not seem to understand/recognise the problem. And told the OP to go to the MikTeX groups, but it is not a MiKTeX specific problem as it is also happening with TeX Live and other distributions.

• At gmane.comp.tex.pdftex they were looking into the use (and versions) of MS Office products. Also not the cause as I am not using MS Office to produce PDFs.

During my search if found the PDFTeX code (pdftoepdf.cc) that spawns this warning, maybe it is of some help in understanding what is happening?

if (pdfpagegroupval == 0) { 
    // another pdf with page group was included earlier on the same page;
    // copy the Group entry as is
    pdftex_warn("PDF inclusion: multiple pdfs with page group included in a single page");
    pdf_newline();
    pdf_puts("/Group ");
    copyObject(&dictObj);
} else {
    // write Group dict as a separate object, since the Page dict also refers to it
    pageDict->lookup((char *) "Group", &dictObj);
    if (!dictObj->isDict())
        pdftex_fail("PDF inclusion: /Group dict missing");
    writeSepGroup = true;
    initDictFromDict(groupDict, page->getGroup());
    pdf_printf("/Group %d 0 R\n", pdfpagegroupval);
}

Does anyone have an idea what is happening, whether it is serious and how I could get rid of these warnings?

Answer 1

The problem is also reported in a german forum mrunix.de. It might be a bug in the tex distribution (pdftex). The problem happens only when you include multiple pdf pages, created in a specific manner (e.g. by MS Office products), in a single page.

Solution: Convert pdf files into ps and then back to pdf using Ghostscript, then the warning will go away (pdf2ps -> ps2pdf). This conversion probably removes the "page group" information from pdf files. (Caveat: This renders your pdf and some text might not be selectable or searchable any more.)

Editing the colorspace of pdf files with ghostscript also resolves the issue (if there is no multiple pages in the pdf file):

gs -o fixed-image.pdf -sDEVICE=pdfwrite -dColorConversionStrategy=/sRGB 
   -dProcessColorModel=/DeviceRGB original-image.pdf

CMYK conversion if RGB does not work for you:

gs -o fixed-image.pdf -sDEVICE=pdfwrite -dColorConversionStrategy=/CMYK 
   -dProcessColorModel=/DeviceCMYK original-image.pdf

P.S. Some programs generate "page group"s in pdf files; for example when you impose different images/objects in illustrator or inkscape. It seesm that pdftex is unable to handle multiple page groups in a single output page. The reason might be that each page groups specifies a different color space or transparency space.

Answer 2

PDF has a feature called "Page Groups" (PDF Reference, section 11.4.7). These descibe transparency effects between top-level objects on one page. When pdfTeX (or LuaTeX or XeTeX) includes a page from a PDF, it converts all pages into "Form XObjects" (section 8.10.1). pdfTeX also converts the Page Groups into /Group entries of the XObjects.

The problem now is that Adobe products need also a /Group entry (whose content should not matter) in the /Page object which contains these XObjects to correctly render transparency (this is just needed to select the right rendering engine; the transparency information for the included pages should be taken from these included pages).

pdfTeX will either use the first /Group it encounters when including PDFs or synthesize one when including PNGs with transparency. The warning is triggered when multiple Page Groups are encountered on one page (since the engine will then use the first one encountered and this may not be the "correct" one) and can probably be ignored. Of course this should be described somewhere in the pdfTeX documentation...

---

Update 2016-03-30:

Since version 1.40.15 (TeXlive 2014) pdfTeX has a parameter \pdfsuppresswarningpagegroup

Ordinarily, pdfTeX gives a warning when more than one included pdf file has a so-called “page group object” (/Group), because only one can “win” — that is, be propagated to the page level. Usually the page groups are identical, but when they are not, the result is unpredictable. It would be ideal if pdfTeX in fact detected whether the page groups were the same and only gave the warning in the problematic case; unfortunately, this is not easy (a patch would be welcome). Nevertheless, often one observes that there is no actual problem. Then seeing the warnings on every run is just noise, and can be suppressed by setting this parameter to a positive number.

So by adding \pdfsuppresswarningpagegroup=1 to the top of your file you can suppress this warning.

Answer 3

Martin Schröder has done a wonderful job of explaining the underlying cause, so I won’t repeat that here. Other than telling pdfLaTeX to shut up, the solution would be to remove/strip the page groups from the PDF inputs. However, all the proposed solutions seem to suffer from one of these problems:

• Lossy: Ghostscript-related solutions apparently rasterize the PDF file, which defeats the whole point of using PDF figures! I’m very picky when it comes to image quality, so this direction is no-go.
• Fragile: Naively doing a find-and-replace (i.e. sed) to fix a PDF file is probably not a good idea. This could corrupt a PDF file.

Turns out there’s this neat open-source tool called QPDF, which can “unpack” PDF files into a regular, quasi-textual format, affectionately named “QDF”. After running this tool, it was really easy to identify the page group inside the QDF file using a plain text editor. A fragment is shown below:

%% Page 1
%% Original object ID: 5 0
4 0 obj
<<
  /Contents 5 0 R
  /Group <<
    /CS /DeviceRGB
    /I true
    /S /Transparency
    /Type /Group
  >>
  /MediaBox [
    0
    0
    460.799988
    345.600006
  ]
  /Parent 3 0 R
  /Resources 7 0 R
  /Type /Page
>>

Mine was created in Inkscape. Yours may be a bit different. Notice the /Group << … >> dictionary. This is what needs to be removed. This can be automated using a Python script:

import sys

stdin = getattr(sys.stdin, "buffer", sys.stdin)
stdout = getattr(sys.stdout, "buffer", sys.stdout)
stderr = getattr(sys.stderr, "buffer", sys.stderr)

page_group = None
for line in stdin:
    if page_group is None:
        if line.rstrip() == b"  /Group <<":
            page_group = [line]
        else:
            stdout.write(line)
    else:
        page_group.append(line)
        if line.rstrip() == b"  >>":
            break
else:
    if page_group:
        stdout.write(b"".join(page_group))
        page_group = None
for line in stdin:
    stdout.write(line)
stdout.flush()

if page_group:
    stderr.write(b"".join(page_group))
else:
    stderr.write(b"note: did not find page group\n")

Save that script to, say, strip_page_group.py and then chain all the commands together:

qpdf --qdf input.pdf - | python strip_page_group.py | fix-qdf >output.pdf

Note 1: Make sure the output filename (output.pdf) is different from the input filename (input.pdf) or you’ll lose the PDF file entirely!

Note 2: If you need deterministic output, supply qpdf with the --deterministic-id option.

Answer 4

Following the pdf2ps -> ps2pdf advice from a previous answer, this is how I solved the issue. I browsed to the folder where my pdf images are located, used find to get all filenames, performed both transformations and deleted the temporary ps file.

In short (specific to Bash because of ${f%.*}):

for f in $(find . -type f  -name "*.pdf"); do \
  echo $f; \
  pdf2ps $f ${f%.*}.ps; \
  ps2pdf ${f%.*}.ps $f; \
  rm -f ${f%.*}.ps; \
done

Answer 5

Additional Info / Workaround for MS Office Users: I have been using pdfLaTeX with PDFs generated from Visio for years. I just reinstalled my PC and then I got the warning - but only for NEWLY saved PDFs, not for the old ones. Therefore I looked for PDF options in Visio: If you tell Visio to generate PDF/A compatible PDFs, the warning will disappear.

Answer 6

Another additional info for Adobe Illustrator users: select "Acrobat 4 (PDF 1.3)" Compatibility when saving the PDF. Then the warning will disappear.

Answer 7

Unfortunately the issue is not always as harmless as stated in previous comments. I have a file (about 50 pages) with about 70 little sketches (typically 15mm x 8mm, all of them images.pdf), up to 8 sketches on one page. When the warning "multiple pdfs with page group included in a single page" appeared, 5 pages got completely black in the output pdf file, with the Acrobat Reader returning an error message. The remaining 45 visible pages were displayed correctly, including the sketches. With shifting some sketches to other places, other pages got black, with no clear relation between certain images and blackened pages discernible. After I opened all images with GSView, applied File>Converte>pdfwrite with Properties>Compatibility/Level 1.2, both the warnings and the black pages disappeared, and everything is fine now.

Remark: The problem turned up when I started with a new TexLive installation under Windows 8.1 on a new computer. There was no problem with the self-same images.pdf with an older TexLive installation under Windows XP on my old computer. When I viewed the dammaged pdf files with Acrobat Reader on the old computer, they were dammaged as well. When I viewed the pdf output file from the old computer with Arobat Reader on the new computer, everything was fine. Thus obviously the new Techlive is handling images.pdf different than the old TexLive did.

Answer 8

I had a similar problem when including PDF figures generated with inkscape (original EPS plot from gnuplot with a PNG overlay with transparency) into a document compiled with pdflatex (from TeX Live 2013 distro). The bad thing was that the resulting PDF document could not be read by some viewers (including acroread 9.5.5).

My solution was to use qpdf to "linearize" the output of pdflatex,

qpdf --linearize corrupted.pdf working.pdf

This does not prevent pdflatex from issuing the warnings but at least you get a working document.

Answer 9

Instead of using pdf2ps or ghostscript (which both introduced some errors in my pdfs, e.g. missing lines), I just removed the Group information from the PDF with

sed -i 's/\/Group.*R//g' file.pdf

I don't know if this could introduce any problems, but in my case, with only single-paged libreoffice-exported images, it works well.

Answer 10

Here is what I found: Somewhere at the beginning the PDFs contain a group which defines the contents of the page. For a ghostscript generated PDF it looks like this:

5 0 obj
<</Type/Page/MediaBox [0 0 172.8 201.6]
/Rotate 0/Parent 3 0 R
/Group 4 0 R
/Resources<</ProcSet[/PDF /Text]
/ExtGState 104 0 R
/XObject 105 0 R
/Font 106 0 R
>>
/Contents 6 0 R
>>

The line /Group 4 0 R refers to a group object which looks like this:

4 0 obj
<</Type/Group
/S/Transparency
/CS/DeviceRGB>>endobj

This object defines the page transparency and color space. To get rid of the pdftex warning it is enough to remove the reference to this group i. e. remove the line /Group 4 0 R with an editor that won’t hurt the binary data like vi or use sed:

sed -i".bak" -e "/^\/Group 4 0 R$/d" "filename.pdf"

For PDFs generated by matplotlibs backend_pdf the group won’t be in a separated group. The main object looks like this:

10 0 obj
<< /Group << /CS /DeviceRGB /S /Transparency /Type /Group >> /Parent 2 0 R
/MediaBox [ 0 0 172.8 201.6 ] /Resources 8 0 R /Type /Page /Contents 9 0 R
>>
endobj

Here /Group << /CS /DeviceRGB /S /Transparency /Type /Group >> has to be removed.

P. S.: The ghostscript method did unfortunately not work for me, the error still appears. Only when I forced PDF version 1.3 in ghostscript it would go away. But that rasterizes the image.

Answer 11

I found another solution for Mac users, as the GhostScript color space conversion did not work for me:

Open the file in Mac Preview and save it again (duplicate).

This solved the issue for me with no obvious downsides.

Answer 12

I’ve also created a qpdf-based workaround but attempted to make it somewhat more reliable. It’s a shell script pdfstrippagegroup.sh that drives a perl loop to strip the PDF page group from the temporary QDF file, keeping a bit more state than Rufflewind’s answer to avoid accidentally deleting the wrong groups (the sed-based answers have this even worse).

The script (POSIX sh)

#!/bin/sh

unset LANGUAGE
LC_ALL=C; export LC_ALL

die() {
    echo >&2 "E: ${0##*/}: $dst: $*"
    exit 1
}

for dst in "$@"; do
    tpf=$dst\~.tmp
    tq1=$tpf.1.qdf
    tq2=$tpf.2.qdf
    tq3=$tpf.3.qdf
    tmp=$tpf.pdf
    qpdf --stream-data=uncompress --normalize-content=n --qdf \
        "$dst" "$tq1" || die 'qpdf error converting to QDF'
    # remove PDF page groups (required)
    # also remove Producer info (size improvement)
    perl -e '
        use strict;
        use warnings;

        my $s = "0";
        my $ispage = 0;
        my $gt = "<ERR>";
        while (my $line = <>) {
            my $skip = 0;
            chomp(my $l = $line);
            if ($l eq "<<") {
                $ispage = 0;
                $s = 1 if $s eq 0;
            } elsif ($l eq ">>") {
                if ($s eq 3) {
                    # only omit page groups
                    print $gt unless $ispage;
                }
                $s = 0;
            } elsif ($l eq "  /Type /Page") {
                if (($s == 1) || ($s == 3)) {
                    $ispage = 1;
                }
            } elsif (($s == 1) && ($l eq "  /Group <<")) {
                $gt = $line;
                $s = 2;
                $skip = 1;
            } elsif (($s == 1) && ($l =~ qr`^  /Group `)) {
                $gt = $line;
                $s = 3;
                $skip = 1;
            } elsif ($s == 2) {
                $gt .= $line;
                $s = 3 if $l eq "  >>";
                $skip = 1;
            } elsif ($l eq "trailer <<") {
                $s = 4;
            } elsif ($s == 4) {
                # size optimisation hack
                # remove CreationDate, Producer, etc.
                $skip = 1 if $l =~ qr`^  /Info `;
            }
            print $line unless $skip;
        }
    ' <"$tq1" >"$tq2" || die 'error during perl QDF filtering'
    fix-qdf <"$tq2" >"$tq3" || die 'error during fix-qdf'
    # bullseye first then buster which lacks options
    if ! (set -x; qpdf --stream-data=compress --recompress-flate \
        --compression-level=9 --normalize-content=n \
        --object-streams=disable --deterministic-id \
        "$tq3" "$tmp") 2>"$tq1" && ! (set -x; qpdf \
        --stream-data=compress --normalize-content=n \
        --object-streams=disable --deterministic-id \
        "$tq3" "$tmp") 2>>"$tq1"; then
        cat "$tq1" >&2
        die 'qpdf error converting from QDF'
    fi
    mv "$tmp" "$dst" || die 'could not create target file'
    rm "$tpf"* || die 'error cleaning up tmp QDF files'
    echo >&2 "I: fixed up $dst"
done
echo >&2 "I: done"

script caveats

The line…

$skip = 1 if $l =~ qr`^  /Info `;

… additionally strips Producer information. Comment it out to retain that, but as it’ll mostly always be just Cairo (for Inkscape) and needlessly take up space, I’m dropping it in our use cases.

real-world SVG → PDF conversion

svg.ksh (which uses mksh as scripting shell; for this answer, I converted the above partial script to POSIX sh, but for the full use I prefer this as I also develop mksh which has a few nice safety features) can be used in automated (and not) scenarios to convert SVG images to PDF using Inkscape, cleaning them up with qpdf in the same go. We use this in Makefiles that use pdfTₑχ/LᴬTᴇΧ to convert source files to PDF documents in one go.

Answer 13

I had the same problem when I was using the SVG file. To solve my problem and to automate the process under Microsoft Windows OS I did:

1) Add to the environment variable 'PATH' the full folder paths of Inkscape and Ghostviewer command line executables;

2) You may need to restart the computer, in order to update the environment variable 'PATH' with your changes;

3) Edit a batch file with the following content and name it 'svg2pdf.bat'. Save it in your main folder of the Texniccenter project:

@echo off
SET FileName=%1

::Convert forward to backward slash
SET FileName=%FileName:/=\%  

::generate pdf image from svg
inkscape -z -D --file=%FileName%.svg --export-pdf=%FileName%.pdf 
::remove the multiple page problem
gswin32c -sDEVICE=pdfwrite -o %FileName%_aux.pdf %FileName%.pdf  
::save the resultant pdf file
move /y %FileName%_aux.pdf %FileName%.pdf       

4) In the preamble of the tex file, define the latex commands:

\newcommand{\executeiffilenewer}[3]{%
    \ifnum\pdfstrcmp{\pdffilemoddate{#1}}%
        {\pdffilemoddate{#2}}>0%
        {\immediate\write18{#3}}%
    \fi%
}

\newcommand{\includesvgpdf}[2][\columnwidth]{%
    \executeiffilenewer{#2.svg}{#2.pdf}{svg2pdf #2}%
    \includegraphics[#1]{#2.pdf}%
}

This new latex command \includesvgpdf accepts 2 arguments, where the first one must be "width=ToDefine", where "ToDefine" will be the width and the second argument must be the SVG file name without the extension.

Instead of use the command

\includegraphics[width=0.5\textwidth]{SVG_FILE_NAME}

use

\includesvgpdf[width=0.5\textwidth]{SVG_FILE_NAME}

Hopping it also works for you.