# Is it ImageMagick's fault or pdflatex's that some JPEGs aren't working?

pdflatex (as well as lualatex and xelatex) cannot process (some) JPEG images which were created by my ImageMagick.

I'm on OSX Mavericks, using ImageMagick v6.9.0-0 Q16 x86_64 provided via MacPorts.

My LaTeX version installed is this:

$latex --version pdfTeX 3.14159265-2.6-1.40.15 (TeX Live 2014/MacPorts 2014_8) kpathsea version 6.2.0 [....] Compiled with libpng 1.6.17; using libpng 1.6.17 Compiled with zlib 1.2.8; using zlib 1.2.8 Compiled with poppler version 0.32.0  I've tried to construct a minimal working example so everybody can reproduce the problem. If you can't, please tell which exact version of IM and LaTeX you run. ## Preparation Here are two JPEGs, both created by ImageMagick. The left one isn't working, the right one is: (see update below the images!) Update: Imgur hosting seems to strip some metadata, so the files aren't any longer in their original state. Please use this link on GoogleDrive to a mini-tarball. It contains the original JPEGs. I used these commands to create them: $ convert wizard:              wiz-standard.jpg   # (left image)
$convert wizard: -density 300 wiz-300densi.jpg # (right image)  ## Data points about the two JPEGs Here are a few data points about their differences. ### File sizes $ ls -l wiz-300densi.jpg wiz-standard.jpg
-rw-r--r--  1 kp  staff  62177  7 Mai 12:09 wiz-300densi.jpg
-rw-r--r--  1 kp  staff  62177  7 Mai 12:09 wiz-standard.jpg


As you can see, exactly same file sizes.

>     compare -metric $i wiz-300densi.jpg wiz-standard.jpg null: ; > echo; > done Metric AE : 0 Metric Fuzz : 0 (0) Metric MAE : 0 (0) Metric MEPP : 0 (0, 0) Metric MSE : 0 (0) Metric NCC : 1 Metric PAE : 0 (0) Metric PHASH : 0 Metric PSNR : inf Metric RMSE : 0 (0)  As you can see, comparing the color values of every single pixel does not yield any differences according to every single available metric. ### Image metadata $ identify -verbose wiz-300densi.jpg > wiz-300densi.log
$identify -verbose wiz-standard.jpg > wiz-standard.log$ sdiff -sbB wiz-densi300.log wiz-standard.log
Image: wiz-300densi.jpg                    | Image: wiz-standard.jpg
Resolution: 300x300                      <
Print size: 1.6x2.13333                  <
date:create: 2015-05-07T12:09:45+02:00 |     date:create: 2015-05-07T12:09:20+02:00
date:modify: 2015-05-07T12:09:45+02:00 |     date:modify: 2015-05-07T12:09:20+02:00
filename: wiz-300densi.jpg             |     filename: wiz-standard.jpg


Here is the first difference: the "not-working" JPEG does no contain any hint in its metadata about the prefered resolution it wants to be be rendered on an output device.

$exiftool wiz-standard.jpg | grep Resolution Resolution Unit : inches X Resolution : 1 Y Resolution : 1$ exiftool wiz-300densi.jpg | grep Resolution
Resolution Unit                 : inches
X Resolution                    : 300
Y Resolution                    : 300


Exiftool also reports a difference: However, it reports not an empty value (like identify -verbose did for the "not-working" JPEG), but a value of 1. I don't know if this is really in the file's data, or if exiftool just assumes and reports this value in leu of a missing metadata entry.

## LaTeX code (MWE)

Here is my LaTeX code:

\documentclass[]{article}
\usepackage{graphicx}

\begin{document}

\begin{figure}[htbp]
\centering
\includegraphics{./wiz-standard.jpg}
\end{figure}

\end{document}


## pdflatex run + output

Running pdflatex on it yields these messages:

This is pdfTeX, Version 3.14159265-2.6-1.40.15 (TeX Live 2014/MacPorts 2014_8) (preloaded format=pdflatex)
restricted \write18 enabled.
entering extended mode
(./mini.tex
LaTeX2e <2014/05/01>
Babel <3.9k> and hyphenation patterns for 50 languages loaded.
(/opt/local/share/texmf-texlive/tex/latex/base/article.cls
Document Class: article 2007/10/19 v1.4h Standard LaTeX document class
(/opt/local/share/texmf-texlive/tex/latex/base/size10.clo))
(/opt/local/share/texmf-texlive/tex/latex/graphics/graphicx.sty
(/opt/local/share/texmf-texlive/tex/latex/graphics/keyval.sty)
(/opt/local/share/texmf-texlive/tex/latex/graphics/graphics.sty
(/opt/local/share/texmf-texlive/tex/latex/graphics/trig.sty)
(/opt/local/share/texmf-texlive/tex/latex/latexconfig/graphics.cfg)
(/opt/local/share/texmf-texlive/tex/latex/pdftex-def/pdftex.def
(/opt/local/share/texmf-texlive/tex/generic/oberdiek/infwarerr.sty)
(/opt/local/share/texmf-texlive/tex/generic/oberdiek/ltxcmds.sty))))
(./mini.aux) (/opt/local/share/texmf-texlive/tex/context/base/supp-pdf.mkii
) (/opt/local/share/texmf-texlive/tex/generic/oberdiek/pdftexcmds.sty
(/opt/local/share/texmf-texlive/tex/generic/oberdiek/ifluatex.sty)
(/opt/local/share/texmf-texlive/tex/generic/oberdiek/ifpdf.sty))
(/opt/local/share/texmf-texlive/tex/latex/oberdiek/epstopdf-base.sty
(/opt/local/share/texmf-texlive/tex/latex/oberdiek/grfext.sty
(/opt/local/share/texmf-texlive/tex/generic/oberdiek/kvdefinekeys.sty))
(/opt/local/share/texmf-texlive/tex/latex/oberdiek/kvoptions.sty
(/opt/local/share/texmf-texlive/tex/generic/oberdiek/kvsetkeys.sty
(/opt/local/share/texmf-texlive/tex/generic/oberdiek/etexcmds.sty)))
(/opt/local/share/texmf-texlive/tex/latex/latexconfig/epstopdf-sys.cfg))

pdfTeX warning: pdflatex: arithmetic: number too big

pdfTeX warning: pdflatex: arithmetic: number too big
<./wiz-standard.jpg, id=1, --32768.0pt x 0.0pt> <use ./wiz-standard.jpg>
[1{/opt/local/var/db/texmf/fonts/map/pdftex/updmap/pdftex.map} <./wiz-standard.
jpg>] (./mini.aux) )</opt/local/share/texmf-texlive/fonts/type1/public/amsfonts
/cm/cmr10.pfb>
Output written on mini.pdf (1 page, 70603 bytes).
Transcript written on mini.log.


So from the MWE-LaTeX above, pdflatex at least did generate an output PDF of 70 KiBytes, while giving some warnings, but...

## Empty (white) page

...but the PDF only shows an "empty", white page. (I'll not show a screenshot of it, though :)

pdfimages reports an embedded image in this PDF:

$pdfimages -list mini.pdf page num type width height color comp bpc enc interp object ID x-ppi y-ppi size ratio -------------------------------------------------------------------------------------------- 1 0 image 480 640 rgb 3 8 jpeg no 1 0 2 2 60.7K 6.7%  It can even extract the JPEG again from the PDF, and it is in its original shape: $  pdfimages -j  mini.pdf mini---

$ls -ltar mini----000.jpg wiz-standard.jpg -rw-r--r-- 1 kp staff 62177 7 Mai 12:09 wiz-standard.jpg -rw-r--r-- 1 kp staff 62177 8 Mai 02:09 mini----000.jpg  Note, that in other cases, with a non-minimal TeX file I also experience a different pdflatex message, and the run doesn't complete and no PDF is created at all (not even an empty page): ## pdflatex with errors (not just warnings) on non-MWE LaTeX pdfTeX warning: pdflatex: arithmetic: number too big pdfTeX warning: pdflatex: arithmetic: number too big <./wiz-standard.jpg, id=4, --32768.0pt x 0.0pt> ! Package graphics Error: Division by 0. See the graphics package documentation for explanation. Type H <return> for immediate help. ... l.63 \includegraphics{./wiz-standard.jpg} ?  Note the Division by 0.-message... ## lualatex not completing run with the MWE from above I also tried lualatex. This reports a slightly different message: LuaTeX warning: arithmetic: number too big LuaTeX warning: arithmetic: number too big ! Dimension too large. <recently read> \dimen@ l.8 \includegraphics{./wiz-standard.jpg} ?  No PDF is created by lualatex, not even an empty page. ## xelatex not completing run with the MWE from above These are xelatex's messages on the MWE: ! Dimension too large. <to be read again> b l.8 \includegraphics{./wiz-standard.jpg} ?  No PDF is created by xelatex, not even an empty page. ## Questions Be aware, that the alternative JPEG, which contains the resolution hints, is successfully embedded and a PDF is created by either of the {pdf,lua,xe}latex commands. My questions are: 1. Can you reproduce this problem with your versions of LaTeX and of ImageMagick? 2. Why are {pdf,lua,xe}latex relying on some metadata inside a JPEG so much, that it even kills itself when this metadata is wrong (or missing) instead of simply just using the real raster pixel data which is there? 3. Isn't that a bug in the respective components (even if we accept that the JPEG itself may be "buggy") ?!? ## Update 1: (In response to comment by Mike Renfro...) My non-MWE LaTeX code contains this snippet: \usepackage{graphicx} % Redefine \includegraphics so that, unless explicit options are % given, the image width will not exceed the width of the page. % Images get their normal width if they fit onto the page, but % are scaled down if they would overflow the margins. \makeatletter \def\ScaleIfNeeded{% \ifdim\Gin@nat@width>\linewidth \linewidth \else \Gin@nat@width \fi } \makeatother \let\Oldincludegraphics\includegraphics {% \catcode\@=11\relax% \gdef\includegraphics{\@ifnextchar[{\Oldincludegraphics}{\Oldincludegraphics[width=\ScaleIfNeeded]}}% }%  So this should take care of the scaling, no? ## Update 2: The images I linked to originally have been stripped of metadata and are no longer in their original shape. Here is a link on GoogleDrive to a mini-tarball containing the original JPEGs. # Update 3 I forgot to mention: I had also tested with $ convert wizard: -density 1 wiz-001densi.jpg


So the resulting file does include the respective metadata, and

• exiftool reports {X,Y} Resolution : 1, and
• identify -verbose reports Resolution: 1x1.

(This is already mentioned in my previous postings to the ImageMagick forums as well as to the Pandoc forum.)

So this fact somehow contradicts the some speculations given in some of the answers below...

(In order to resolve the question wether the JPEG itself is buggy, I also reported this issue in the appropriate ImageMagick forum).

(Update: Just saw a new message on the ImageMagick "Bugs" forum by Dirk Lemstra saying: "I just committed a patch to our SVN repository to make sure that the default density units is set to 0 (unknown) instead of 1 (pixels per inch).")

• I'd guess it's the "1 dpi" resolution, given that you end up with a 3000+ inch image error. I've never included a raster graphic without a [width=] parameter. [width=0.5\textwidth] or similar is most common. Occasionally [scale=], but not nearly as often. – Mike Renfro May 8 '15 at 0:53
• @MikeRenfro: I've been asked to provide an MWE, so here it is. I've also updated my posting with he snippet from my non-MWE-TeX code, where this problem occurred the first time. And there the scaling is taken care of... Unless you can point me to the flaw in it (I'm a LaTeX Greenhorn, really.) – Kurt Pfeifle May 8 '15 at 1:08
• PDF may support media up to 280km but TeX does not. There is a divide by zero error but there is also a dimension too large error right before that, which I'm assuming is due to the enormous width. @MikeRenfro It is generally possible to include raster images without width or anything - just at natural size. It just rarely works out very well unless you've designed the images specifically to fit ;). – cfr May 8 '15 at 3:23
• BTW: 2^15 == 32768... and pdflatex's error message was: "number too big <./wiz-standard.jpg, id=4, --32768.0pt x 0.0pt". – Kurt Pfeifle May 8 '15 at 4:11
• – Reinstate Monica - M. Schröder May 8 '15 at 8:33

As it happens I stumbled upon the same problem last week and the next version of LuaTeX will detect these images and issue a warning:

LuaTeX warning (file wiz-standard.jpg): The image specifies an unusual resolution of 1dpi by 1dpi.


pdfTeX/luaTeX read jpegs bytewise (no exif involved) and images of these kind really specify a resolution of 1dpi by 1dpi (see the JFIF spec, page 5). This typically gives images that are too large for TeX to handle (see \maxdimen; the limit would be around 200 x 200 pixels). wiz-standard.jpg has 480x640 pixels, so TeX thinks you want to include an image that's 480inx640in - that's way too large.

• I appreciate your feedback as well as the LuaTeX fix (whoever is responsible for it -- you?). – Kurt Pfeifle May 8 '15 at 10:36
• @KurtPfeifle: Me. I was at a TeX conference (BachoTeX) with the LuaTeX developers in the same room and a friend had the exact same problem: Some of his hundreds of images would show this problem. – Reinstate Monica - M. Schröder May 8 '15 at 10:47

The image wiz-standard.jpg does not have any resolution DPI settings set (the other image is set to 300 and 300).

Several image software packages do not set this properly as it is rarely used (including by TeX as far as I know), but for some odd reason TeX requires it to be set (to anything you like - because it is not used - whether you choose 100x100 or 400x400 you will get the same LaTeX output).

It is called the JFIF header which actually doesn't need to exist in JPEG anyway - so it's a bug in LaTeX as far as I am concerned.

You can correct it (in a lossy way) by re-saving the file; but you can also do it losslessly by simply adding the DPI values to the header.

This works in Windows, but may work on other platforms:

• Use exiftool as a command line tool (you can do this in the exiftoolgui via exiftooldirect button) to add in a special header.

These are the command line arguments to exiftool:

-jfif:Xresolution=300  -jfif:Yresolution=300  -jfif:ResolutionUnit=inches


The inches may be replaced by cm or None to insert the other two valid values for the respective field.

Added information: Clearly DPI information is used by \includegraphics if no size is specified - I tried a few experiments, and if there is a JFIF header, and if the image is set at its "natural" resolution with

\includegraphics{xxxx.jpg}


then the image size in the compiled PDF does depend on the DPI. However if any option is set to constrain the height or width the DPI setting is ignored. It seems reasonable to me that compilation should not fail if the DPI is missing but is not used because it is irrelevant. Interestingly the option [resolution=300,width=0.3\textwidth] does not correct the error as (I think) the resolution option to \includegraphics only works if the JFIF is absent (rather than present with no DPI setting)

• I appreciate our feedback. "TEX requires it to be set (to anything ...)". I can confirm that: it works even when set to 1x1. I had forgotten to mention this in my original posting. Yes, it is a stupid bug in TeX/LaTeX/pdflatex -- I came to that conclusion meanwhile. It is a pain in the ass to try and "fix" all those conforming JPEGs after they killed a pdflatex run... – Kurt Pfeifle May 8 '15 at 10:34
• The image in question explicitely declares itself has a JFIF image having "X and Y are dots per inch" and then 1 horizontal pixel density and 1 vertical pixel density. See the JFIF spec, page 5. – Reinstate Monica - M. Schröder May 8 '15 at 10:42
• I think we can word the bug in a different way: DPI data may or may not exist in jpg images. Some image software is DPI aware and uses the DPI. LaTex is DPI aware and requires the DPI, but does not actually use the DPI. – Aubrey Blumsohn May 8 '15 at 11:14
• An earlier comment of mine reads: "I appreciate our feedback." Of course it should be your feedback which I appreciate. My keyboard is bouncing and blocking for individual keystrokes, and I do not always notice it immediately... – Kurt Pfeifle May 8 '15 at 15:44
• Clearly part of my answer is incorrect - I have corrected – Aubrey Blumsohn May 10 '15 at 22:46

My own conclusion, after running a series of more tests, and after studying a bit more the official spec for JFIF (JPEG File Interchange Format) file format (PDF) is this:

1. ImageMagick is not at fault.

2. All of pdflatex, lualatex and xelatex are wrong and exhibit a bug when they do not properly embed the JPEG into the PDF page.

# JFIF spec

The JFIF spec describes 3 fields in the JPEG header:

1. At byte offset 0x0d the "Units" field. This field has a 1 byte length. It can hold 3 values. The values describe "Units of the X and Y densities".
• 0, means: "no units, X and Y describe the pixel aspect ratios".
• 1, means: "X and Y are dots per inch";
• 2, means: "X and Y are dots per cm".
2. At byte offset 0x0e the "XDensity" field. This field has 2 bytes length. It describes the "Horizontal pixel density".
3. At byte offset 0x10 the "YDensity" field. Its length is 2 bytes. It describes the "Vertical pixel density".

+---------------+---------+-----------------+---------------------+
|   Byte-Offset |   0x0d  |        0x0e     |        0x10         |
+---------------+---------+-----------------+---------------------+
|    Field-Name | "Units" |   "XDensity"    |   "YDensity"        |
+---------------+---------+-----------------+---------------------+
| Value/Meaning |      0  | "X and Y describe pixel aspect ratio" |
+---------------+---------+-----------------+---------------------+
| Value/Meaning |      1  |     "X and Y are dots per inch"       |
+---------------+---------+-----------------+---------------------+
| Value/Meaning |      1  |      "X and Y are dots per cm"        |
+---------------+---------+-----------------+---------------------+


## Test script

I created a test script (Bash) which does the following:

1. Create JPEG files from the same source passing different -density parameters to convert.
2. Log different properties of the JPEG files in text files.
3. Use the respective JPEG file in a LaTeX document (MWE) which was created by the same script.
4. Run each of {pdf,xe,lua}latex to create a PDF.
5. Log some properties of the PDF files in text files.
6. Uncompress the PDFs' /Content streams for manual inspection.

The script code is here:

#!/bin/bash
#   Script written by  <kurt AT pfeifle DOT mykolab AT com>
#   No warranty --
#     -- feel free to use/modify for whatever purpose --
#     -- use at your own risk.

mkdir jpeg-imagemagick-latex-pdflatex-lualatex-xelatex-testing
cd jpeg-imagemagick-latex-pdflatex-lualatex-xelatex-testing
rm -rf testwiz.jpg wiz-*

cat << _EOFHERE_ > mini.tex
\documentclass[a4paper]{article}
\usepackage{graphicx}
\begin{document}
\thispagestyle{empty}
\begin{figure}[htbp]
\centering
\includegraphics{./testwiz.jpg}
\end{figure}
\end{document}
_EOFHERE_

for density in 000 0.2 0.5 0.8 001 1.2 1.5 1.8 002 2.2 2.5 2.6 010 100 ; do

convert wizard: -density ${density} wiz-${density}densi.jpg

md5sum wiz-${density}densi.jpg >> wiz-md5sums.txt ls -l wiz-${density}densi.jpg                     >> wiz-filesizes.txt

mkdir pdfoutdir-${density}densi ln -fs wiz-${density}densi.jpg testwiz.jpg

file wiz-${density}densi.jpg >> wiz-filemagic.txt echo "wiz-${density}densi.jpg :"                  >> wiz-identify-verbose-densi.txt
identify -verbose wiz-${density}densi.jpg \ | grep -E '(Resolution:|Print size:|filename:)' \ | gsed 's#^# #' >> wiz-identify-verbose-densi.txt echo "" >> wiz-identify-verbose-densi.txt for latexengine in pdflatex lualatex xelatex; do echo echo mkdir pdfoutdir-${density}densi/${latexengine} echo "=================================================" echo mkdir pdfoutdir-${density}densi/${latexengine}${latexengine} -interaction=nonstopmode                       \
-output-directory=pdfoutdir-${density}densi/${latexengine} mini.tex

ls -l pdfoutdir-${density}densi/${latexengine}/mini.pdf >> wiz-pdfsizes.txt

qpdf --qdf --object-streams=disable                           \
pdfoutdir-${density}densi/${latexengine}/mini.pdf         \
pdfoutdir-${density}densi/${latexengine}/qdf---mini.pdf

ls -l pdfoutdir-${density}densi/${latexengine}/qdf---mini.pdf \
>> wiz-qdfsizes.txt
done
done

for density in std ; do

convert wizard:              wiz-${density}densi.jpg md5sum wiz-${density}densi.jpg                    >> wiz-md5sums.txt

ls -l wiz-${density}densi.jpg >> wiz-filesizes.txt mkdir pdfoutdir-${density}densi

ln -fs wiz-${density}densi.jpg testwiz.jpg file wiz-${density}densi.jpg                      >> wiz-filemagic.txt

echo "wiz-${density}densi.jpg :" >> wiz-identify-verbose-densi.txt identify -verbose wiz-${density}densi.jpg \
| grep -E '(Resolution:|Print size:|filename:)' \
| gsed 's#^#   #'                             >> wiz-identify-verbose-densi.txt
echo ""                                           >> wiz-identify-verbose-densi.txt

for latexengine in pdflatex lualatex xelatex; do

echo
echo mkdir pdfoutdir-${density}densi/${latexengine}
echo "================================================="
echo

mkdir pdfoutdir-${density}densi/${latexengine}

${latexengine} -interaction=nonstopmode \ -output-directory=pdfoutdir-${density}densi/${latexengine} mini.tex ls -l pdfoutdir-${density}densi/${latexengine}/mini.pdf >> wiz-pdfsizes.txt qpdf --qdf --object-streams=disable \ pdfoutdir-${density}densi/${latexengine}/mini.pdf \ pdfoutdir-${density}densi/${latexengine}/qdf---mini.pdf ls -l pdfoutdir-${density}densi/\${latexengine}/qdf---mini.pdf \
>> wiz-qdfsizes.txt
done
done

echo
echo wiz-pdfsizes.txt
echo ================
cat  wiz-pdfsizes.txt
echo

echo wiz-qdfsizes.txt
echo ================
cat  wiz-qdfsizes.txt
echo

echo wiz-filesizes.txt
echo =================
cat  wiz-filesizes.txt
echo

echo wiz-filemagic.txt
echo =================
cat  wiz-filemagic.txt
echo

echo wiz-identify-verbose-densi.txt
echo ==============================
cat  wiz-identify-verbose-densi.txt
echo

echo wiz-md5sums.txt
echo ===============
cat  wiz-md5sums.txt | sort

cd -


## ImageMagick's different output

My test script for generating JPEGs based on ImageMagick's builtin demo wizard: image yielded 3 different results:

1. JFIF headers with "Units" field set to 0 ("pixel aspect ratio") and "X/Y Densities to 0 each.
2. JFIF headers with "Units" field set to 0 ("pixel aspect ratio") and "X/Y Densities to 1 or bigger each.
3. JFIF headers with "Units" field set to 1 ("dots per inch") and "X/Y Densities to 1 each. (I could not successfully set higher values to these fields while the "Units" was set to the default PixelsPerInch being equivalent to 1 in the JFIF header).

# {pdf,xe,lua}latex behavior

Each of the above three combinations of content in the JFIF header fields led to different results when inserting the respective JPEG into LaTeX-PDFs. (I didn't test the "X and Y are dots per cm" variation, which could be generated by additionally passing -units PixelsPerCentimeter on the command line. By default ImageMagick silently uses -units PixelsPerInch.)

Here are the details:

1. Units == 0 ("X and Y describe pixel aspect ratios") with XxY == 0x0 (XDensity x YDensity):

• This causes xelatex to mis-place and mis-scale the image on the page. It is then only partially visible.
• pdflatex has no problem here and creates a correct PDF page.
• lualatex has no problem here and creates a correct PDF page.
• The condition in the JPEG was created with this command:
convert wizard: -density 0.2 wiz-0.2densi.jpg
(Note that I "abused" the parameter by passing -density 0.2 -- pixels normally don't come in fracions... :)
2. Units == 0 ("X and Y describe pixel aspect ratios") with XxY == 1x1 or XxY == 2x2 or XxY == 3x3 or ... (XDensity x YDensity):

• This causes no problem, for neither of {pdf,xe,lua}latex.
• All PDF pages look correctly here. They are also identical to each other.
• So the actual value of these two fields didn't matter, as long as it wasn't 0 / 0. (I didn't test with 0 / 1 or 1 / 0 or 0 / 2 or 2 / 0 or any similar combination, though.)
• This condition in the JPEG was created with the command: convert wizard: -density 001 wiz-001densi.jpg (Note that passing -density 0.5 -- or 0.8 or 1 or 1.2 -- all inserted also 1/1 into the "X/YDensity" fields, despite of using "fractions" for the pixel settings...)
3. Units == 1 ("X and Y are dots per inch") with XxY == 1x1 (XDensity x YDensity):

• This causes each of {pdf,xe,lua}latex to create a PDF with an empty, white page.
• Each of these PDF producers emits different warnings or error messages during its run.
• This condition in the JPEG I could create with two different commands:
convert wizard: wiz-001densi.jpg or
convert wizard: -density 0 wiz-001densi.jpg

In all cases where the resulting PDF had a white page only or where the image was only partially visible the JPEG image data was completely embedded and...

1. ...I could again extract a valid and unbroken JPEG image with the help of the pdfimages command line tool;
2. ...I could uncompress the /Contents stream in the respective PDF, change a few parameters for the cm operator and make the image to become visible.

Here is what had to be changed in the /Contents streams:

## 1. PPDF files created by pdflatex

• created from wiz-stddensi.jpg or from wiz-000densi.jpg:

+--------------+------------------------------------------------+---------------------+
|              |           PDF source code line                 | Image on PDF page   |
+--------------+------------------------------------------------+---------------------+
| In PDF obj 5 | -32645.5791 0 0 -32645.5791 357.908 706.129 cm | invisible (outside) |
|    Change to |  480        0 0    640      124.802  76.093 cm | Image now visible   |
+--------------+------------------------------------------------+---------------------+

• created from all other JPEGs: PDFs were OK.

## 2. PDF files created by xelatex

• created from wiz-stddensi.jpg or from wiz-000densi.jpg:

+--------------+-----------------------------------------+---------------------+
|              |           PDF source code line          | Image on PDF page   |
+--------------+-----------------------------------------+---------------------+
| In PDF obj 5 | 34560 0 0 46080 52.8  -63.76 cm /Im0 Do | invisible (outside) |
|    Change to |   480 0 0   640 52.8 -693.8  cm /Im0 Do | Image now visible   |
+--------------+-----------------------------------------+---------------------+

• created from wiz-0.2densi.jpg:*

+--------------+-----------------------------------------+---------------------+
|              |           PDF source code line          | Image on PDF page   |
+--------------+-----------------------------------------+---------------------+
| In PDF obj 5 |   480 0 0   640 52.8  -63.76 cm /Im0 Do | partially outside   |
|    Change to |   480 0 0   640 52.8 -693.8  cm /Im0 Do | Image now visible   |
+--------------+-----------------------------------------+---------------------+

• created from all other JPEGs: PDFs were OK.

## 3. PDF files created by lualatex

• created from wiz-stddensi.jpg or from wiz-000densi.jpg:

+--------------+--------------------------------------------------------+---------------------+
|              |           PDF source code line                         | Image on PDF page   |
+--------------+--------------------------------------------------------+---------------------+
| In PDF obj 5 | -32645.5776 0 0 -32645.5776 124.802 706.129 cm /Im1 Do | invisible (outside) |
|    Change to |     481     0 0    640      124.802  76.093 cm /Im1 Do | Image now visible   |
+--------------+--------------------------------------------------------+---------------------+

• created from all other JPEGs: PDFs were OK.

# Inserting other JFIF header values with exiftool

After I inserted various other values of JFIF headers into the JPEG file (with the help of exiftool), it turned out that each *latex PDF generator:

1. ...successfully created a valid PDF when embedding the JPEG into its pages, but
2. ...didn't actually seem to care about scaling, resolution and aspect ratios requested by the header.

All generators only seem to trip over the JFIF header when the following condition applied:

+-------------+-------------------------+------------+------------+
| Byte-Offset |                   0x0d  |      0x0e  |      0x10  |
+-------------+-------------------------+------------+------------+
|  Field-Name |                 "Units" | "XDensity" | "YDensity" |
+-------------+-------------------------+------------+------------+
|       Value |                      1  |      1     |    1       |
+-------------+-------------------------+------------+------------+
|     Meaning | "X/YDensities are DPI"  |      1 dpi |    1 dpi   |
+-------------+-------------------------+------------+------------+


Funnily the following condition lets none of the PDF generators trip. This is despite the "more extreme" setting for "X/YDensity" to 0. pdflatex and lualatex generate a valid PDF looking as it ought to look. xelatex however places most part of the image outside of the pages /MediaBox:

+-------------+-----------------------------+------------+------------+
| Byte-Offset |                       0x0d  |      0x0e  |      0x10  |
+-------------+-----------------------------+------------+------------+
|  Field-Name |                     "Units" | "XDensity" | "YDensity" |
+-------------+-----------------------------+------------+------------+
|       Value |                          0  |      0     |    0       |
+-------------+-----------------------------+------------+------------+
|     Meaning | "X/YDensities pixel ratios" |      0 px  |    0 px    |
+-------------+-----------------------------+------------+------------+


## "X/YDensities pixel ratios"

All of the following conditions let pdflatex and lualatex work produce valid PDFs. Each one looking identical, regardless the actual X/YDensity values. None is scaled, buckled or stretched in any way. Each JPEG is displayed on the PDF page with a resolution of 72 PPI.

+-------------+-----------------------------+------------+------------+
| Byte-Offset |                       0x0d  |      0x0e  |      0x10  |
+-------------+-----------------------------+------------+------------+
|  Field-Name |                     "Units" | "XDensity" | "YDensity" |
+-------------+-----------------------------+------------+------------+
|       Value |                          0  |      7     |    7       |
+-------------+-----------------------------+------------+------------+
|       Value |                          0  |    100     |  100       |
+-------------+-----------------------------+------------+------------+
|       Value |                          0  |    200     |  200       |
+-------------+-----------------------------+------------+------------+
|       Value |                          0  |    300     |  300       |
+-------------+-----------------------------+------------+------------+
|       Value |                          0  |    100     |  300       |
+-------------+-----------------------------+------------+------------+
|       Value |                          0  |    300     |  100       |
+-------------+-----------------------------+------------+------------+
|       Value |                          0  |    300     |    1       |
+-------------+-----------------------------+------------+------------+
|     Meaning | "X/YDensities pixel ratios" | <above> px | <above> px |
+-------------+-----------------------------+------------+------------+


Most of the time xelatex also produced identical output. In the last, extreme case where XDensity was set to 300 and YDensity to 1, xelatex moved the biggest part of the image to outside of the /MediaBox.

## "X/YDensities are DPI"

All of the following conditions lets all three, xelatex, pdflatex and lualatex work produce valid PDFs. The results for the three tools are slighly different, but seem to actually follow the given X/YDensity values. If the value of "XDensity" is different from the value for "YDensity" the image is scaled or buckled. For identical values between the two fields the image is scaled accordingly keeping the aspect ratio: small values enlarge the image, large values scale it down.

+-------------+-----------------------------+------------+------------+
| Byte-Offset |                       0x0d  |      0x0e  |      0x10  |
+-------------+-----------------------------+------------+------------+
|  Field-Name |                     "Units" | "XDensity" | "YDensity" |
+-------------+-----------------------------+------------+------------+
|       Value |                          1  |      7     |    7       |
+-------------+-----------------------------+------------+------------+
|       Value |                          1  |    100     |  100       |
+-------------+-----------------------------+------------+------------+
|       Value |                          1  |    200     |  200       |
+-------------+-----------------------------+------------+------------+
|       Value |                          1  |    300     |  300       |
+-------------+-----------------------------+------------+------------+
|       Value |                          1  |    100     |  300       |
+-------------+-----------------------------+------------+------------+
|       Value |                          1  |    300     |  100       |
+-------------+-----------------------------+------------+------------+
|       Value |                          1  |    300     |    1       |
+-------------+-----------------------------+------------+------------+
|     Meaning |      "X/YDensities are DPI" | <above> px | <above> px |
+-------------+-----------------------------+------------+------------+


# Scope of this bug

ImageMagick does not change the JFIF headers of JPEGs it processes, unless told so on the command line.

So there is probably no danger for lots of people running into this bug when they create PDFs through LaTeX which include their precious digital photos.

This bug initially showed up because I used the builtin IM special file named wizard: as a source to create a JPEG. This triggered the extreme case of this bug, because unfortunately ImageMagick used a default DPI value of "1" for its output file (which in itself is probably a bug in ImageMagick).

This case can be worked around in ImageMagick by adding f.e. -density 72 to the command line.

However, it is still very questionable for the *latex PDF producing utilities to use the JFIF header metadata AT ALL for embedding JPEG images into pages. Because:

• most users are unaware of these JFIF headers
• most users don't know how to change these headers
• these headers control the scaling of JPEGs on PDF pages only indirectly
• most users expect to control the scaling of images in PDF output by different (LaTeX-provided) means directly
• the treatment of the various JFIF headers is inconsistent:
• if set as "X/Densities are DPI" they are NOT respected.
• if set as "X/YDensities are pixel ratios" they are (somehow) respected.

## Executive Summary

• The LaTeX-family PDF generators behave totally unreasonable here. On the one hand, whatever value "XDensity" and "YDensity" field hold if they are given as "DPI", they are NOT respected. On the other hand, if the "XDensity" and "YDensity" fields hold values meant to be "pixel aspect ratios", they ARE influencing the scaling of images on a page, but in very un-expected ways.

• The LaTeX PDF-generators should ignore the JFIF fields completely and only obey the statements given by the documents LaTeX code.
In the absence of such code, the PDF generators should assume 72 DPI.

• It is one thing if a user specifies absolute dimensions for scaling, but if they say e.g. scale=.5 that doesn't help. And to always ignore a specified DPI and impose a value seems potentially problematic. But I agree that they should treat the data consistently. – cfr May 11 '15 at 0:46
• there isn't really a "latex family" involved here, firstly latex isn't involved, latex just relies on the underlying tex engine to report the image size, so if there is a bug it is a bug (separately) in pdftex, luatex and xdvipdfmx (the dvi driver used by xetex) – David Carlisle May 11 '15 at 0:48
• Can you please open a bug with LuaTeX so we can track it there (a plain TeX test file would be much preferred)? I still believe we do it right, though... – Reinstate Monica - M. Schröder May 11 '15 at 12:43
• @MartinSchröder: Done! (But I must say, this bug tracking software's editor is, from a user's point of view one of the most crappy interfaces to handle. Can't you at least set it up with a mono-spaced font?!) – Kurt Pfeifle May 11 '15 at 16:37
• the test script creates the mini.tex file in the wrong directory; move that statement to just before the first for` statement. – Wybo Dekker May 13 '15 at 8:11