13

Take the simple latex source file:

\documentclass{article}
\usepackage{lipsum}
\usepackage{amsgen}
\begin{document}
\lipsum[1-10]
\end{document}

When processed with "latex" you always get the same DVI file, identical everywhere except for the date.

When processed (multiple times) with "pdflatex" you get the same PDF file, with the exception of an ID and the Date, and the same thing for "lualatex".

But when processed (multiple times) with "xelatex" you get wildly different PDF files, with different sizes. One can see the differences easily with "vimdiff".

Why is it the processing with "xelatex" not deterministic - not the same for the same sources?

Improve this question
7
  • 1
    I get no visual changes with xelatex. The pdf filesize oscillates between 10727, 10728, or 10729 bytes. Thus "wildly different" definitely does not apply on my system (mac os x). Which OS do you use? From which editor do you compile the source?
    – user4686
    Feb 13, 2016 at 7:22
  • vimdiff indeed indicates differences inside the pdf. Perhaps something having to do with font compression. Needs a pdf expert here.
    – user4686
    Feb 13, 2016 at 7:33
  • 2
    XeTeX produces exactly the same xdv files on multiple runs, i.e. their TeX contents are the same. Where's the issue here? (xdvipdfmx and dvipdfmx both produce some variation in the find PDF file but as already commented this is likely to do with font compression.)
    – Joseph Wright ♦
    Feb 13, 2016 at 8:12
  • 4
    @AFeldman: That's wrong: luatex does not claim to be 100% compatible to TeX.
    – Martin Schröder
    Feb 13, 2016 at 23:37
  • 1
    @AFeldman We are not talking about a run of xelatex producing the same file as a run of pdflatex!!! We are talking about multiple runs of xelatex on the same files, that is: running "xelatex file.tex" repeated time!
    – Paulo Ney
    Feb 14, 2016 at 15:25

2 Answers 2

Reset to default
11

The issue is related to the driver: xdvipdfmx. In order to generate unique tags for fonts, random number is used. Try

xelatex -no-pdf test

xdvipdfmx test.xdv
pdffonts test.pdf

The tag will change like

LYKESP+CMR10
CBIVMK+CMR10
...

each time you run

xdvipdfmx test.xdv
Improve this answer
8
  • But this doesn't explain the difference in size, does it?
    – Robert
    Feb 20, 2016 at 18:48
  • 1
    @Robert: The data are compressed by zlib as a stream. Different data may be compressed differently.
    – Akira Kakuto
    Feb 20, 2016 at 22:50
  • @AkiraKakuto Indeed this is the core of the problem, but I still do not understand why "xdvipdfmx" needs to generate a "unique" tag per font! Aren't these fonts uniquely identified by their names?
    – Paulo Ney
    Feb 21, 2016 at 21:43
  • 1
    It’s a very old practice to prepend tags to subsetted fonts embedded in PostScript and PDF files. I think it was a recommendation early on (probably because the embedded font, being a subset, is formally not the same as the full font).
    – Arthur Reutenauer
    Feb 22, 2016 at 13:27
  • 1
    @PauloNey: In TeX Live 2016, you will obtain exactly the same pdf file from a unique xdv file, if you define an environment variable SOURCE_DATE_EPOCH correctly like SOURCE_DATE_EPOCH=1456304492. The name of an output pdf file must be the same.
    – Akira Kakuto
    Feb 24, 2016 at 11:54
9

What seems to differ is the actual binary encodings of some parts. I trust that the pdf rendering does not vary at all. In my experience the file size changed only by plus or minus 1 byte (mac os x). I have done an ediff in hexadecimal mode of two such pdf's, here is a snapshot of where the first differences appear:

Blockquote

This first difference thus occurs in a part of the pdf relative to the partially embedded font. I don't know what causes this.

Generally speaking I can imagine that if you have say some 213 bytes thing which must be stored in 256 bytes, then the last 43 bytes may be random memory, if moreover one or many such things together are compressed then you get varying result. On undecompressing, there will be some varying random junk after structures terminators (or after a given number of bytes). Like non-coding DNA. Which perhaps is not so much non-coding, but let's not digress.

Only someone familiar with XeTeX source code can answer convincingly I guess.

Don't worry about the C-M-' undefined, I was trying to capture the thing via keyboard shortcuts I had forgotten.

Improve this answer
1
  • I have to admit that you flying over my head right now and I have not fully understood the meaning of everything you said ... nonetheless even if it is the compression if a font -- why would it compress differently on different runs? I still would expect the whole process to be deterministic and compress in exactly the same way.
    – Paulo Ney
    Feb 14, 2016 at 18:09

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .