1

I am trying to get a build script to run both on Linux and Windows. On Linux we're using TeXLive and on Windows MikTeX. In both cases latexmk is used to rerun pdflatex as often as needed to resolve the references.

Coming from the development of the script on Linux, it fails miserably on Windows after only one run (on Linux latexmk takes 3 runs of pdflatex) and I cannot figure out why.

On Linux, at the end of run 1, I get mytexfile.aux shown as a changed file. On Windows nothing like that happens. Consequently on Windows latexmk seems to see no need to rerun pdflatex.

Initially I thought this could be related to the fact that latexmk is a Perl script and perhaps curtails the precision of file timestamps. However ...

When running the pdflatex commands, which latexmk claims it executes (after cleaning, of course), manually the .aux file does indeed not change between run 1 and run 2 of pdflatex. The only two files that change are the .fls and .log files (.aux, .fdb_latexmk, .out, .toc remain unchanged). What's more, the .toc file is empty on Windows, which it isn't on Linux.

What I don't know as of yet is whether latexmk or pdflatex is the culprit here and whether the version differences between the individual packages in MikTeX and TeXLive respectively, play a relevant role in all this.

So how can I diagnose this issue further and more importantly how do I get this to run on Windows with MikTeX (I have no say in regards to why it has to be MikTeX on Windows and TeXLive on Linux)?

Edit 1: I basically grepped for error in the .log file after one comment on the question. The output was as follows:

! Package inputenc Error: Invalid UTF-8 byte sequence.
\GenericError  ...
!  ==> Fatal error occurred, no output PDF file produced!

Digging further I can see that the character tripping up the LaTeX engine appears to be é ... no idea why it barfs on Windows, but not on Linux, though.


On Windows (10 x64):

Versions: latexmk 4.63b, MiKTeX-pdfTeX 2.9.7029 (1.40.20) (MiKTeX 2.9.7050 64-bit)

Latexmk: This is Latexmk, John Collins, 17 March 2019, version: 4.63b.
Latexmk: applying rule 'pdflatex'...
Rule 'pdflatex': The following rules & subrules became out-of-date:
      'pdflatex'
------------
Run number 1 of rule 'pdflatex'
------------
------------
Running 'pdflatex -halt-on-error -synctex=0 -interaction=batchmode  -recorder  "mytexfile.tex"'
------------
This is pdfTeX, Version 3.14159265-2.6-1.40.20 (MiKTeX 2.9.7050 64-bit)
entering extended mode
=== TeX engine is 'pdfTeX'
Latexmk: List of undefined refs and citations:
...
Latexmk: Summary of warnings from last run of (pdf)latex:
  Latex failed to resolve 83 reference(s)
Latexmk: Errors, so I did not complete making targets
Latexmk: Use the -f option to force complete processing,
 unless error was exceeding maximum runs, or warnings treated as errors.

On Linux (Ubuntu 16.04):

Versions: latexmk 4.41, pdfTeX 3.14159265-2.6-1.40.16 (TeX Live 2015/Debian)

NB: ellipsis denotes that I removed some lines for brevity (but only those naming the references).

Latexmk: This is Latexmk, John Collins, 1 January 2015, version: 4.41.
Latexmk: applying rule 'pdflatex'...
Rule 'pdflatex': Rules & subrules not known to be previously run:
   pdflatex
Rule 'pdflatex': The following rules & subrules became out-of-date:
      'pdflatex'
------------
Run number 1 of rule 'pdflatex'
------------
------------
Running 'pdflatex -halt-on-error -synctex=0 -interaction=batchmode  -recorder  "mytexfile.tex"'
------------
This is pdfTeX, Version 3.14159265-2.6-1.40.16 (TeX Live 2015/Debian) (preloaded format=pdflatex)
 restricted \write18 enabled.
entering extended mode
Latexmk: References changed.
Latexmk: References changed.
Latexmk: Log file says output to 'mytexfile.pdf'
Latexmk: List of undefined refs and citations:
...
Latexmk: Summary of warnings:
  Latex failed to resolve 83 reference(s)
Latexmk: applying rule 'pdflatex'...
Rule 'pdflatex': File changes, etc:
   Changed files, or newly in use since previous run(s):
      'mytexfile.aux'
------------
Run number 2 of rule 'pdflatex'
------------
------------
Running 'pdflatex -halt-on-error -synctex=0 -interaction=batchmode  -recorder  "mytexfile.tex"'
------------
This is pdfTeX, Version 3.14159265-2.6-1.40.16 (TeX Live 2015/Debian) (preloaded format=pdflatex)
 restricted \write18 enabled.
entering extended mode
Latexmk: Log file says output to 'mytexfile.pdf'
Latexmk: List of undefined refs and citations:
...
Latexmk: Summary of warnings:
  Latex failed to resolve 54 reference(s)
  Latex found 1 multiply defined reference(s)
Latexmk: applying rule 'pdflatex'...
Rule 'pdflatex': File changes, etc:
   Changed files, or newly in use since previous run(s):
      'mytexfile.aux'
      'mytexfile.out'
      'mytexfile.toc'
------------
Run number 3 of rule 'pdflatex'
------------
------------
Running 'pdflatex -halt-on-error -synctex=0 -interaction=batchmode  -recorder  "mytexfile.tex"'
------------
This is pdfTeX, Version 3.14159265-2.6-1.40.16 (TeX Live 2015/Debian) (preloaded format=pdflatex)
 restricted \write18 enabled.
entering extended mode
Latexmk: Log file says output to 'mytexfile.pdf'
Latexmk: List of undefined refs and citations:
...
Latexmk: Summary of warnings:
  Latex failed to resolve 54 reference(s)
  Latex found 1 multiply defined reference(s)
Latexmk: All targets (mytexfile.pdf) are up-to-date
  • The latexmk log for windows says Latexmk: Errors, so I did not complete making targets. What does the pdflatex log say? – Phelype Oleinik Jun 13 at 16:18
  • I figured I'd check for updates, but on Windows I get this issue reproducibly even with the latest pdflatex version (Version 3.14159265-2.6-1.40.20 (MiKTeX 2.9.7050 64-bit)). – 0xC0000022L Jun 13 at 16:18
  • 3
    you're using Windows...that's your source of problem(s)... – naphaneal Jun 13 at 16:24
  • @naphaneal well, unfortunately sometimes one doesn't have the choice, although I concur. – 0xC0000022L Jun 13 at 16:27
  • 1
    Windows works just fine, except when people start taking random advice from "experts" on the internet about how to improve it, and screw it up :) – alephzero Jun 13 at 16:31
1

I had come to rely on latexmk showing me what went wrong with references so much, I didn't even think of looking into the .log file. Fortunately Phelype Oleinik set me straight in a comment.

From that I could see:

! Package inputenc Error: Invalid UTF-8 byte sequence.

See the inputenc package documentation for explanation.
Type  H <return>  for immediate help.
 ...                                              

\GenericError  ...                                

... followed by a few (mangled) lines seemingly showing the cause.

The problem is in the (Python-based) pre-processing step which differs between Windows and Linux, it seems. In the source code from which we extract bits and pieces for documentation I can see the following bytes: c3 a9 which decode to é when treated as Unicode (see Python output):

>>> b"\xc3\xa9".decode('utf-8')
'é'

... but these get converted to latin-1 (or probably that closely related standard Windows ANSI codepage depending on the locale), yielding an invalid byte sequence when treated as UTF-8 ... which in turn make pdflatex choke.

  • told ya, Windows is the source of problems... – naphaneal Jun 13 at 16:56
  • 3
    It is not clear if it is the preprocessing step: you are using a rather new miktex with a new latex with utf8 preloaded, while your linux tex system is old and without utf8. – Ulrike Fischer Jun 13 at 16:59
  • @UlrikeFischer the opposite. The preprocessing step is a Python script. Roughly we extract documentation and type information from (C/C++) headers by way of Doxygen. That XML is then digested by a Python script which uses Jinja2 as templating engine to create LaTeX files. Now on Linux I know for sure the locale defaults to using UTF-8 and on Windows it doesn't. So the issue is that on Windows a file is produced with an invalid code (in terms of UTF-8), leading to pdfLaTeX barfing on it. – 0xC0000022L Jun 17 at 7:54
  • Did you try to copy the file created on linux to the windows system and compile it? – Ulrike Fischer Jun 17 at 8:02
  • @UlrikeFischer Yep, it works that way around. But not the other. The problems were 1. that Python defaults to a particular encoding (and a different one) on either platform and 2. that I didn't diligently check the log file for issues. On my end the mitigation is now to inspect the log file more thoroughly. – 0xC0000022L Jun 17 at 8:04

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.