2

I use latexmk in texstudio on win7. Working on a 400+ pages book with a lot of figures and tables, citations and indexterms, it takes more than five minutes to compile the book once. The full compilation (at least for times) takes close to half an hour. This I could stomach easily. However, I noticed that half of the compilation time it is not pdflatex running but kpsewhich.

I have organized the whole book and texlive on a fast ssd. Texlive is updated regularly. Please can anybody tell me why kpsewhich takes over for an extended time? I was told earlier that it searches the entire computer. When this is the case how to tell kpsewhich to focus on the ssd? Are there any environnemental variables to add?

BTW Running PDFlatex directly and not latexmk does not involve kpsewhich.

Kind regards Bernhard

Edit:

Here the (partial) output when removing the -silent option from the latexmk command. Pdflatex stops after

Latexmk: Log file says output to 'VierteDeutscheEdition.pdf'

and kpsewhich takes over. Unfortunately there is no kpsewhich output. After kpsewhich is run the rest of the message appears. ...

Output written on VierteDeutscheEdition.pdf (581 pages, 170933099 bytes).

SyncTeX written on VierteDeutscheEdition.synctex.gz.

Transcript written on VierteDeutscheEdition.log.

Latexmk: Index file 'VierteDeutscheEdition.idx' was written

Latexmk: Found input bbl file 'VierteDeutscheEdition.bbl'

Latexmk: Log file says output to 'VierteDeutscheEdition.pdf'

Stop of pdflatex

Latexmk: List of undefined refs and citations:

Reference `sec:sugarmetabolism' on page 39 undefined on input line 39

Reference `sec:sugarmetabolism' on page 473 undefined on input line 8

Reference `sec:sugarmetabolism' on page 85 undefined on input line 68

Biber warning: [32485] Utils.pm:164> WARN - I didn't find a database entry for 'CSV+87' (section 5) Biber warning: [32486] Utils.pm:164

WARN - I didn't find a database entry for 'THH+13' (section 5)

Biber warning: [39095] Utils.pm:164> WARN - I didn't find a database entry for 'Loveland11a' (section 9) Biber warning: [44320] Utils.pm:164

WARN - month field 'Nov' in entry 'CWT+98' is not an integer - this will probably not sort properly. Biber warning: [44336] Utils.pm:164

WARN - month field 'Nov' in entry 'RYF+98' is not an integer - this will probably not sort properly. Biber warning: [44348] Utils.pm:164

WARN - month field 'Apr' in entry 'HLH08' is not an integer - this will probably not sort properly. Biber warning: [44357] Utils.pm:164

...

Latexmk: Found biber source file(s) [Literatur/EndokrinologieKunde.bib Literatur/FatMetabolism.bib Literatur/Leptin.bib VierteDeutscheEdition.bcf]

Latexmk: Summary of warnings: Latex failed to resolve 5 reference(s)

Latexmk: All targets (VierteDeutscheEdition.pdf) are up-to-date

  • 1
    by default it does not search the disk outside the current directory so you must have a non standard setting for TEXINPUTS I would guess. hard to say with just this information. – David Carlisle Jan 8 '17 at 11:35
  • Maybe you need to configure TeXStudio in order to run pdflatexdirectly? – Jan Jan 8 '17 at 11:35
  • 2
    I sent the questioner a special version of latexmk which lists the invocations of kpsewhich (which are otherwise not visible in the output). Then we can determine definitively whether the excess time is taken by an enormous number of invocations of kpsewhich or whether it is a lot of time taken by one or a few invocations. (For reference, with my own 600+ page book, which has 100s of source files, exactly one invocation of kpsewhich is made. That's to locate a bst file, and that invocation takes a fraction of a second.) – John Collins Jan 11 '17 at 16:25
  • 2
    @daleif You've found the cause of the problem. Thanks. I see several things that can be greatly improved (beyond the improvements that you sent me). I should have an improved release of latexmk soon (i.e., within a day). – John Collins Jan 13 '17 at 21:03
  • 1
    A new version of latexmk, 4.51, is available for testing from personal.psu.edu/jcc8/latexmk/versions.html I think it solves the problem. Let me know the outcome. – John Collins Jan 14 '17 at 3:52
6

This behaviour is due to a bug in latexmk which is fixed in version 4.52b. Where possible, you should update your TeX distribution to install the current release. If this is not possible, for example if you want to change only this one package or where you do not have the privileges to update the entire distribution, you will need to install the package locally. This option should be considered a last resort, since for more complex packages there may be package dependencies that will make local installation more complicated and error-prone. (Note that the latexmk script is a single file, so the generally important warning about complex packages is not so critical here.)

(Note: In case of delays in propagation of new files to CTAN mirror sites and to the distributions, you can get the new version at http://personal.psu.edu/jcc8/latexmk/, if you have an urgent need for it.)

Diagnosis and solution of problem

The problem was triggered in a document that included lots of graphics files where the files were found by a non-trivial use of kpsearch to find them in a texmf tree. (Many documents have their graphics files in or under the document directory and don't need the searches; that's why the problem wasn't noticed earlier, even while latexmk's author was writing a 600+ page book.) Latexmk searches for dependency information in both the .fls file and in the .log file. Some relevant information in the .log file doesn't have the necessary path information, so latexmk invoked kpsewhich to find them. But it had the sometimes highly non-optimal behavior of invoking it once per file. At even 0.2s per call (on a recent iMac) that mounts up quickly. It's not that kpsewhich is actually searching the whole computer, but that it was being called far too many times. Strictly speaking, it's not a bug but a lack of an important optimization.

If latexmk's recorder mode is on, the necessary information is in the .fls file, so concerning graphics and other input files, the calls to kpsewhich are no longer made in the new version. If recorder mode is off, invocation of kpsewhich is needed, but the calls are now batched into one invocation, which still gives a dramatic improvement in run time in the problematic case. Remaining uses of kpsewhich are for finding .bst and .bib files (if needed); these are not batched (at least, not yet), but are normally very small in number, so they shouldn't be a problem.

(The new version also some ways of getting diagnostics about calls to kpsewhich.)

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.