13

I know that similar ground has been covered before (e.g. Speeding up LaTeX compilation), but this question is more specific.

I want to create some reports programmatically, so I wrote a program that produces the necessary TeX file. The file contains hundreds of longtable tables (longtabu actually). The compilation time in not acceptable (more than a minute and the reports may get much bigger) and I am searching for ways of decreasing it.

Overally there are 4 types of tables. Each type has the same header line, the same number of rows and the same number of columns. The only thing that changes is the data within the tables. Is it possible to speed up the compilation time?

If it helps, I know the maximum width of each column (although obviously TeX doesn't) and the number of rows for each table type. Is this type of knowledge helpfull at all?

I also thought of creating different *.tex files for each table type, spawning different processes for their compilation and combining them with pdfpages. Has anyone used this technique, or is it going to be a waste of time?

I have already precompiled the preamble, but the gain was very small. The output has to be pdf.

edit

The reason why I do care about the performance is because the compilation is going to be done server-side.

@Yiannis The following is a mwe of one of the tables' types. The preamble is common in all the tables. I don't use computer modern but I don't think that font selection is relevant in this context. I don't really need the headers to repeat themselves on each page, although it would be nice. On second thought neither alignment to the decimal point is truly necessary, but if I didn't want the output to look nice, I wouldn't have used LaTeX on the first place.

In regard of plain TeX, could you provide some pointers? In the past I have only used LaTeX.

@Bruno Thank you very much! Your solution is very fast, but it doesn't work with dcolumn and most importantly the spacing is very bad. Try duplicating the table you provided and you will see that the \bottomrule of the first one and the \toprule of the second one have no space between them at all. 

\documentclass[a4paper,10pt,fleqn,final]{article}

\usepackage{xcolor}
\usepackage[a4paper,top=2cm,bottom=2cm,left=1.5cm,right=1.5cm]{geometry}
\usepackage{longtable, booktabs, multirow, tabu}
\usepackage{caption}
\usepackage{dcolumn}

\newcolumntype{d}[1]{D{.}{.}{#1}}   
\newcommand{\mc}[1]{\multicolumn{1}{c}{#1}}
\newcommand{\mr}[1]{\multirow{2}{*}{#1}}

\begin{document}

\begin{longtabu}{cd{3.2}cd{3.2}cd{3.2}cd{3.2}cd{3.2}l}
\caption*{Synopsis of P75 - 97-558} \\
\toprule
    \mc{\mr{LoadCase}} &  \multicolumn{2}{c}{$N$} & \multicolumn{2}{c}{$V_{in}$} & \multicolumn{2}{c}{$V_{out}$} & \multicolumn{2}{c}{$M_{in}$} & \multicolumn{2}{c}{$M_{out}$} \\
    \cmidrule(rl){2-3} \cmidrule(rl){4-5} \cmidrule(rl){6-7} \cmidrule(rl){8-9} \cmidrule(rl){10-11}
    &\mc{UF} & State & \mc{UF} & State & \mc{UF} & State & \mc{UF} & State & \mc{UF} & State \\
\midrule
\endhead
\bottomrule
\endlastfoot
\midrule \multicolumn{11}{r}{{Continued on next page}} \\ \bottomrule
\endfoot
21 & 0.80 & Pass & 0.32 & Pass & 0.04 & Pass & 0.05 & Pass & 0.15 & Pass \\
22 & 0.76 & Pass & 0.02 & Pass & 0.03 & Pass & 0.09 & Pass & 0.11 & Pass \\
23 & 0.61 & Pass & 0.32 & Pass & 0.03 & Pass & 0.06 & Pass & 0.10 & Pass \\
24 & 0.56 & Pass & 0.02 & Pass & 0.02 & Pass & 0.06 & Pass & 0.07 & Pass \\
25 & 0.59 & Pass & 0.22 & Pass & 0.03 & Pass & 0.02 & Pass & 0.09 & Pass \\
26 & 0.56 & Pass & 0.02 & Pass & 0.02 & Pass & 0.06 & Pass & 0.07 & Pass \\
27 & 0.58 & Pass & 0.16 & Pass & 0.03 & Pass & 0.00 & Pass & 0.08 & Pass \\
28 & 0.56 & Pass & 0.02 & Pass & 0.02 & Pass & 0.06 & Pass & 0.07 & Pass \\
29 & 0.58 & Pass & 0.14 & Pass & 0.02 & Pass & 0.01 & Pass & 0.08 & Pass \\
30 & 0.56 & Pass & 0.02 & Pass & 0.02 & Pass & 0.06 & Pass & 0.07 & Pass \\
31 & 0.87 & Pass & 0.51 & Pass & 0.06 & Pass & 0.31 & Pass & 0.01 & Pass \\
32 & 0.64 & Pass & 0.31 & Pass & 0.07 & Pass & 0.13 & Pass & 0.03 & Pass \\
33 & 0.66 & Pass & 0.04 & Pass & 0.02 & Pass & 0.15 & Pass & 0.14 & Pass \\
34 & 0.85 & Pass & 0.78 & Pass & 0.01 & Pass & 0.29 & Pass & 0.15 & Pass \\
35 & 54.29 & \bfseries \color{red}Fail & 1.41 & \bfseries \color{red}Fail & 0.03 & Pass & 0.77 & Pass & 0.04 & Pass \\
36 & 6.89 & \bfseries \color{red}Fail & 1.21 & \bfseries \color{red}Fail & 0.00 & Pass & 0.72 & Pass & 0.08 & Pass \\
37 & 3.66 & \bfseries \color{red}Fail & 1.44 & \bfseries \color{red}Fail & 0.05 & Pass & 0.71 & Pass & 0.08 & Pass \\
38 & -1.00 & \bfseries \color{red}Fail & 1.62 & \bfseries \color{red}Fail & 0.02 & Pass & 0.75 & Pass & 0.12 & Pass
\end{longtabu}
\end{document}
Improve this question
12
  • Can you post the structure of the table, using TeX and boxes will be the quickest rather than LaTeX and packages.
    – Yiannis Lazarides
    Jan 30, 2012 at 23:45
  • Compiling each table, and then putting them all together, for example using make, because make checks the modification date so only new tables will get latexed. I'm not sure if its helpful in your situation.
    – yo'
    Jan 30, 2012 at 23:58
  • dcolumn itself does a fair amount of processing to get the alignment that really you don't need to do in a generated table as you could add the padding directly. Your sample already show ".00" rather than ".0" so if all numbers are 2dp right alignment would give automatic alignment on the ".", if you want centered alignment you could pad each cell something like 123.45 or \?\?1.1\? where \? is \phantom{0}
    – David Carlisle
    Jan 31, 2012 at 10:03
  • @DavidCarlisle for some reason, I was under the impression that digits were proportionally sized like letters so I had to use something like dcolumn. But they aren't... and of course you are right, right alignment does result in automatic alignment on the decimal seperator. I will try it and see if there is an improvement. Thanks!
    – pmav99
    Jan 31, 2012 at 10:34
  • 1
    so if Bruno's tabbing is quicker you could use that and just force right alignment in each cell (with hfill or something) Hmm used LaTeX for 25 years and never used tabbing, but I suppose there's always a first time...
    – David Carlisle
    Jan 31, 2012 at 10:58

5 Answers 5

Reset to default
11

If you know the table widths in advance you can "seed" the data that LT writes to the aux file so that it gets the correct widths first time, that won't speed up each run but means that it doesn't take several runs for LT to converge. (Basically look at the format of the command Lt writes to the aux file, recording the column widths, and put that into the document preamble.)

It's possible that compilation speed is improved a bit if you increase LTchunksize, with modern TeX memory requirements you can probably increase that a lot, so the whole table is processed in one chunk.

If you really know all the widths, and don't need any fancy spanning column behaviour, there is always the option of not using the TeX alignment methods at all and just making each row be a row of fixed-width hboxes. that saves TeX the bother of saving all the data in unset boxes, and working out the column widths.

Of course the time taken depends rather on how complicated the cells are, if you got rid of all the table markup and just set each cell as a paragraph, that wouldn't give the layout you want but would give a limit on the achievable time.

Improve this answer
2
  • Perhaps using a variant of tabbing could help?
    – Bruno Le Floch
    Jan 31, 2012 at 0:19
  • @BrunoLeFloch: Or a list.
    – Martin Schröder
    Jan 31, 2012 at 10:17
6

There are many actions one can take to optimize compilation speed, one is to use TeX directly in a program such as:

... code
\bye

without loading a lot of overhead such as the pdfLaTeX macros. The solution below, however uses pdfLaTeX as one can still get satisfactory results with pdfLaTeX as well, by eliminating overhead and writing in files. This works well for computer generated data, where the format of the data and the output is known in advance. I used basic TeX commands to define macros and for processing. We build the rows one by one. The MWE produces an 11 page Table in about 6-7 seconds (it can get lowered by not loading hyphenation patterns etc., silencing warnings and the like). The MWE example is using a counter for the first cell to generate some data and keep track of how many lines the output is and to slow compilation slightly to simulate reading data from disk.

\documentclass{article}
\usepackage[a4paper,top=2cm,bottom=2cm,left=1.5cm,right=1.5cm]{geometry}
\newcounter{ctr}
\begin{document}
\def\boxit#1{\hbox to 1cm{#1}}

\long\gdef\row{\hbox{\leavevmode\par  \boxit{\thectr\stepcounter{ctr}}  \boxit{0.85}
\boxit{Pass} \boxit{0.78}
\boxit{Pass}
\boxit{0.01}  
\boxit{Pass}
\boxit{0.29}
\boxit{Pass}
\boxit{0.15}
\boxit{Pass}}\par}

\def\toprule{%
\hrule
\vskip5pt}

\def\bottomrule{\vskip5pt\hrule}

% save some typing
\long\def\many{\row\row\row\row\row\row\row\row\row\row\row\row\row\row
\row\row\row\row\row\row\row\row\row\row\row\row\row\row
\row\row\row\row\row\row\row\row\row\row\row\row\row\row
\row\row\row\row\row\row\row\row\row\row\row\row\row\row
\row\row\row\row\row\row\row\row\row\row\row\row\row\row
\row\row\row\row\row\row\row\row\row\row\row\row\row\row}

\long\def\manymore{\many\many\many\many\many\many\many
\many}
% the table
\toprule
\manymore
\bottomrule
\end{document}

I haven't included the header due to time limitations (it need to be stitched together like embroidery, piece by piece, but not difficult to do), but if you battle with it please post a comment and I will post a bit later. The header can also be build with a normal tabular environment as is not contributing heavily to compilation.

Improve this answer
4
  • 1
    this works great and also it is very easy to apply for computer generated data! But I think that you are mistaken... On my 4 year old laptop it compiles on 1.23 sec and if I enable batchmode in 0.21 sec!!! Are you sure that your timings are correct?
    – pmav99
    Jan 31, 2012 at 12:39
  • 1
    @pmav99 -- Glad you found it useful. I ran it through textworks and left synctex on:) If you want to check the compilation time through pdfLateX check this tex.stackexchange.com/posts/13207/edit
    – Yiannis Lazarides
    Jan 31, 2012 at 13:27
  • @Yiannis I would love to see that row header.
    – ipavlic
    Feb 15, 2012 at 14:43
  • 1
    @ipavlic I would simply use \begin{tabular}...\end{tabular and use the OP code with minor tweaks. It will not add to compilation time if you want to try, I uploaded the modification to as the code is long. github.com/yannisl/MWE/blob/master/speed-up-compilation.tex
    – Yiannis Lazarides
    Feb 15, 2012 at 15:05
5

Another solution, which I don't have time for to implement, is to add an extra option to your environment. The option depends on a global mode, which may be set to final or draft.

  • When the global mode is final all tables are fully processed, regardless of the extra option.
  • When the global mode is draft only the tables are processed that provide the option. This is useful for (the few) tables you're currently working on. All other table environments simply ignore all material inside the environment.

This solution is as good as you can get when the global mode is final. When the global mode is draft it should be very fast because LaTeX only has to typeset a few tables that are currently under development.

Improve this answer
1
  • Thank you. This would be an interesting addition to the tabular environments. Unfortunately, this is not what I need at the moment. I produce the tables programmatically, so when I am tweaking them, I write to the *.tex file only the relevant tables and not the rest. My problem presents itself , when I am compiling all the tables together.
    – pmav99
    Jan 31, 2012 at 8:52
4

If you don't need the headers to be repeated when the table is split between pages, you could use the following (booktabs is only loaded for its nice horizontal lines with \toprule, \midrule and \bottomrule).

\documentclass{article}
\makeatletter
\newenvironment{mytab}[1]
  {%
    \let \@acol \@tabacol
    \let \@classz \@tabclassz
    \let \@classiv \@tabclassiv
    \let \\\@tabularcr
    \@mkpream {#1}%
    \let \@sharp ##%
    \set@typeset@protect
    \everycr{\noalign{\penalty0}}% allow page break after each line
    \halign\bgroup\@arstrut\span\@preamble\tabskip\z@skip\cr
  }
  {\crcr\egroup}
\makeatother
\usepackage{booktabs}
\begin{document}

\hrule\vskip 400pt\hrule % to push the table to the page break.

\begin{mytab}{cc}
  \toprule
  Header & Header \\
  \midrule
  a&b\\
  a&b\\
  a&b\\
  a&b\\
  a&b\\
  a&b\\
  a&b\\
  a&b\\
  a&b\\
  a&b\\
  a&b\\
  a&b\\
  a&b\\
  a&b\\
  a&b\\
  a&b\\
  a&b\\
  a&b\\
  a&b\\
  a&b\\
  a&b\\
  a&b\\
  \bottomrule
\end{mytab}
\end{document}
Improve this answer
1
  • If he doesn't need repeated headers/footers and the rows all have a fixed known width, a list would probably be even faster.
    – Martin Schröder
    Jan 31, 2012 at 10:30
3

Here's some ideas based on David Carlisle's mentioned Plain \hbox stacking:

\newdimen\colwd \colwd=3em
\newbox\passbox \setbox\passbox\hbox to\colwd{Pass\hfil}
\newbox\failbox \setbox\failbox\hbox to\colwd{%
  \special{color push rgb 1 0 0}Fail\special{color pop}\hfil}
\def\Pass{\copy\passbox}
\def\Fail{\copy\failbox}
\def\M#1 {\hbox to2\colwd{\hfil #1\hfil}}
\def\C#1 {\hbox to\colwd{\hfil #1\hfil}}
\def\L#1 {\hbox to\colwd{#1\hfil}}
\def\D#1.#2 {\hbox to\colwd{\hfil\llap{$#1$}.\rlap{$#2$}\hskip1.5em\hfil}}
\def\line#1{\hbox to\hsize{#1}}% just to show what they look like
\def\leftline#1{\line{#1\hfil}}
\leftline{\L \llap{Load}Case \M$N$ \M$V_{in}$ \M$V_{out}$ \M$M_{in}$ \M$M_{out}$ }
\leftline{\L{} \L UF \L State \L UF \L State \L UF \L State \L UF \L State \L UF \L State }
\medskip
\leftline{\L21 \D0.80 \Pass\D0.32 \Pass\D0.04 \Pass\D0.05 \Pass\D0.15 \Pass}
\leftline{\L22 \D0.76 \Pass\D0.02 \Pass\D0.03 \Pass\D0.09 \Pass\D0.11 \Pass}
\leftline{\L23 \D0.61 \Pass\D0.32 \Pass\D0.03 \Pass\D0.06 \Pass\D0.10 \Pass}
\smallskip
\leftline{\L24 \D0.56 \Pass\D0.02 \Pass\D0.02 \Pass\D0.06 \Pass\D0.07 \Pass}
\leftline{\L25 \D0.59 \Pass\D0.22 \Pass\D0.03 \Pass\D0.02 \Pass\D0.09 \Pass}
\leftline{\L26 \D0.56 \Pass\D0.02 \Pass\D0.02 \Pass\D0.06 \Pass\D0.07 \Pass}
\smallskip
\leftline{\L27 \D0.58 \Pass\D0.16 \Pass\D0.03 \Pass\D0.00 \Pass\D0.08 \Pass}
\leftline{\L28 \D0.56 \Pass\D0.02 \Pass\D0.02 \Pass\D0.06 \Pass\D0.07 \Pass}
\leftline{\L29 \D0.58 \Pass\D0.14 \Pass\D0.02 \Pass\D0.01 \Pass\D0.08 \Pass}
\smallskip
\leftline{\L30 \D0.56 \Pass\D0.02 \Pass\D0.02 \Pass\D0.06 \Pass\D0.07 \Pass}
\leftline{\L31 \D0.87 \Pass\D0.51 \Pass\D0.06 \Pass\D0.31 \Pass\D0.01 \Pass}
\leftline{\L32 \D0.64 \Pass\D0.31 \Pass\D0.07 \Pass\D0.13 \Pass\D0.03 \Pass}
\smallskip
\leftline{\L33 \D0.66 \Pass\D0.04 \Pass\D0.02 \Pass\D0.15 \Pass\D0.14 \Pass}
\leftline{\L34 \D0.85 \Pass\D0.78 \Pass\D0.01 \Pass\D0.29 \Pass\D0.15 \Pass}
\leftline{\L35 \D54.29 \Fail\D1.41 \Fail\D0.03 \Pass\D0.77 \Pass\D0.04 \Pass}
\smallskip
\leftline{\L36 \D6.89 \Fail\D1.21 \Fail\D0.00 \Pass\D0.72 \Pass\D0.08 \Pass}
\leftline{\L37 \D3.66 \Fail\D1.44 \Fail\D0.05 \Pass\D0.71 \Pass\D0.08 \Pass}
\leftline{\L38 \D-1.00 \Fail\D1.62 \Fail\D0.02 \Pass\D0.75 \Pass\D0.12 \Pass}
\bye

enter image description here

I used XeTeX's color for \Fail (I couldn't find how it works in pdftex). I also used whitespace instead of rules because I find that way nicer. You could further save the "header" line in a box as well. I am a little skeptic if this way is really faster than \halign.

Improve this answer
5
  • It is surely more memory efficient since halign requires TeX to read the whole table in memory before outputting anything. Knuth mentions that somewhere in the TeXbook as a reason to use tabbing.
    – Bruno Le Floch
    Jan 31, 2012 at 13:06
  • @Bruno: seeing how tabbing is a LaTeX thing, I find it highly unlikely that Knuth would mention it in the TeXbook. But you're right, I have a vague memory of something along those lines being mentioned. Thing is, if memory serves, at least \+ uses \halign nevertheless.
    – morbusg
    Jan 31, 2012 at 13:18
  • You are right. I think tabbing is based on \tabalign (aka \+), though. Neither use \halign I believe. Instead, they require you to give an example line, then model each line on that initial one. That means lines are treated one at a time.
    – Bruno Le Floch
    Jan 31, 2012 at 21:42
  • @Bruno: I just checked it out: \tabalign calls \m@ketabbox, which in turn calls \ialign. I got to admit though that I don't quite follow what's happening in those macros...
    – morbusg
    Feb 1, 2012 at 23:03
  • Odd. You are right, @morbusg. I really need to study those macros someday. I guess the only true thing that remains in what I've said is that each line is processed individually, not the whole table at once (\cr is defined to end as roughly \crcr\egroup\egroup, ending the \halign). Memory-wise, it's more efficient. Speed-wise, it's probably irrelevant.
    – Bruno Le Floch
    Feb 2, 2012 at 3:41

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .