ConTeXt provides a macro \testfeatureonce to benchmark performance. The syntax is

\testfeatureonce{n}{...}

which runs the code in the second argument n times (n is assumed to be an integer) and stores the elapsed time (in seconds) in the macro \elapasedtime. The following diagnostic message is also printed on the terminal:

system > starting feature test (n=1) system > 1 feature tests done (1.353s)

Here is a minimal ConTeXt document showing it's usage:

\starttext

\testfeatureonce{1000}
    {\setbox0\hbox
        {\startMPcode draw (0,0) -- (1cm, 1cm); \stopMPcode}}

\elapsedtime
\stoptext

What is the functionally equivalent macro in LaTeX used for benchmarking performance?

up vote 9 down vote accepted

For all I know, there's no such macro in the LaTeX2e base (perhaps LaTeX3 has something equivalent?). But we can look up the definition of \testfeatureonce in the ConTeXt source code. The relevant definitions are in the syst-aux module.

That implementation makes use of the eTeX primitives \pdfresettimer and \pdfelapsedtime to reset an internal timer and get the elapsed time since the last reset, respectively. I didn't find a proper source for this, but it seems 65536 equals one second in the value returned by \pdfelapsedtime.

A reimplementation in pure LaTeX code might look like this:

\documentclass{article}
\usepackage{tikz}

\makeatletter
\let\resettimer=\pdfresettimer
\let\elapsedtime=\pdfelapsedtime

\newcount\c@syst@helpers@test@feature@n
\newcount\c@syst@helpers@test@feature@m

\newcommand\testfeature[2]{%
    \c@syst@helpers@test@feature@m=#1\relax
    \def\syst@helpers@test@feature@step{%
        \advance\c@syst@helpers@test@feature@n by 1\relax
        \ifnum\c@syst@helpers@test@feature@n>\c@syst@helpers@test@feature@m\else
            #2\expandafter\syst@helpers@test@feature@step
        \fi
    }%
    \retestfeature
}

\newcommand\retestfeature{
    \bgroup
    \ifcase\interactionmode \let\wait\relax \fi
    \resettimer
    \c@syst@helpers@test@feature@n=0\relax
    \syst@helpers@test@feature@step
    \wait
    \egroup
}

\newcommand\testfeatureonce[2]{%
    \begingroup
    \let\wait\relax
    \testfeature{#1}{#2}%
    \endgroup
}

\makeatother

\begin{document}

\testfeatureonce{1000}
    {\setbox0\hbox
        {\tikz{ \draw (0,0) -- (1cm, 1cm); }}}

\the\elapsedtime

\end{document}
  • Thanks! I had looked at that definition but didn't realize that etex provides the primitives \pdfresettimer and \pdfelapsedtime. – Aditya Oct 23 at 0:59
  • 1
    65536 (2^16) is the number of scaled points in a point (pt) and the number of "scaled seconds" in a second. TeX represents decimal numbers as "scaled integers", this this value. Nice though, I didn't think of looking ConTeXt's definition :) +1 – Phelype Oleinik Oct 23 at 3:07
  • @aditya They are pdfTeX primitives also available in pTeX and upTeX. I have an open request to add them to XeTeX. – Joseph Wright Oct 26 at 7:53

With pdfTeX you can simulate the behaviour using \pdfresettimer and \pdfelapsedtime. I used some expl3 wrappers to make the same syntax as the ConTeXt version.

\documentclass{article}
\usepackage{xparse}
\usepackage{tikz}

%% For LuaTeX
% \def\pdfresettimer{\directlua{pdfelapsedtimer_basetime = os.clock()}}
% \def\pdfelapsedtime{\directlua{tex.print((os.clock()-pdfelapsedtimer_basetime)*65536)}}
%%

\ExplSyntaxOn
\cs_set_eq:NN \__tfo_start_timer: \pdfresettimer
\cs_set_eq:NN \__tfo_elapsed_time: \pdfelapsedtime
\cs_new:Npn \__tfo_output:n #1
  {
    \iow_term:x
      {
        >~#1~feature~tests~done~
        (\fp_eval:n { round ( \__tfo_elapsed_time: / 65536 , 3 ) }s)
      }
  }
\cs_new:Npn \__tfo_run_n_times:nn #1 #2
  {
    \if_int_compare:w #1 > 0 \scan_stop:
      \exp_args:No \__tfo_run_n_times:nnw { \int_value:w \__int_eval:w #1 - 1 } { #2 }
    \fi:
  }
\cs_new:Npn \__tfo_run_n_times:nnw #1 #2 \fi:
  {
    \fi:
    #2
    \__tfo_run_n_times:nn { #1 } { #2 }
  }
\NewDocumentCommand\TestFeatureOnce
  { m m }
  {
    \__tfo_start_timer:
    \__tfo_run_n_times:nn { #1 } { #2 }
    \__tfo_output:n { #1 }
  }
\ExplSyntaxOff

\begin{document}

\TestFeatureOnce{10000}
    {\setbox0\hbox
        {\tikz \draw (0,0) -- (1cm, 1cm);}}

\end{document}

For LuaTeX I used the code from this post (which is also implemented in pdftexcmds). However I used a slightly modified version that does not use a \protected\def to allow the returned \pdfelapsedtime to be f-expanded by l3fp.

I don't know if this is possible in XeTeX without resorting to system commands...


Edit: Bug fix. A large number of tests (~5000) would exceed TeX's input stack size by piling up tons of \fi:s, so I added a helper macro to make sure everything is executed outside the \if...\fi so that an arbitrary number of runs is possible.


With jfbu's idea to convert scaled seconds, one can even do it in plain pdfTeX or LuaTeX:

\input tikz.tex

%% For LuaTeX
% \def\pdfresettimer{\directlua{pdfelapsedtimer_basetime = os.clock()}}
% \def\pdfelapsedtime{\directlua{tex.print((os.clock()-pdfelapsedtimer_basetime)*65536)}}
%%

\catcode`@=11
% Stolen from latex.ltx
\begingroup
  \catcode`P=12
  \catcode`T=12
  \lowercase{%
    \def\x{\def\rem@pt##1.##2PT{##1\ifnum##2>\z@.##2\fi}}}%
  \expandafter\endgroup\x
\def\strip@pt{\expandafter\rem@pt\the}%
%
\let\TFO@start@timer\pdfresettimer
\let\TFO@elapsed@time\pdfelapsedtime
\def\TFO@output#1{%
  \immediate\write17{%
    > #1 feature tests done (\strip@pt\dimexpr\TFO@elapsed@time sp\relax s)
  }%
}
\long\def\TFO@runNtimes@nn#1#2{%
  \ifnum#1>0\relax
    \expandafter\TFO@runNtimes@nnw\expandafter{\number\numexpr#1-1}{#2}%
  \fi
}
\long\def\TFO@runNtimes@nnw#1#2\fi{%
  \fi
  #2%
  \TFO@runNtimes@nn{#1}{#2}%
}
\long\def\TestFeatureOnce#1#2{%
  \TFO@start@timer
  \TFO@runNtimes@nn{#1}{#2}%
  \TFO@output{#1}%
}
\catcode`@=12

\TestFeatureOnce{1000}
    {\setbox0\hbox
        {\tikz \draw (0,0) -- (1cm, 1cm);}}

\bye
  • Thanks. This is useful and a good introduction to Expl syntax as well. – Aditya Oct 23 at 11:24
  • Glad you liked :) I used expl3 mainly to use l3fp to convert the scaled seconds to seconds. The code can be translated into plain LaTeX quite easily. P.S.: Fixed a bug in the code :P – Phelype Oleinik Oct 23 at 16:44
  • 2
    @PhelypeOleinik to convert you only need \the\dimexpr <number>sp\relax which will give in seconds (indicated by pt...). Precise enough if your benchmark test take of the order of 1/10th to a few seconds; – jfbu Oct 23 at 16:46

Code using the new l3benchmark package (released to CTAN 2018-10-26):

\documentclass{article}
\usepackage{l3benchmark}
\ExplSyntaxOn
\cs_new_protected:Npn \testfeatureonce #1#2
  {
    \benchmark_once:n
      { \prg_replicate:nn {#1} {#2} }
  }
\ExplSyntaxOff
\usepackage{tikz}

\begin{document}
\testfeatureonce{1000}
    {\setbox0\hbox
        {\begin{tikzpicture} draw (0,0) -- (1cm, 1cm); \end{tikzpicture}}}
\end{document}

The benchmark code can also auto-determine how many cycles to do:

\documentclass{article}
\usepackage{l3benchmark}
\ExplSyntaxOn
\cs_new_protected:Npn \testfeatureonce #1#2
  { \benchmark:n {#2} }
\ExplSyntaxOff
\usepackage{tikz}

\begin{document}
\testfeatureonce{1000}
    {\setbox0\hbox
        {\begin{tikzpicture} draw (0,0) -- (1cm, 1cm); \end{tikzpicture}}}
\end{document}

(The code itself is of course much the same as the other answers here.)

  • Thanks. What is the algorithm to determine the number of cycles? – Aditya Oct 23 at 22:10
  • 1
    @Aditya You set a target time (default: 1s). The system then starts by doing one run of the user code, and timing it. If that takes over half the target, we are done. If not, we run the user code 4 times, and see the timing. That keeps happening until we get to at least half of the target time. Then we divide the time required by the number of loops, to give an shortest time to execute the user code. – Joseph Wright Oct 24 at 6:27

A primitive benchmarking utility:

\documentclass{article}

\usepackage{xintkernel}

\makeatletter
\def\testfeatureonce#1#2{\pdfresettimer
    \romannumeral\xintreplicate{#1}{#2}%
    \edef\theelapsedtime{\strip@pt\dimexpr\pdfelapsedtime sp\relax s}%
    % \theelapsedtime % commented-out
}
\makeatother

\usepackage{tikz}

\begin{document}

\testfeatureonce{1000}
   {\setbox0\hbox
        {\tikz{ \draw (0,0) -- (1cm, 1cm); }}}

\typeout{Previous test took \theelapsedtime}

\end{document}

The result is stored (but after conversion to seconds) for being re-usable.

....
(/usr/local/texlive/2018/texmf-dist/tex/latex/latexconfig/epstopdf-sys.cfg))
Previous test took 0.92303s
[1{/usr/local/texlive/2018/texmf-var/fonts/map/pdftex/updmap/pdftex.map}]
....

The package xintkernel is very low weight package (provides also \xintUniformDeviate) which has nothing of the computations facilities of xint itself.

The \xintreplicate is basically the same code (plagiarized) originally in l3kernel.

For more advanced benchmarking see l3benchmark.

The above does not compute the overhead from the expansion of \xintreplicate (which here would be a tiny fraction of the tikz things). If you want 1000000 replications better to nest two \romannumeral\xintreplicate{1000}.

  • Thanks. Why does \xintreplicate need to be preferred by \romannumeral? – Aditya Oct 23 at 18:36
  • @Aditya well, in xint, most macros have a capitalized version \xintFoo and lower case version \xintfoo with \xintFoo expanding to \romannumeral0\xintfoo, allowing to nest macros and maintain complete expandability in two steps (using \def\xintBar#1#2{\xintFoo{...}{...}} would mean three steps.) In the case of \xintreplicate I did not define the user level \xintReplicate. And it uses \romannumeral because it inserts a \z@ when done to stop it, rather than most xint macros which trigger expansion by \romannumeral0 and stop it by inserting a space token at the end. – jfbu Oct 23 at 21:14
  • Thanks for the explanation – Aditya Oct 23 at 22:20

Your Answer

 

By clicking "Post Your Answer", you acknowledge that you have read our updated terms of service, privacy policy and cookie policy, and that your continued use of the website is subject to these policies.

Not the answer you're looking for? Browse other questions tagged or ask your own question.