Importing database (created using datatool) to datatooltk

Question

I have created database (very large database) of questions using datatool as

\DTLnewdb{finitemath}
\long\def\newproblem#1#2#3#4#5#6#7#8#9{%
  \DTLnewrow{finitemath}%
  \DTLnewdbentry{finitemath}{seq}{#1}%
  \DTLnewdbentry{finitemath}{yr}{#2}%
  \DTLnewdbentry{finitemath}{sub}{#3}%
  \DTLnewdbentry{finitemath}{top}{#4}%
 \DTLnewdbentry{finitemath}{mar}{#5}%
\DTLnewdbentry{finitemath}{qus}{#6}%
\DTLnewdbentry{finitemath}{url}{#7}%
\DTLnewdbentry{finitemath}{exp}{#8}%
\DTLnewdbentry{finitemath}{ans}{#9}%
}

\newproblem{1}{2018}{DM}{comb}{1}{
\textbf{Which one of the following\begin{lstlisting}
 is a closed form expression for the generating function of\end{lstlisting}
the sequence $\{a_n\}$, where $a_n = 2n +3 \text{ for all } n=0, 1, 2, \dots$?}\\
$($A$)$ $\frac{3}{(1-x)^2}$\\
$($B$)$ $\frac{3x}{(1-x)^2}$\\
$($C$)$ $\frac{2-x}{(1-x)^2}$\\
$($D$)$ $\frac{3-x}{(1-x)^2}$\\
}{\url{http://gatefreaks.com/index.php?qa=9036&qa_1=dm-gate-2015-3-1}}{.}{Ans:d}

Database has 9 different columns.I have 10 such database files .I use master file to combine all these database and when i compile it output a sorted pdf file (acc to any of the field). But compilation takes hours. Is there any way to make it efficient.

I read somewhere that it can be done using datatooltk.But i am not able to figure out, how to import my database to datatooltk? Is there any other way of doing it ?

Answer 1

Make sure you have at least version 1.8 (2018-01-26) of datatooltk. You also need at least Java 7. (Java 8 is the current version.)

To install datatooltk, download datatooltk-installer.jar from CTAN and run it. If your operating system recognises the .jar extension as a Java application, you should be able to double-click or use "Open with..." otherwise you can install from the command line:

java -jar datatooltk-installer.jar

There's a GUI mode and a batch mode. The installer should put a link to the GUI version in the applications menu, although this may vary according to the operating system. In GUI mode use File -> Open and select the .tex file containing your problem definitions. With a bit of luck it should be able to parse your definition of \newproblem. If not try replacing

\long\def\newproblem#1#2#3#4#5#6#7#8#9{%

with

\newcommand{\newproblem}[9]{%

As long as there isn't anything too complicated in your .tex file, it should be able to create the database. (If it's a big file it may take a while to parse.) You can then save it to a .dbtex file using Save as.... Then you can load the .dbtex file in your document using \input or \DTLloaddbtex.

Alternatively, from the command line:

datatooltk --in inputfile.tex --output outputfile.dbtex

If you need to sort the data, it's more efficient to perform it in the same step. For example:

datatooltk --in inputfile.tex --sort yr --output outputfile.dbtex

When you open a .tex file that isn't in the .dbtex format, you'll get a message:

Quick load failed (Premature end of file. Failed to find '\newtoks\csname').
Trying again with TeX Parser Library.

This just means that datatooltk is switching to using the TeX parser library to interpret the file contents, which is slower than its default method of loading a .dbtex file.

Related: datatool performance.