1

I am using data tool to create the database of questions, where I provided Tag to every question. All database files(50 files) are included in one master file while compiling.After compilation output is a pdf file which contains questions, sorted according to the TAG.But compilation took 28 hours to produce an output file of 300 pages. Is it possible that output file is a sorted tex file so that I don't need to include all database files every time?

2

It's much better to use an external tool to perform the sorting. The sorting commands provided by datatool are really only designed for small ASCII databases.

The datatooltk application is designed to preprocess data for use with datatool. It can either be run in batch mode from the command line or in GUI mode. It can import data from CSV files, ODS or XLS spreadsheets or from SQL databases (currently only mysql). It saves the data in a format that can easily be loaded with datatool's \DTLloaddbtex.

By way of an example, here's a document that just uses datatool to load and sort data, and then displays it:

\documentclass{article}

\usepackage[utf8]{inputenc}
\usepackage[T1]{fontenc}
\usepackage{longtable}
\usepackage{datatool}

\DTLloaddb{data}{country-codes.csv}
\DTLsort{name}{data}

\begin{document}

\DTLdisplaylongdb{data}

\end{document}

The country-codes.csv file contains over 200 country codes. The first few lines are:

code,name
ad,Andorra
ae,United Arab Emirates
af,Afghanistan
ag,Antigua and Barbuda

Since the CSV file contains UTF-8 characters, the test file needs to load inputenc (and fontenc). It also needs longtable in order to use \DTLdisplaylongdb. Since longtable is used, the build process requires two LaTeX calls, and the sorting is performed on each call.

I named this file test.tex and created a small bash script called buildtest:

#!/bin/sh

pdflatex test
pdflatex test

and then ran time buildtest which showed the build time:

real    0m31.654s
user    0m31.477s
sys     0m0.051s

I then modified test.tex to:

\documentclass{article}

\usepackage[utf8]{inputenc}
\usepackage[T1]{fontenc}
\usepackage{longtable}
\usepackage{datatool}

\DTLloaddbtex{\countrydata}{country-codes.dbtex}

\begin{document}

\DTLdisplaylongdb{\countrydata}

\end{document}

and changed buildtest to

#!/bin/sh

datatooltk --output country-codes.dbtex --csv country-codes.csv --sort name
pdflatex test
pdflatex test

Now only one sort is performed before pdflatex is run. The sorted data is stored in country-codes.dbtex. The build time is now

real    0m2.182s
user    0m1.202s
sys     0m0.126s

which is significantly faster. The datatooltk step only needs to be performed when the original data changes.

The other difference between using \DTLsort (or \dtlsort) and using datatooltk --sort can be seen in the location of Åland Islands, which starts with a UTF-8 character. In the second case (using datatooltk) it's placed at the end of the list. In the first case (using \DTLsort) it's placed at the start of the list.

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.