# Long List of Files Names with Paths

I need to keep a list of files (max of around 500) along with the path.

Based on the fact that each file name and path could be up to 100 characters, should I just save it in a comma separated list using \g@addto@macro, or should I use the datatool package, or something else?

My main concern is any limitations I might run into.

In case it matters, the list processing that I am contemplating is to simply iterate through each file via pgf's \foreach to

1. Count the types of files.
2. Perhaps keep create separate sub-lists based on the file attributes/contents.
3. Generate a PDF with links to the files.

## References

-

It is generally more efficient to arrange that the list is executable rather than use a comma separated list (which needs to be parsed), traditionally LaTeX uses \@elt for this but any macro will do, the idea basically is that instead of

\def\foo{aaa,bbb,ccc,ddd}
\foreach \tmp in \foo ...{\mymacro{\tmp}}


you do

\def\foo{\mymacro{aaa}\mymacro{bbb}\mymacro{ccc}\mymacro{ddd}}


so it takes a little bit more storage space, but to execute it you just do

\foo


and to get different processing you locally define \mymacro so

\count@=0
\foo


will leave a count of the number of items in \count@

-
Hmmm. Another case where I really should change my thinking to the TeX way. :-) –  Peter Grill May 3 at 0:24

(David can obviously type faster than me! But then, he had less to type.)

Here's a possible method using datatool:

\documentclass{article}

\usepackage{etoolbox}
\usepackage{datatool}
\usepackage{hyperref}

\DTLnewdb{files}

\newcommand*{\extlist}{}

\makeatletter
\newcommand*{\getfileext}[2]{%
% Get the file extension
\def\thisfile{#2}%
\filename@parse\thisfile
\let#1\filename@ext
}

\getfileext{\thisfileext}{#1}%
% Has a count register been defined?
\ifcsundef{\thisfileext @count}%
{%
\expandafter\global
\expandafter\newcount\csname \thisfileext @count\endcsname
}%
{}%
\expandafter\global
\DTLnewrow{files}%
\dtlgetrow{files}{\DTLrowcount{files}}%
\dtlappendentrytocurrentrow{File}{#1}%
\dtlappendentrytocurrentrow{Ext}{\thisfileext}%
\@onelevel@sanitize\thisfile
\dtlrecombine
}
\makeatother

\begin{document}

File types:

\renewcommand{\do}[1]{Extension: #1.
Count: \number\csname#1@count\endcsname\par}
\dolistloop{\extlist}

File: \File\par
Ext: \Ext\par
}

\end{document}


This produces:

Notes:

1. I sanitized the path name using \@onelevel@sanitize for use in \url, but this probably isn't needed, unless you have funny characters in your file name, but that might cause a problem for the database as well, so it's probably overkill.
2. If you want to add a row to a database, it's faster to use the \dtlgetrow method than using \DTLnewdbentry. (Since you have 500 entries, you may prefer the faster less tidy option that the slower neater option.)
-

I made a file list of /usr/local/texlive/2013/texmf-dist/tex/latex/base, the first lines of which look like

/usr/local/texlive/2013/texmf-dist/tex/latex/base/alltt.sty
/usr/local/texlive/2013/texmf-dist/tex/latex/base/ansinew.def
/usr/local/texlive/2013/texmf-dist/tex/latex/base/applemac.def
/usr/local/texlive/2013/texmf-dist/tex/latex/base/article.cls
/usr/local/texlive/2013/texmf-dist/tex/latex/base/article.sty
/usr/local/texlive/2013/texmf-dist/tex/latex/base/ascii.def
/usr/local/texlive/2013/texmf-dist/tex/latex/base/bezier.sty


I saved the list in the file filelist.txt. Then I prepared this document:

\documentclass{article}

\usepackage{xparse}

\ExplSyntaxOn

\NewDocumentCommand{\MakeFileList}{mm}
{%1 = list name, #2 = file name
\grill_make_file_list:nn { #1 } { #2 }
}

\NewDocumentCommand{\CountItemsInList}{ O{} m }
{
\grill_count_items_in_list:nn { #1 } { #2 }
}

\ior_new:N \l_grill_file_list_stream
\seq_new:N \l__grill_file_list_temp_seq

\cs_new_protected:Npn \grill_make_file_list:nn #1 #2
{
\seq_new:c { g_grill_file_list_#1_seq }
\ior_open:Nn \l_grill_file_list_stream { #2 }
\ior_map_inline:Nn \l_grill_file_list_stream
{
\seq_gput_right:cn { g_grill_file_list_#1_seq } { ##1 }
}
}

\cs_new_protected:Npn \grill_count_items_in_list:nn #1 #2
{
\seq_clear:N \l__grill_file_list_temp_seq
\seq_map_inline:cn { g_grill_file_list_#2_seq }
{
\tl_if_in:nnT { ##1 } { #1 }
{
\seq_put_right:Nn \l__grill_file_list_temp_seq { ##1 }
}
}
\seq_count:N \l__grill_file_list_temp_seq
}

\ExplSyntaxOff

\begin{document}

\MakeFileList{LaTeX}{filelist.txt}

There are \CountItemsInList{LaTeX} files in total.

There are \CountItemsInList[.tex]{LaTeX} \texttt{.tex} files.

There are \CountItemsInList[.def]{LaTeX} \texttt{.def} files.

There are \CountItemsInList[.ltx]{LaTeX} \texttt{.ltx} files.

There are \CountItemsInList[.sty]{LaTeX} \texttt{.sty} files.

\end{document}


There are of course other methods for building the list. A finer control over the file types could be obtained with regular expressions. The same method used for counting the items could be used for building new lists based on the file extension or for creating links.

Here's the result.

Just to prove the correctness of the result, here's a set of shell commands:

> ls /usr/local/texlive/2013/texmf-dist/tex/latex/base/ | wc -l
149
> ls /usr/local/texlive/2013/texmf-dist/tex/latex/base/*.tex | wc -l
12
> ls /usr/local/texlive/2013/texmf-dist/tex/latex/base/*.def | wc -l
32
> ls /usr/local/texlive/2013/texmf-dist/tex/latex/base/*.ltx | wc -l
6
> ls /usr/local/texlive/2013/texmf-dist/tex/latex/base/*.sty | wc -l
30

-