How do you store all your TeX files long-term?

Question

Reading through As an expert, can you always use TeX for (nearly) any kind of document?, it seems most people use TeX for almost all their documents.

My question is this: how do you store these documents on your system? At the moment, I create a folder called document_name for each document, create document_name.tex inside it, and after typesetting to get out document_name.pdf I leave all the temp/aux files in the folder with the documents. So for every single document I have a folder with a dozen files in it.

It just seems a bit messy - so how do you store these documents on your system? Do you remove (automatically?) the ~10-20 typesetting files LaTeX generates? Or am I over-thinking this?

Answer 1

Each document in a different folder, is a good archiving. Though I usually remove auxiliary files, keeping the tex source and pdf output, for convenience. The dedicated document folder keeps your file system clean if you compile again.

I organize my documents in topic folders, first such as letters, articles, books, and below subfolders such as job, private etc.

If you would like to ensure that you can easily compile your stored documents later, you could use

• snapshot: lists the external dependencies of a LaTeX document
• bundledoc: bundles together all the files needed to build a LaTeX document, works with snapshot
• arlatex: a LaTeX-based archiving program; which takes the name of a master .tex file and the ancillary files it uses (such as .tex, .sty, .cls and .eps files). From these files, arlatex creates a single file that will recreate all the ancillary files when compiled with LaTeX

Answer 2

For portability purposes, as well as keeping things clean/tidy, I use embedfile to attach the source .tex (as well as other required sources) to the output .pdf. The attachment is compressed, which also saves some space. Then, once a project is complete, you can erase everything but the .pdf.

The requirement though is that you use pdflatex.

Here is a minimal example showing the output of compiling main.tex:

\documentclass{article}
\usepackage{graphicx}% http://ctan.org/pkg/graphicx
\usepackage{lipsum}% http://ctan.org/pkg/lipsum
\usepackage{embedfile}% http://ctan.org/pkg/embedfile
\usepackage{filecontents}% http://ctan.org/pkg/filecontents
\begin{filecontents*}{section1.tex}
\section{First section}\lipsum[1]
\end{filecontents*}
\begin{filecontents*}{section2.tex}
\section{Second section}\lipsum[2]
\end{filecontents*}
\begin{filecontents*}{section3.tex}
\section{Last section}\lipsum[3]
\begin{center}\includegraphics[width=\linewidth]{tiger}\end{center}
\end{filecontents*}
\begin{document}
% Input external files
\input{section1}
\input{section2}
\input{section3}

% Attach external files
\embedfile[desc={Main document}]{main.tex}
\embedfile[desc={First section}]{section1.tex}
\embedfile[desc={Second section}]{section2.tex}
\embedfile[desc={Last section}]{section3.tex}
\embedfile[desc={Rooooaaarrrr!}]{tiger.pdf}
\end{document}

The above output PDF view is from Adobe Acrobat. The PDF document now contains all relevant non-standard content in a single, portable format^*.

desc={..} provides a description for the files in the attachment, while filespec={..} allows one to modify the attached file name. The embedfile package documentation describes all the possible key-value options.

_{^* Of course, since main.tex uses filecontents to write section?.tex and therefore implicitly contains these files, they're embedded in the PDF in this example for completeness.}

---

The PDF Toolkit is another option that can be used to attach files to PDFs using the attach_files key. From the PDFtk Server Man Page:

attach_files < attachment filenames | PROMPT > [ to_page < page number | PROMPT > ]

Packs arbitrary files into a PDF using PDF’s file attachment features. More than one attachment can be listed after attach_files. Attachments are added at the document level unless the optional to_page option is given, in which case the files are attached to the given page number (the first page is 1, the final page is end). For example:

pdftk in.pdf attach_files table1.html table2.html to_page 6 output out.pdf

Answer 3

I have my private version control repository (I am old, so I use CVS; many people now use subversion or git). There I put a directory for each project, a subdirectory for subprojects, etc. Under the directory are .tex files, data, etc., and Makefiles. So cvs checkout and make compiles all pertinent pdf files.

Answer 4

This is probably not a good answer, but I just have my tex files scattered all over the place, inside more or less logical directory structure, where each "project" has its own directory. So for example all files related to a particular class I teach are in the same directory. Since I just pack everything in a tarball and unpack it again when moving to a new computer or disk, some of this mess is now almost 20 years old. Every once a while I go through it and delete some aux, log and other such files, but that does not happen very often. Do I need to mention that locate is my friend?

Answer 5

Well what is your general oranization and back-up plan? TeX should not diverge much from your general practice.

Here is mine:

$ ls
Books     Desktop   Documents Music     Pictures  Programs  Videos

TeX files as you can guess are exclusively in Documents folder.

$ ls Documents
Cov_Let   Labels    Posters   Razno     Teaching  Workshops
Grants    Portfolio Powerdot  Research  Tenure

I use Make utility (BSD Make to be precise) and my own home brown Makefile to do TeX-ing. I always remove all auxiliary files and tend to keep only tex source files, images and final pdf versions for convenience. I tend to keep each of my current research project in the separate folder of the directory.

/home/predrag/Documents/Research/current

I use version control system CVS (I am an old guy) to keep the track of changes. So there are no duplicate file.

For teaching I have the following directory for the courses I teach this semester at Augusta State University.

/home/predrag/Documents/Teaching/AUG/spring2012

Can you guess what sub-folders do I have?

 $ ls spring2012
math1111 csci3030

Here how the typical course folder looks

$ ls csci3030
exams    homework roster   syllabus

I think that things at this point are self explanatory. I use CVS only in the in a few sub-folders of roster directory to keep the track of the class list changes and most importantly grades changes.

I think that you get the rest.

I also use Unison to synchronize

/home/predrag/Documents

on various machines (several home computers, 2 laptops, workstation at work and one of my super computers at work).

Even though I have Documents directory in sync on at least six different computers + 2 USB sticks at every given moment as my final backup solution I also do periodic tar+zip of the Documents directory and burn at least 2 DVDs one of which I keep at home one of which I keep at work.

On my todo list is to install DragonFly BSD on a new file server at work and my home file server and start using Hammer (Journaling File System). Right now I use FFS (Unix Fast File System) on everything except of course DVDs where I use UDF file system and super computer which runs RedHat (ext2 file system) due to the lack of proprietary driver for NVidia GPU Tesla for NetBSD (which I would otherwise use for super computing).

Answer 6

For letters and other one-shot documents: I tend to leave them in a "mail" directory, though this is probably suboptimal.

For documents that I need to work on for longer periods, or with multiple people, I use a revision control system: previously CVS and then SVN, now I tend to use GIT.

Answer 7

I have headerfiles that I input with an \input{general.teh} statement. If have several headerfiles.

And yes, I use the extension .teh for TeX header files. This is what a small document would look like:

\documentclass[paper=a4, 11pt]{scrartcl}
\usepackage{lipsum}
\input{general.teh}
\title{title}
\begin{document}
    \section{Lorem Ipsum}
    \lipsum[1]
\end{document}

.teh files cannot be compiled on their own, and go after the \documentclass statement. Some header files react differently on the document class at hand:

%% --- page settings ---
\makeatletter
\@ifclassloaded{g-brief}{%
}{%
\@ifclassloaded{g-brief2}{%
}{%
\@ifclassloaded{beamer}{%
}{%
    \usepackage[a4paper, left=2.5cm, right=2.5cm, top=2cm, bottom=3cm]{geometry}
}}}

\@ifclassloaded{beamer}{%
    \usetheme{Darmstadt}
    \usecolortheme{albatross}
    \setbeamercovered{transparent}
    \beamertemplatenavigationsymbolsempty
}{%
    \pagestyle{plain}
    \pagenumbering{arabic}
}
\makeatother

I use version control for some .tex files, but more often than not, I just use rsync on my TeX root to archive them over to an external hard drive. There, they get .tar.xz'd, for backup reasons. All TeX documents are stored under a common root (~/Documents/latex/) and in there are folders for various documents.

Now, when the document is quite large, like applications for work, including CV, etc. I use Makefiles to facilitate quick creation of PDFs. Those files also contain functions to "clean" the directory:

all: mappe.pdf

mappe.pdf: bewerbung.pdf lebenslauf.pdf

%.pdf: %.tex
    -xelatex -interaction nonstopmode $<
    -rm -f *.log *.aux

.PHONY: clean
clean:
    -rm -f *.pdf *.log *.aux

All I have to do to recreate the PDFs, is run make in the directory. When make fails, I can use the output of the console for debugging, just as I'd do with anything else, where a build process fails, such as compiling programs.

Using makefiles, is only advised, when the document is maturing, and all you do, is basically fill it with content and not tweaking with its fundamental structure, whatever that may be. You can still see the errors without problems when you change the documents, but when beginning a new document from scratch, I would advise against writing the Makefile first...

As Stefan Kottwitz pointed out, you could use arlatex, which works quite ok, with a compressor:

arlatex --document=xetest.tex gemeral.teh | xz > bundle-xetest.tex

But the functionality spectrum isn't really what I'm looking for right now. I tend to archive all files, just like that.