# How do you store all your TeX files long-term?

Reading through As an expert, can you always use TeX for (nearly) any kind of document?, it seems most people use TeX for almost all their documents.

My question is this: how do you store these documents on your system? At the moment, I create a folder called document_name for each document, create document_name.tex inside it, and after typesetting to get out document_name.pdf I leave all the temp/aux files in the folder with the documents. So for every single document I have a folder with a dozen files in it.

It just seems a bit messy - so how do you store these documents on your system? Do you remove (automatically?) the ~10-20 typesetting files LaTeX generates? Or am I over-thinking this?

-
Don't see why you would not just delete the temporary files that are no longer needed and can be recreated when needed. Assuming of course that you are not using them for some other purpose (as in cross referencing between other docs). –  Peter Grill Jan 23 '12 at 20:16
I usually delete all temp files except for the most recent pdf or dvi output just in case I need to mail it or print. –  azetina Jan 23 '12 at 20:21
RE deleting temp files: I usually use a Makefile to build my TeX documents. I have a "clean" target that removes all the temporary files, and a "realclean" target that invokes the "clean" target and removes the generated PDF/PS/dvi file as well. –  cm2 Jan 23 '12 at 23:21
@PeterGrill: because it is not so straightforward to know which files are temporary files and which are not, especially if you use an environment like TexnicCenter –  Rabarberski Jan 26 '12 at 15:09

Each document in a different folder, is a good archiving. Though I usually remove auxiliary files, keeping the tex source and pdf output, for convenience. The dedicated document folder keeps your file system clean if you compile again.

I organize my documents in topic folders, first such as letters, articles, books, and below subfolders such as job, private etc.

If you would like to ensure that you can easily compile your stored documents later, you could use

• snapshot: lists the external dependencies of a LaTeX document
• bundledoc: bundles together all the files needed to build a LaTeX document, works with snapshot
• arlatex: a LaTeX-based archiving program; which takes the name of a master .tex file and the ancillary files it uses (such as .tex, .sty, .cls and .eps files). From these files, arlatex creates a single file that will recreate all the ancillary files when compiled with LaTeX
-
I also archive my .synctex.gz files as retaining the SyncTeX data is handy if you want to check how you did something! –  Joseph Wright Jan 23 '12 at 20:33
+1 for arlatex. I have not heard of that one before. –  Predrag Punosevac Jan 24 '12 at 4:38
arlatex is OK-ish, but the fact that it refuses to recreate files, that are anywhere where latex can find them is a bit meh. Gotta check if it works with xelatex. –  polemon Jan 25 '12 at 3:25
I also keep each folder under version control as a separate git repository. git clean makes it easy to nuke unneeded files and git push allows me to send the whole history to another computer for backup. –  Sharpie Jan 27 '12 at 18:11
The snapshot package doesn't seem to be fully appropriate when compiling with XeTeX/LuaTeX, since it won't list what fonts you have loaded to typeset your document with. –  Sverre Sep 25 '13 at 17:45

For portability purposes, as well as keeping things clean/tidy, I use embedfile to attach the source .tex (as well as other required sources) to the output .pdf. The attachment is compressed, which also saves some space. Then, once a project is complete, you can erase everything but the .pdf.

The requirement though is that you use pdflatex.

Here is a minimal example showing the output of compiling main.tex:

\documentclass{article}
\usepackage{graphicx}% http://ctan.org/pkg/graphicx
\usepackage{lipsum}% http://ctan.org/pkg/lipsum
\usepackage{embedfile}% http://ctan.org/pkg/embedfile
\usepackage{filecontents}% http://ctan.org/pkg/filecontents
\begin{filecontents*}{section1.tex}
\section{First section}\lipsum[1]
\end{filecontents*}
\begin{filecontents*}{section2.tex}
\section{Second section}\lipsum[2]
\end{filecontents*}
\begin{filecontents*}{section3.tex}
\section{Last section}\lipsum[3]
\begin{center}\includegraphics[width=\linewidth]{tiger}\end{center}
\end{filecontents*}
\begin{document}
% Input external files
\input{section1}
\input{section2}
\input{section3}

% Attach external files
\embedfile[desc={Main document}]{main.tex}
\embedfile[desc={First section}]{section1.tex}
\embedfile[desc={Second section}]{section2.tex}
\embedfile[desc={Last section}]{section3.tex}
\embedfile[desc={Rooooaaarrrr!}]{tiger.pdf}
\end{document}


The above output PDF view is from Adobe Acrobat. The PDF document now contains all relevant non-standard content in a single, portable format*.

desc={..} provides a description for the files in the attachment, while filespec={..} allows one to modify the attached file name. The embedfile package documentation describes all the possible key-value options.

* Of course, since main.tex uses filecontents to write section?.tex and therefore implicitly contains these files, they're embedded in the PDF in this example for completeness.

The PDF Toolkit is another option that can be used to attach files to PDFs using the attach_files key. From the PDFtk Server Man Page:

attach_files < attachment filenames | PROMPT > [ to_page < page number | PROMPT > ]


Packs arbitrary files into a PDF using PDF’s file attachment features. More than one attachment can be listed after attach_files. Attachments are added at the document level unless the optional to_page option is given, in which case the files are attached to the given page number (the first page is 1, the final page is end). For example:

pdftk in.pdf attach_files table1.html table2.html to_page 6 output out.pdf

-
This seems extremely useful. Can you post a small example? Also, for a complex document will it include all the other files that are inserted via \input or \include, or even \usepackage{} (for custom packages not in the standard distributions)? –  Peter Grill Jan 25 '12 at 6:01
@PeterGrill: Anything that is non-standard, including images, text files, etc. I'll add an MWE. –  Werner Jan 25 '12 at 6:05
I use xelatex, does it work equally well with it? –  polemon Jan 25 '12 at 15:07
This also works nicely with LuaTeX on MikTeX 2.9. +1 –  Alexander Jan 27 '12 at 19:27

I have my private version control repository (I am old, so I use CVS; many people now use subversion or git). There I put a directory for each project, a subdirectory for subprojects, etc. Under the directory are .tex files, data, etc., and Makefiles. So cvs checkout and make compiles all pertinent pdf files.

-

This is probably not a good answer, but I just have my tex files scattered all over the place, inside more or less logical directory structure, where each "project" has its own directory. So for example all files related to a particular class I teach are in the same directory. Since I just pack everything in a tarball and unpack it again when moving to a new computer or disk, some of this mess is now almost 20 years old. Every once a while I go through it and delete some aux, log and other such files, but that does not happen very often. Do I need to mention that locate is my friend?

-

Well what is your general oranization and back-up plan? TeX should not diverge much from your general practice.

Here is mine:

$ls Books Desktop Documents Music Pictures Programs Videos  TeX files as you can guess are exclusively in Documents folder. $ ls Documents
Cov_Let   Labels    Posters   Razno     Teaching  Workshops
Grants    Portfolio Powerdot  Research  Tenure


I use Make utility (BSD Make to be precise) and my own home brown Makefile to do TeX-ing. I always remove all auxiliary files and tend to keep only tex source files, images and final pdf versions for convenience. I tend to keep each of my current research project in the separate folder of the directory.

/home/predrag/Documents/Research/current


I use version control system CVS (I am an old guy) to keep the track of changes. So there are no duplicate file.

For teaching I have the following directory for the courses I teach this semester at Augusta State University.

/home/predrag/Documents/Teaching/AUG/spring2012


Can you guess what sub-folders do I have?

 $ls spring2012 math1111 csci3030  Here how the typical course folder looks $ ls csci3030
exams    homework roster   syllabus


I think that things at this point are self explanatory. I use CVS only in the in a few sub-folders of roster directory to keep the track of the class list changes and most importantly grades changes.

I think that you get the rest.

I also use Unison to synchronize

/home/predrag/Documents


on various machines (several home computers, 2 laptops, workstation at work and one of my super computers at work).

Even though I have Documents directory in sync on at least six different computers + 2 USB sticks at every given moment as my final backup solution I also do periodic tar+zip of the Documents directory and burn at least 2 DVDs one of which I keep at home one of which I keep at work.

On my todo list is to install DragonFly BSD on a new file server at work and my home file server and start using Hammer (Journaling File System). Right now I use FFS (Unix Fast File System) on everything except of course DVDs where I use UDF file system and super computer which runs RedHat (ext2 file system) due to the lack of proprietary driver for NVidia GPU Tesla for NetBSD (which I would otherwise use for super computing).

-
Nice!! I was about to comment that putting the year second in the directory names such as spring2012 is not a good idea but perhaps if you are teaching then you want to see all your spring courses together across the years? –  Peter Grill Jan 25 '12 at 6:08

I have headerfiles that I input with an \input{general.teh} statement. If have several headerfiles.

And yes, I use the extension .teh for TeX header files. This is what a small document would look like:

\documentclass[paper=a4, 11pt]{scrartcl}
\usepackage{lipsum}
\input{general.teh}
\title{title}
\begin{document}
\section{Lorem Ipsum}
\lipsum[1]
\end{document}


.teh files cannot be compiled on their own, and go after the \documentclass statement. Some header files react differently on the document class at hand:

%% --- page settings ---
\makeatletter
}{%
}{%
}{%
\usepackage[a4paper, left=2.5cm, right=2.5cm, top=2cm, bottom=3cm]{geometry}
}}}

\usecolortheme{albatross}
\setbeamercovered{transparent}
}{%
\pagestyle{plain}
\pagenumbering{arabic}
}
\makeatother


I use version control for some .tex files, but more often than not, I just use rsync on my TeX root to archive them over to an external hard drive. There, they get .tar.xz'd, for backup reasons. All TeX documents are stored under a common root (~/Documents/latex/) and in there are folders for various documents.

Now, when the document is quite large, like applications for work, including CV, etc. I use Makefiles to facilitate quick creation of PDFs. Those files also contain functions to "clean" the directory:

all: mappe.pdf

mappe.pdf: bewerbung.pdf lebenslauf.pdf

%.pdf: %.tex
-xelatex -interaction nonstopmode \$<
-rm -f *.log *.aux

.PHONY: clean
clean:
-rm -f *.pdf *.log *.aux


All I have to do to recreate the PDFs, is run make in the directory. When make fails, I can use the output of the console for debugging, just as I'd do with anything else, where a build process fails, such as compiling programs.

Using makefiles, is only advised, when the document is maturing, and all you do, is basically fill it with content and not tweaking with its fundamental structure, whatever that may be. You can still see the errors without problems when you change the documents, but when beginning a new document from scratch, I would advise against writing the Makefile first...

As Stefan Kottwitz pointed out, you could use arlatex, which works quite ok, with a compressor:

arlatex --document=xetest.tex gemeral.teh | xz > bundle-xetest.tex


But the functionality spectrum isn't really what I'm looking for right now. I tend to archive all files, just like that.

-
Why the -xelatex? Wouldn't you want the log or aux files to help you debug build problems if xelatex fails? –  sarnold Jan 24 '12 at 3:08
@sarnold Because I usually use the console for debugging. I simply scroll up and use the text search of my terminal emulator. It is not super optimal, but usually, I use Makefiles only, when the document is practically finished. I use the xelatex command manually when working with the individual .tex files. –  polemon Jan 24 '12 at 4:08
+1 for TeX header files. I have no idea how that has never occurred to me before. –  Predrag Punosevac Jan 24 '12 at 4:31
@polemon: ahhh, that makes sense. Thanks! –  sarnold Jan 24 '12 at 23:09
Why not make your .teh file a .sty file and load it with \usepackage? This way you don't need to put \makeatletter etc in it… –  Seamus Jan 25 '12 at 12:34

For letters and other one-shot documents: I tend to leave them in a "mail" directory, though this is probably suboptimal.

For documents that I need to work on for longer periods, or with multiple people, I use a revision control system: previously CVS and then SVN, now I tend to use GIT.

-