I first started commenting on Will's and Joseph's comments and answers, but my
reply may be too long for a comment. So, first of all, concerning the
withdrawal of restricted \write18 in TL2009, if I remember correctly there was
mainly three reasons:
There was specific issues on Windows, mainly related to the fact that
Windows picks binaries from the current directory first (even if
. is not in
PATH variable). Combined with TeX ability to write arbitrary files in
the current directory, this was a huge threat. The approach taken now is (a)
forbid writing files with "executable extensions":
com, etc. (taken from the standard environment variable
PATHEXT if set at runtime, or from a built-in list otherwise); (b) invoke executables with full path whenever possible.
As you rightly point out, the system is as weak as the weakest allowed executable. There's two things we want from an allowed executable: (a) don't allow execution of arbitrary commands and (b) respect TeX file I/O restrictions as implemented in TeX Live. Many executables in the initial default list (most notably bibtex and makeindex) didn't satisfy (b), now they do. More importantly, the script epstopdf (which was the primary motivation for restricted write18) didn't, and at some point was prone to shell injection (hence arbitrary command execution). Now we believe epstopdf is robust in this respect (more precisely, its newly-created restricted version repstopdf is).
A implementation mistake (more a typo than a design issue btw) was discovered lately on Unix, which allowed shell injection directly from TeX. While we could have fixed it immediately (hence delaying the release further for rebuild), at this point we really felt like the feature was not ready for prime-time and required more relaxed (as opposed to rushing for release) thinking and careful proofreading.
To sum up, the current approach is not new, it's basically the same as planned for 2009, only a lot of "details" have been fixed, but the devil is in the details. Beside the "historic" interest, I hope this explanation sheds some light on which issues were considered.
Now, security is not only about executing commands, it's also about file I/O. The "texhack" paper cited by Joseph deals only with writing or reading files, which is nothing new (but not completely unrelated to restricted \write18 either, see points 1 & 2b above). Basically, file reading may lead to information leak in some contexts (tex as a web service), while file writing may lead to data loss or arbitrary command execution. As pointed out in the paper, TeX Live provides some protection. By default, only file output is restricted to the current directory or TEXMFOUTPUT if set, (and writing dot-files on Unix and files with "executable extensions" on Windows are forbidden), but file input can easily be restricted in the same way. It is very important to notice that this protection is efficient only if you process untrusted files in a "safe" directory (a temporary one for each document), as opposed to your home.
Concerning documentation, a section was added in the TeX Live guide (section 1.4). It doesn't discuss the subject extensively, but at least sums up the mains issues and the precautions a user can take.