In what applications doesn't LaTeX give you much?
If you have a simple document, with precise layout requirements, then TeX can be a more direct solution, because there are fewer things to 'turn off'. The convenient features of LaTeX -- the ones that make it massively more useful than TeX in most cases -- are implemented via sometimes rather arcane logic, which can get in the way of your desire for a box to go here, dammit!
For example, I remember once producing a set of auto-numbered tickets, eight to a sheet, for some event. They had (if I do say so myself) a rather elegantly sparse design. Using TeX probably wasn't the wisest technology choice for that, but doing it in LaTeX would have resulted in rather more hair being torn out.
On another occasion, I wanted to produce a short booklet of simple texts (that is, no maths, tables, figures, and so on). I had a very clear idea of my intended layout, and it wasn't planned to be generalisable. I got exactly what I wanted, reasonably quickly and unhackily, using XeTeX, in combination with some nice platform fonts.
The great advantage of LaTeX(2e) over TeX is that content and formatting are well separated. This allows for logical markup, which improves the organisation of the document and allows you to make global changes to the "look" without affecting the content.
Hence if you want to exert total typographical control over the document, go for TeX, but if the document is in any way structurally complex, go for LaTeX.