In what applications doesn't LaTeX give you much?
If you have a simple document, with precise layout requirements, then TeX can be a more direct solution, because there are fewer things to 'turn off'. The convenient features of LaTeX -- the ones that make it massively more useful than TeX in most cases -- are implemented via sometimes rather arcane logic, which can get in the way of your desire for a box to go here, dammit!
For example, I remember once producing a set of auto-numbered tickets, eight to a sheet, for some event. They had (if I do say so myself) a rather elegantly sparse design. Using TeX probably wasn't the wisest technology choice for that, but doing it in LaTeX would have resulted in rather more hair being torn out.
On another occasion, I wanted to produce a short booklet of simple texts (that is, no maths, tables, figures, and so on). I had a very clear idea of my intended layout, and it wasn't planned to be generalisable. I got exactly what I wanted, reasonably quickly and unhackily, using XeTeX, in combination with some nice platform fonts.
TeX is also useful if you are generating code. To compile a piece of TeX, you don't need to have all these preamble stuff. You can simply have
$x^2$ \bye and it will compile.
The great advantage of LaTeX(2e) over TeX is that content and formatting are well separated. This allows for logical markup, which improves the organisation of the document and allows you to make global changes to the "look" without affecting the content.
Hence if you want to exert total typographical control over the document, go for TeX, but if the document is in any way structurally complex, go for LaTeX.
It's worth remebering that almost anything in TeX will be valid in LaTeX as well, so you can always do most of heavy lifting in LaTeX, and then use TeX to tweak things.
Also, I use the TeX
$ for in-line math rather than than LaTeX
Usually you want to use TeX directly when you don't want the typographic settings been done as it is done by LaTeX. If you want to use direct control of the kerning etc., you probably better off using directly TeX.