I think you are wanting to know about iniTeX and the plain format, mainly. The TeX binary knows roughly 300 primitives: things which are actually defined in the binary. This includes some you probably know (such as
\def) and others you probably don't (such as
\output). An easy way to see if something is a primitive is to
\show it: you'll just get the name again.
If you start TeX in iniTeX mode (
tex -ini), that is all you get. The problem is that this is not very useful. There are no fonts loaded, there is no output routine (so nothing will print), and so on. The primitives are all you need to build a working typesetting system, but there is 'some assembly required'. What iniTeX is good for is creating a format: you can only dump TeX's memory into a format when in iniTeX mode. So this is what is used 'behind the scenes' by TeX distributions to produce the LaTeX format, the plain TeX format (see below) and so on.
This brings us to the plain TeX format, which is often referred to simply as 'plain'. This is a minimal set of macros, etc., that Knuth wrote to make the TeX typesetting engine produce something usable. It doesn't provide very much, but there is a lot more than just the 300-ish iniTeX primitives. For example, there is an output routine, so things actually print. Plain TeX defines a lot of useful basics, such as constants and methods to allocate variables. If you type
tex at the Terminal, you will be using this plain format (unless you specifically ask for iniTeX).
LaTeX2e uses most of the plain format as a base, and so you will find a source file 'l2plain.dtx' which copies much of plain TeX into LaTeX. There are some differences (for example, LaTeX's output routine is very different from plain TeX's). In that sense, plain TeX is a 'base'. If you look at the LaTeX3 work, things are rather different in implementation. However, the base requirements (set up variables, load fonts, do output, provide minimal macro support) remain the same. It's not really possible to drop parts of plain TeX and still do anything useful.
Both the TeX engine itself and the plain TeX format are maintained by Knuth. There are no new features added to either of these by Knuth, but he does fix bugs. The later are very rare indeed, so if you think you've found one make very sure! One of the reasons for projects such as pdfTeX, XeTeX and LuaTeX is to add new features to the engine, something that will not happen with TeX itself.
Of course, over time there have been other formats. Knuth's model (as seen with The TeXbook) was to have dedicated formats for dedicated purposes. Both LaTeX and ConTeXt have gone the other way: they are general formats that can be used for many things. Other specialised formats do exist (see CTAN for example), but these have never really caught on. It turns out to be easier to have more generalised formats and choose which bits to use than to have many formats and have to pick which one to use.