When writing an a in TeX, it is typeset as an upright "a" by default. When writing an a in math mode it is typeset in italics. Furthermore, in math mode white spaces are ignored and no empty lines are allowed.
What settings are changed internally when TeX switches to math mode? Any pointers where to find a more or less detailed explanation or an overview are welcome, too.
Math mode is very different from normal text mode, for a number of reasons: math typesetting requires positioning symbols in particular ways, choosing them among a wide range of fonts, spacing them according to rules that differ from the ones valid in text.
To give a couple of examples: the space around = in a+b=c should be wider than the space around the +, because a relation binds less than an addition; in sin x there should be a thin space after "sin" that shouldn't appear in sin(x+y).
Such rules are managed by TeX itself, because it would be too difficult for a normal (and also for an experienced) user to remember them all. Only in some circumstance one has to override some of the decisions made by TeX.
Let's see some of the peculiarities of math mode.
Spaces are ignored in making the output. (Of course a space after a control sequence still marks its end, but it's another question.) Typing $a+b$ or $a + b$ is perfectly equivalent.
Characters are interpreted in a special way; TeX maintains an array of "mathcodes" (look for \mathcode in the site) so that it's able to decide for each one which mathematical object it represents (relation, operation symbol, relation and so on), which font it's to be taken from and the position from the font.
Mathematical control sequences such as \sum or \alpha are defined in terms of \mathchar, so they have a similar mechanism for deciding how they have to be typeset.
Delimiters have special codes so they can grow when needed and requested.
Fractions and radicals start a special mode for their typesetting.
There are "math styles" that influence how a math object is typeset: \displaystyle, \textstyle, \scriptstyle and \scriptscriptstyle; they are generally chosen automatically, but the user can override the decision. The math style can influence the typesetting of some symbols (for instance \sum or \int).
A thorough description of math mode can be found in three chapters of the TeXbook and in a very long part of TeX by Topic, so this can't be the place to describe all the workings.
When a math formula has been processed, TeX passes it to an internal processor, which the user has no control of for being transformed into the final typesetting instructions. Note that LuaTeX can control also that part, but this is much beyond the scope of this answer.
The look of the glyph is purely a feature of the font, and TeX usually switches fonts when you go into math mode (although neither the text nor math font is built into the TeX engine, all font choices are essentially user settings). The question "what settings are changed" isn't really answerable, essentially everything is different in math mode. Internally (as explained in appendix G of the TeXBook math mode lists are eventually converted into "normal" horizontal lists before being shipped out to the dvi (or pdf) file, but that is an internal operation not accessible from within TeX macros.
