Why do lower case mathcal letters show up as random symbols?

amsfonts doesn't have lowercase caligraphic letters, so when you do $\mathcal{Hello World}$ you get some odd results. (Likewise for \mathbb I believe)

Now, what surprises me is not that there aren't lowercase mathcal letters, but that given this, the document compiles at all! What is going on that means that \mathcal{lowercase} is interpreted as being a string of random symbols? It seems what letter gets turned in to what symbol is constant, so this isn't just random: there's a reason why this happens as it does.

What is going on? Can some explain the details of what's happening beneath the surface that means that LaTeX interprets lowercase letters inside \mathcal as a variety of mathematical symbols?

[I don't understand enough to work out what tags would be relevant here...]

-

TeX was created long before character sets and fonts were standardized by Unicode and OpenType. Each mathematical font contains only 128 glyphs ("newer" versions of TeX allow 256 glyphs, but the Computer Modern math fonts use only 128. For Latin letters, the default font is cmmi (Computer Modern Math Italic) which contains uppercase and lowercase letters; the \mathcal command simply switches all letters to cmsy (Computer Modern Math Symbols), which happens to contain calligraphic uppercase letters at the correct positions, but no lowercase calligraphic letters; instead it has various operators and other symbols (thus the font name "Symbols") at the position where normally the lowercase letters reside. See the LaTeX Encoding Guide (texdoc encguide), section A.4 for the exact encoding tables. There have been attempts to standardize mathematical encodings, but they have never been successful (at least in the LaTeX world), and today only OpenType and Unicode matter, where such issues cannot arise because everything is completely standardized.

Side note: This behavior is unrelated to the AMS fonts; the AMS fonts don't contain any medium-weight calligraphic characters.

-

All fonts "live" in special positioning tables. You can view for example the Euler Fraktur font and the calligraphic font tables using the package fonttable and the following minimal example:

\documentclass[11pt]{article}
\usepackage{fonttable}
\usepackage{amsmath}
\usepackage{amsfonts}
\begin{document}
\fonttable{eufm10} %fraktur
\fonttable{eusm10} %calligrafic ?
$\mathfrak{A} \neq \mathcal{A}$

$\mathfrak{Hello World}$
$\mathcal{Hello World}$
\end{document}


If you use lowercase and there is no symbol in its place the computer will just pick whatever is at that memory location. Sometimes, font authors stick various other symbols in these slots and you will see what appear as random symbols. That is why $\mathcal{HELLO WORLD}$ will work but not $\mathcal{hello world}$.

-
Is there any reason why you marked live as code? –  Hendrik Vogt Jan 12 '11 at 13:17
@Hendrik Vogt: I suspect he tried to use quotation marks, but the grave accents turn it into inline code. –  Philipp Jan 12 '11 at 14:45
Do you want to emphasize the "live", or do you want to point out that "live" is not to be taken literally since fonts are not alive? In the latter case, writing it with quotation marks (as I did) is the Right Thing, I think. –  Hendrik Vogt Jan 12 '11 at 16:29