# \mathcal(O) and font size

I use the following mystic TeX so as to display the Landau's notations for big and small O.

There is one partial solution to improve the scriptscriptstyle so as to not use a small o : see the secund answer below. The problemen with \smallOBis is the wrong spacing, contrary to \smallO. How can I do this ?

% Sources :
%     1) http://forum.mathematex.net/latex-f6/bonnes-commandes-de-base-t12278.html
%     2) http://tex.stackexchange.com/questions/30944/mathcalo-and-font-size

\documentclass{article}
\usepackage{amsmath}
\usepackage{amsfonts}
\usepackage{amssymb}

\usepackage{graphicx}

\newcommand{\bigO}[1]{\ensuremath{\mathop{}\mathopen{}\mathcal{O}\mathopen{}\left(#1\right)}}

\newcommand\smallO[1]{
\mathchoice
{% \displaystyle
\ensuremath{\mathop{}\mathopen{}{\scriptstyle\mathcal{O}}\mathopen{}\left(#1\right)}
}
{% \textstyle
\ensuremath{\mathop{}\mathopen{}{\scriptstyle\mathcal{O}}\mathopen{}\left(#1\right)}
}
{% \scriptstyle
\ensuremath{\mathop{}\mathopen{}{\scriptscriptstyle\mathcal{O}}\mathopen{}\left(#1\right)}
}
{% \scriptscriptstyle
\ensuremath{\mathop{}\mathopen{}{o}\mathopen{}\left(#1\right)}
}
}

\newcommand\smallOBis[1]{
\mathchoice
{% \displaystyle
\scriptstyle\mathcal{O}\left(#1\right)
}
{% \textstyle
\scriptstyle\mathcal{O}\left(#1\right)
}
{% \scriptstyle
\scriptscriptstyle\mathcal{O}\left(#1\right)
}
{% \scriptscriptstyle
\scalebox{0.8}{$\scriptscriptstyle\mathcal{O}$}\left(#1\right)
}
}

\begin{document}

$\renewcommand{\arraystretch}{1.2}% \begin{array}{llll} \textbf{Style} & \verb!\bigO! & \verb!\smallO! & \verb!\smallOBis! \\ \hline \verb!\displaystyle! & \displaystyle \bigO{\frac{1}{n}} & \displaystyle 1 + \smallO{\frac{1}{n}} - 2 & \displaystyle 1 + \smallOBis{\frac{1}{n}} - 2 \\ \verb!\textstyle! & \textstyle \bigO{\frac{1}{n}} & \textstyle 1 + \smallO{\frac{1}{n}} - 2 & \textstyle 1 + \smallOBis{\frac{1}{n}} - 2 \\ \verb!\scriptstyle! & \scriptstyle \bigO{\frac{1}{n}} & \scriptstyle 1 + \smallO{\frac{1}{n}} - 2 & \scriptstyle 1 + \smallOBis{\frac{1}{n}} - 2 \\ \verb!\scriptscriptstyle! & \scriptscriptstyle \bigO{\frac{1}{n}} & \scriptscriptstyle 1 + \smallO{\frac{1}{n}} - 2 & \scriptscriptstyle 1 + \smallOBis{\frac{1}{n}} - 2 \end{array}$

\end{document}

-
It's hard to understand about what spacing problem you're referring to. – egreg Oct 9 '11 at 9:30

One problem here is the available font sizes at that small-a-size. Using a scaling factor supported by graphicx by means of \scalebox{<factor>}{<stuff>}:

\documentclass{article}
\usepackage{amsmath}% http://ctan.org/pkg/amsmath
%\usepackage{amsfonts}% http://ctan.org/pkg/amsfonts
%\usepackage{amssymb}% http://ctan.org/pkg/amssymb
\usepackage{graphicx}% http://ctan.org/pkg/graphicx

\newcommand{\bigO}[1]{\ensuremath{\mathop{}\mathopen{}\mathcal{O}\mathopen{}\left(#1\right)}}

\newcommand\smallO[1]{
\mathchoice
{% mode \displaystyle
\scriptstyle\mathcal{O}\left(#1\right)
}
{% mode \textstyle
\scriptstyle\mathcal{O}\left(#1\right)
}
{% mode \scriptstyle
\scriptscriptstyle\mathcal{O}\left(#1\right)
}
{% mode \scriptscriptstyle
\scalebox{0.8}{$\scriptscriptstyle\mathcal{O}$}\left(#1\right)
}
}

\begin{document}

$\renewcommand{\arraystretch}{1.2}% \begin{array}{lll} \textbf{Style} & \verb!\bigO! & \verb!\smallO! \\ \hline \verb!\displaystyle! & \displaystyle\bigO{n} & \displaystyle\smallO{n} \\ \verb!\textstyle! & \textstyle\bigO{n} & \textstyle\smallO{n} \\ \verb!\scriptstyle! & \scriptstyle\bigO{n} & \scriptstyle\smallO{n} \\ \verb!\scriptscriptstyle! & \scriptscriptstyle\bigO{n} & \scriptscriptstyle\smallO{n} \end{array}$

\end{document}


The choice was made to scale a \scriptscriptstyle\mathcal{O} by 80%, since the shape of \mathcal{O} at larger font sizes looks different from that of \scriptscriptstyle. I'm sure that playing around with larger size and smaller scale factors would provide a suitable solution.

Edit: An update to the definition of \smallOBis, as defined in @projetmbc's updated question:

\documentclass{article}
\usepackage{amsmath}% http://ctan.org/pkg/amsmath
\usepackage{graphicx}% http://ctan.org/pkg/graphicx

\newcommand{\bigO}[1]{\ensuremath{\mathop{}\mathopen{}\mathcal{O}\mathopen{}\left(#1\right)}}

\newcommand\smallO[1]{
\mathchoice
{% \displaystyle
\mathop{}\mathopen{}{\scriptstyle\mathcal{O}}\mathopen{}\left(#1\right)
}
{% \textstyle
\mathop{}\mathopen{}{\scriptstyle\mathcal{O}}\mathopen{}\left(#1\right)
}
{% \scriptstyle
\mathop{}\mathopen{}{\scriptscriptstyle\mathcal{O}}\mathopen{}\left(#1\right)
}
{% \scriptscriptstyle
\mathop{}\mathopen{}{o}\mathopen{}\left(#1\right)
}
}

\newcommand\smallOBis[1]{
\mathchoice
{% \displaystyle
\operatorname{\scriptstyle\mathcal{O}}\!\left(#1\right)
}
{% \textstyle
\operatorname{\scriptstyle\mathcal{O}}\!\left(#1\right)
}
{% \scriptstyle
\operatorname{\scriptscriptstyle\mathcal{O}}\left(#1\right)
}
{% \scriptscriptstyle
\operatorname{\scalebox{0.8}{$\scriptscriptstyle\mathcal{O}$}}\left(#1\right)
}
}

\begin{document}

$\renewcommand{\arraystretch}{1.2}% \begin{array}{llll} \textbf{Style} & \verb!\bigO! & \verb!\smallO! & \verb!\smallOBis! \\ \hline \verb!\displaystyle! & \displaystyle \bigO{\frac{1}{n}} & \displaystyle 1 + \smallO{\frac{1}{n}} - 2 & \displaystyle 1 + \smallOBis{\frac{1}{n}} - 2 \\ \verb!\textstyle! & \textstyle \bigO{\frac{1}{n}} & \textstyle 1 + \smallO{\frac{1}{n}} - 2 & \textstyle 1 + \smallOBis{\frac{1}{n}} - 2 \\ \verb!\scriptstyle! & \scriptstyle \bigO{\frac{1}{n}} & \scriptstyle 1 + \smallO{\frac{1}{n}} - 2 & \scriptstyle 1 + \smallOBis{\frac{1}{n}} - 2 \\ \verb!\scriptscriptstyle! & \scriptscriptstyle \bigO{\frac{1}{n}} & \scriptscriptstyle 1 + \smallO{\frac{1}{n}} - 2 & \scriptscriptstyle 1 + \smallOBis{\frac{1}{n}} - 2 \end{array}$
\end{document}


Two changes were made to \smallOBis:

• Declared the resized \mathcal{O} as an operator via \operatorname{..}, therefore correcting the spacing with other relational/binary operators on the left side; and
• Corrected for spacing between new operator and brackets on the right side with \!.
-
\ensuremath in the arguments of \mathchoice is really useless. :) – egreg Oct 8 '11 at 21:34
@egreg: ...riiiiight! – Werner Oct 8 '11 at 21:48
Your solution is only good for lonely big O and small O. See the change of my first post to look for wrong spacings with things like 1 + \smallO{\frac{1}{n}} - 2. – projetmbc Oct 9 '11 at 9:23
@projetmbc: See my updated answer. Corrected for the spacing using a combination of \operatorname{...} and \!. – Werner Oct 9 '11 at 14:59
Thanks a lot, that's great ! – projetmbc Oct 9 '11 at 18:43

This is a case that shows why Landau chose O and o and not other fancy alphabets: an uppercase O is quite easily distinguishable from a lowercase o.

So it would suffice

\newcommand{\bigO}[1]{O{\left(#1\right)}}
\newcommand{\smallO}[1]{o{\left(#1\right)}}


or, if one feels the necessity of spacing the "O" or "o" from what precedes it (a factor, for instance), use

\newcommand{\bigO}[1]{\mathop{}\!O{\left(#1\right)}}
\newcommand{\smallO}[1]{\mathop{}\!o{\left(#1\right)}}


No \ensuremath (it doesn't hurt, but it's better to segregate math), no other complicated maneuvers with \mathop and \mathopen.

In case \mathcal{O} is really needed, I wouldn't use it for the "small o", particularly if these objects must be used in subscripts or superscripts. Nevertheless, here is a simpler definition (requires graphicx):

\newcommand{\bigO}[1]{\mathcal{O}{\left(#1\right)}}
\newcommand\smallO[1]{
\mathchoice
{{\scriptstyle\mathcal{O}}}% \displaystyle
{{\scriptstyle\mathcal{O}}}% \textstyle
{{\scriptscriptstyle\mathcal{O}}}% \scriptstyle
{\scalebox{.7}{$\scriptscriptstyle\mathcal{O}$}}%\scriptscriptstyle
{\left(#1\right)}}


possibly with \mathop{}\! as before.

Here is a showcase; above the first way, below the "script O"

-