# How do I identify the encoding and family of a new math font?

In this question, sfranky asked to put a double accent over \bm{\phi}, produced by the bm package. Double accents are implemented in both the amsmath and accents packages, but those packages don't work together well with bm. What I found out myself: One can use \use@mathgroup to make it work.

Now in my answer to sfranky's question I proposed to use

\use@mathgroup{\M@OMS}{5}{\bm{\phi}}


to get a bold \phi that the accents package can handle. However, half of that was just educated guesses. OMS is the encoding, I think (and maybe it should be OML in this case), and 5 is the font family of the \bm{phi}. Altough it seems to work, surely this is not the Right Way of doing it.

Thus my question: Is there a way to automatically identify the encoding and family of the symbol I'm going to use?

-

It is possible to extract the family of a symbol by looking either at its \mathcode (this is for explicit characters like 1, a, etc.; for example, \the\mathcode1 gives 28721 which is "7031 in hex) or by looking at it's \meaning which begins with \mathchar (e.g. \meaning\phi gives \mathchar"11E). The second of the four hex digits is the math family (if there are less, the leading digits are zeros). For example, 1 has mathcode "7031 so it's from family 0 (operators), and \phi has mathcode "011E so it's from family 1 (letters).

To get the family from the symbol, you can thus proceed this way: you look at the \meaning of the symbol. The symbols which normally get accents (1, \phi, etc.) typically have a \meaning which is either \mathchar".... or the character 1 or the letter a. For the first type, you must remove the \mathchar from the \meaning and keep only the number. For the second type, you must check if the \meaning begins with the, look at the \mathcode and then convert it to hex (for example by using the binhex.tex package). In each case, you might need to pad the number with leading zeros to extract the second digit reliably.

To be able to say whether the meaning of the character begins with \mathchar or with the, there are various ways. Here, I used the same way as amsmath.sty does for its automatic \dots. As \meaning produces catcode 12 characters, you must produce a \, an m, a t, etc. with catcode 12 to test against. A way of doing this is to use an uppercase/lowercase trick (the principle is the same as in my answer to the question How to make a real backslash (escape) character?): you take catcode 12 characters (e.g. !, ? etc.) and make their uppercase versions be \, m, etc. and then uppercase your whole definition.

Here's a code showing how to do all this. I've commented it, so I hope it's understandable.

\documentclass{article}

% the code works with all font packages
%\usepackage{lmodern}
%\usepackage{fourier}
%\usepackage[utopia]{mathdesign}
%\usepackage[charter]{mathdesign}
%\usepackage[garamond]{mathdesign}
%\usepackage{txfonts}
%\usepackage{pxfonts}
%\usepackage{mathpazo}

\usepackage{amsmath}
\usepackage{bm,accents}
\input{binhex}% for hex conversion

%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
%%%%%%%%%%%%%%%%%%%% CODE TO DETECT MATH FAMILY %%%%%%%%%%%%%%%%%%%%%%%
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
%
\makeatletter
%
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
% Utility macros to pad zeros (e.g. transform 34 into 0034)
% The delimiter is \empty so that if it is expanded, it does nothing
\ifx#1\empty 0\fi\ifx#2\empty 0\fi\ifx#3\empty 0\fi\ifx\\#4\\0\fi#1#2#3#4%
}
%
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
% Macro extracting the second digit (e.g. 7163 -> 1)
\def\@extract@second@of@four#1#2#3#4\@nil{#2}%
%
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
% new boolean which will be used to test if a \meaning begins with \mathcha (e.g. \mahchar"11E)
\newif\if@begins@by@mathchar@
%
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
% uppercase trick to obtain characters with catcode 12
\begingroup
\uccode!=\\ % ! is \ with catcode 12
\uccode?=m  % ? is m with catcode 12
\uccode,=a  % , is a with catcode 12
\uccode.=t  % . is t with catcode 12
\uccode;=h  % ; is h with catcode 12
\uccode/=c  % / is c with catcode 12
\uccode|=r  % | is r with catcode 12
\uppercase{\endgroup
\def\@if@begins@by@mathchar@#1#2#3#4#5#6#7#8#9\@nil{%
\@begins@by@mathchar@false % initiate boolean to false
% test if #1#2#3#4#5#6#7#8 is !?,.;/;, i.e. \mathcha
\ifx !#1\ifx ?#2\ifx ,#3\ifx .#4\ifx ;#5\ifx /#6\ifx ;#7\ifx ,#8%
\@begins@by@mathchar@true % switch boolean to true
\fi\fi\fi\fi\fi\fi\fi\fi
\if@begins@by@mathchar@ % if boolean true
\expandafter\@firstoftwo
\else % if boolean false
\expandafter\@secondoftwo
\fi
}
% macro which strips prefix \mathchar
\gdef\@strip@mathchar@prefix!?,.;/;,|#1\@nil{#1}%
% end of uppercase trick
}
%
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
% new boolean which will be used to test if a \meaning begins with the (e.g. the character 1)
\newif\if@begins@by@the@
%
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
% uppercase trick to obtain characters with catcode 12
\begingroup
\uccode!=t % ! is t with catcode 12
\uccode?=h % ? is h with catcode 12
\uccode,=e % , is e with catcode 12
\uppercase{\endgroup
\def\@if@begins@by@the@#1#2#3#4\@nil{%
\@begins@by@the@false % initiate boolean to false
% test if #1#2#3 is !?, i.e. the
\ifx !#1\ifx ?#2\ifx ,#3%
\@begins@by@the@true % switch boolean to true
\fi\fi\fi
\if@begins@by@the@ % if boolean true
\expandafter\@firstoftwo
\else % if boolean false
\expandafter\@secondoftwo
\fi
}
}
%
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
% macro which finds the font family of a math glyph
% does not work for characters modified by amsmath (because of the
% \DOTSB before them) but as they are not normally accented, it
% shouldn't be a problem
\def\find@font@family#1{%
% #1 = glyph from which to extract font family
\edef\@the@mathcode@{-1}% dummy definition (-1 will give a 0 family)
% test if #1 begins with \mathchar
\expandafter\@if@begins@by@mathchar@\meaning#1\@nil
%   if so, store the value of \mathchar inside \@the@mathcode@
{\edef\@the@mathcode@{%
\number\expandafter\@strip@mathchar@prefix\meaning#1\@nil}}%
%   otherwise, do nothing
{}%
% test if #1 begins with the
\expandafter\@if@begins@by@the@\meaning#1\@nil
%   if so, store the mathcode inside \@the@mathcode@
{\edef\@the@mathcode@{\the\mathcode#1}}%
%   otherwise, do nothing
{}%
% convert the mathcode into hex with package binhex.tex
\edef\@the@mathcode@hex{\hex{\@the@mathcode@}}%
% convert the hex number to a 4-digit one
\expandafter{\@the@mathcode@hex}}%
% store the second digit (which is the math family) in \@the@math@family
\edef\@the@math@family{%
\expandafter\@extract@second@of@four\@the@mathcode@hex\@nil}%
}
%
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
%%%%%%%%%%%%%%%%%%%%%%%%% NEW \accbm MACRO %%%%%%%%%%%%%%%%%%%%%%%%%%%%
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
%
\newcommand{\accbm}[1]{%
% store the font family inside \@the@math@family
\find@font@family{#1}%
% assign counter \@tempcnta to \@the@math@family so that \bm@boldtable
% gives the right corresponding bold family offset
\@tempcnta=\@the@math@family
% select the right \fam and apply it to \bm
% \@the@math@family+\bm@boldtable is the number of the bold family
\use@mathgroup{}% not currently used by LaTeX so can be empty
{\number\numexpr\@the@math@family+\bm@boldtable}%
{\bm{#1}}%
}
%
\makeatother
%
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
%%%%%%%%%%%%%%%%%%%%%%%%%%% END OF CODE %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%

\begin{document}

\newcommand{\test}[1]{$\hat{\dot{\bm{#1}}} \hat{\dot{\accbm{#1}}} \bm{{\hat{\dot{#1}}}}$\par}

\test{\phi}
\test{A}
\test{1}
\test{\Gamma}
\test{+}

% should give 3 but gives 0 if amsmath is loaded
\makeatletter$\find@font@family{\sum}\@the@math@family$\makeatother

\end{document}


In the tests, I compare \hat{\dot{\bm{...}}}, \hat{\dot{\accbm{...}}}, and \bm{{\hat{\dot{...}}}} (the last puts the accents in bold, but works out of the box). Here's the result, from left to right, when the font is mathdesign/utopia:

A few explanations of the code.

1. About the encoding. The encoding information (e.g. \M@OMS, \M@OML, etc.) is not really used by LaTeX. The \meaning of these macros is normally \default@M . or \default@M \noaccents@ . where \default@M is empty and \noaccents@ is used by the math accent mechanism of amsmath to change between families 0 and 7 to avoid strange results when the accents are used inside a math alphabet (try $\mathcal{\hat{A}}$ with and without amsmath to see the effect).

Once you have the \fam number, you can extract the encoding in by looking at \the\textfont2 (which is typically \OMS/cmsy/m/n/10) and extracting the tokens between \ and the first /, which can be done thanks to a delimited macro:

\makeatletter
\def\@extract@encoding@aux#1#2/#3\@nil{#2}
\newcommand{\extractencoding}[1]{%
\expandafter\@extract@encoding@aux\detokenize\expandafter{\the\textfont#1}/\@nil
}
\makeatother
\extractencoding{0} \extractencoding{1} \extractencoding{2} \extractencoding{3} \extractencoding{4}


If the font family is not currently used, the encoding extracted in this way will be "nullfont".

2. About the boldtable macro. The \bm@boldtable macro holds the offsets between the families and their bold versions, stored in an \ifcase structure in the following way:

\ifcase \@tempcnta 9\relax \or 9\relax \or 9\relax \or 9\relax \or \z@ \or \z@ \or \z@ \or \z@ \or \z@ \else \z@ \fi .


This means that the offset is 9 for \fam0 to \fam3 and is zero for all the others. To access these numbers, you must have the current \fam number stored inside the count register \@tempcnta, which you do with

\@tempcnta=\@the@math@family


and then you get the corresponding bold \fam number with

\number\numexpr\@the@math@family+\bm@boldtable

-
Finally I found time to study your answer. It seems to work great (except for \mathcal A, unfortunately), and I've learned a lot from it. (What I don't understand is your use of \@tempcnta and the last line of the code with \sum.) I'll dare bumping your answer by improving the grammar a bit, so that it gets some more upvotes :-) You may want to make another edit and add a short explanation to the text why one doesn't need to identify the encoding. And maybe you want to post an answer to sfranky's question, pointing him here? –  Hendrik Vogt Feb 20 '11 at 7:31
Very nice answer! –  TH. Feb 20 '11 at 8:06
@Hendrik: the parser I used to extract the \mathcode is (very) incomplete, and so doesn't work with \mathcal or with some tokens redefined by AMSmath for the \dots mechanism (e.g. \sum becomes \DOTSB \sum@ \slimits@ so you must skip the \DOTSB part to get the right family). I'll try to improve this later on (probably in a few days). I've updated the answer to add more details on \@tempcnta and the encoding issue. Concerning sfranky's question, maybe the simplest would be to put a link here in your answer other there? –  Philippe Goutet Feb 26 '11 at 0:19
Thanks a lot for your additional explanations. So \@tempcnta is hidden in \bm@boldtable; now I get it. Concerning sfranky's question, I'll add a link to your answer here (since it was you who proposed that). –  Hendrik Vogt Mar 2 '11 at 14:02

I'm not aware of any pre-existing mechanism for identifying the encoding and family of a symbol.

The encguide package may offer some illumination. It uses a command such as the following to print the the set of encodings in each family:

\ftable{cmr10}{OT1}


In particular, it prints each character belonging to a family and encoding as follows:

\font\X=cmr10
\X{\char86} % "V"


A list of families and encodings are at the bottom of encguide.tex (starting around line 1298).

That being said, I'm not sure if you can extract the family and encoding from a given symbol name or character. I tried using the ifthen package to make a comparison (i.e. \X\ifthenelse{\equal{V}{\char86}}{Same}{Different}), but the comparison doesn't work as expected. If a comparison were possible, one could compare the symbol against every character of every family.

Rather than building this reverse map, it could be a sensible alternative for sfranky (and others with similar issues) to use unicode via XƎTEX or LuaTeX.

While not a solution, I hope this is helpful food for thought.

-
Even though I'm still hungry: thanks for the food. –  Hendrik Vogt Jan 21 '11 at 14:33

This is not really an answer to my question, but an explanation why it's the fault of the bm package that it doesn't identify the encoding and family. To give an example, \mathcal is defined in fontmath.ltx via

\DeclareSymbolFontAlphabet{\mathcal}{symbols}


In effect, this makes \mathcal expand to

\use@mathgroup\M@OMS\symsymbols


when used in math mode, i.e., both encoding (\M@OMS) and family (\symsymbols) are provided. This enables the amsmath and accents packages to place multiple accents correctly. Thus, the correct fix to the problem in question should be that eventually, \bm expands to the correct \use@mathgroup<encoding><family>.

-