# Mapping Private Use Area characters in pdflatex?

Let's say I want to use the font Desconsolata.ttf from Oracc: The Open Richly Annotated Cuneiform Corpus - Unicode fonts for Oracc; and in particular, I want to use the Private Use Area characters:

... which fontforge shows as U+E025 and U+E026. I'm not concerned with correct font families etc here, I'd just want to pull out these glyphs without resorting to fontspec/Lua/XeLatex.

I have prepared a small MWE, which in the \write18 has a bash script that downloads and processes the font so pdflatex can read it (so needs at least one run of pdflatex -shell-escape). The compilation of it is successful (though note the interesting bug where "Desconsolata" cannot be used as a font family name, because it contains "at" as substring), in that font tables for both lmodern and Desconsolata are printed (click for full size):

Now, I'm aware that Latex can only support up to 255 glyphs in a font table; and it is those glyphs that are shown in the typeset font table.

I'm also aware that in principle I can solve this by opening Desconsolata.ttf in FontForge, and re-mapping the Private Use Area glyphs so that they end up in positions < 255 - and then exporting that as a new font, and then using that with pdflatex; obviously, I wouldn't want to modify an already existing font.

So I was wandering - is there a way to do a map, such that Latex will look up glyph nr. (say) 63 at U+E025 of the .ttf font? Then I could simply use \char at 63 in the Latex document, with the .ttf font loaded... I guess something like that may be possible, because I see in tex/latex/cjk/texinput/UTF8/UTF8.enc stuff like:

\CJK@namegdef{^^f0}##1##2##3{\@gobble\CJK@unicodexError
\CJK@altxchar{-2080}{##1}{##2}{##3}}


... but I simply cannot parse what this means in context. I understand .enc files go command by command, which either defines a char, or not; e.g. texmf-dist/fonts/enc/dvips/tex-gyre/q-ts1.enc has:

...
/dotaccent
/cedilla
/ogonek
/quotesinglbase.ts1
/.notdef
/.notdef
...


... but cjk's UTF8.enc seems to directly address UTF-8 byte sequences. (Note that I didn't really know what .enc to use, so I took T1-WGL4.enc just because I've seen it used in another example)..

Here is the code (note: probably best to do something like mkdir /tmp/mytest; cd /tmp/mytest before compiling this, due to the amount of files that will be generated):

\documentclass{article}

% use pdflatex -shell-escape test.tex to run these commands:
% (tested in bash shell, Linux)
% expect:   Warning: ttf2afm (file /tmp/aa/Desconsolata.ttf): glyph endash' not found ... ;
% note: cannot do bash globbing here, like: echo $(ls *.\string\{afm,map,tfm,fd,log,aux\string\}) ; %\def\tname{Desconsolata} % NOTE: cannot use Desconsolata as name here, because of f@tstripsize #1at#2pt->#1 -> will cut the name at 'at' in Desconsol'at'a; so: kpathsea: Running mktexmf Desconsol -> ! Font \f@ttestfont=Desconsol at 4.99947pt not loadable: Metric (TFM) file not found. ; so... just call it Desconsol: \def\tname{Desconsol} \immediate\write18{% rm -v *.afm *.map *.tfm *.fd *.log *.aux ; ls ; wget -nc http://oracc.museum.upenn.edu/downloads/Desconsolata-ttf.zip ; unzip -n Desconsolata-ttf.zip ; ENCFILE="T1-WGL4.enc" ; TNAME="\tname" ; ttf2afm -e "$ENCFILE" -o $PWD/$TNAME.afm $PWD/Desconsolata.ttf ; maplineorig=$(afm2tfm $PWD/$TNAME.afm -T "$ENCFILE") ; mapline="$TNAME $TNAME "'" T1Encoding ReEncodeFont " <'"$PWD/Desconsolata.ttf T1-WGL4.enc" ;
echo $maplineorig ; echo$mapline ;
echo "$mapline" >$TNAME.map ;
echo "\string\ProvidesFile{T1$TNAME.fd}[$TNAME Font name/desc/comment]\string" > T1$TNAME.fd ; echo >> T1$TNAME.fd ;
echo "\string\DeclareFontFamily{T1}{$TNAME}{}" >> T1$TNAME.fd ;
echo "\string\DeclareFontShape{T1}{$TNAME}{m}{n}{ <->$PWD/$TNAME}{}" >> T1$TNAME.fd ;
echo >> T1$TNAME.fd ; echo '\string\endinput' >> T1$TNAME.fd ;
}

\usepackage[utf8]{inputenc}
\usepackage[T1]{fontenc}
\usepackage{lmodern}
\usepackage{trace}

\pdfmapfile{=\tname.map}

\usepackage{fonttable}
\makeatletter
\def\myfontinfo{font: encoding \f@encoding, family: \f@family, series: \f@series, shape: \f@shape, size: \f@size, baselineskip: \f@baselineskip}
\makeatother

\begin{document}

\myfontinfo
\vspace{-2em}
%% \xfonttable{ encoding }{ family }{ series }{ shape }
\makeatletter
\xfonttable{\f@encoding}{\f@family}{\f@series}{\f@shape}
\makeatother

\clearpage

\footnotesize% just to make small enough to fit on one page
\fontencoding{T1}\fontfamily{\tname}\selectfont
\myfontinfo
\vspace{-2em}
\makeatletter
\xfonttable{\f@encoding}{\f@family}{\f@series}{\f@shape}
\makeatother

\end{document}

• Replace in (a local version of) T1-WGL4.enc one of the char names (e.g. /cedilla) by the char name you want to test (looks like /uniE025) and then look what happens. If you can see the glyph you will have to create a suitable tfm-file too. Btw: imho you must embed such fonts fully, so use << in the map-file. – Ulrike Fischer Feb 11 '15 at 11:32
• Many thanks, @UlrikeFischer - it worked, so I posted a working code below; if you feel like it, post an answer, and I'll eventually accept it. I tried with << in the map file: Desconsol Desconsol " T1Encoding ReEncodeFont " <</tmp/aa/Desconsolata.ttf /tmp/aa/T1-WGL4-D.enc - but this gives pdfTeX warning: pdflatex (file Desconsol.map): invalid entry for 'Desconsol': only subsetted TrueType font can be reencoded and mktexpk: perhaps Desconsol is missing from the map file. ... Font Desconsol at 480 not found Fatal error. Any ideas about this? Cheers! – sdaau Feb 11 '15 at 12:28
• I mixed up ttf fonts and otf fonts. – Ulrike Fischer Feb 11 '15 at 12:31
• Thanks @UlrikeFischer - good to know there is that difference, too; cheers! – sdaau Feb 11 '15 at 12:32

Thanks to @UlrikeFischer, this is now solved; the answer was:

Replace in (a local version of) T1-WGL4.enc one of the char names (e.g. /cedilla) by the char name you want to test (looks like /uniE025) and then look what happens. If you can see the glyph you will have to create a suitable tfm-file too.

That's exactly what the embedded bash code does now: it copies the system T1-WGL4.enc (found using kpsewhich), and uses sed to do an in-place replacement -- so instead of the old line in T1-WGL4.enc:

% 0x00
/grave /acute /circumflex /tilde
/dieresis /hungarumlaut /ring /caron
/breve /macron /dotaccent /cedilla
...


... now there is the new line in the local T1-WGL4-D.enc:

  /breve /macron /dotaccent /uniE025
...


Notably, this is the 12th definition, and so it shows as glyph 11 (=0x0B hex):

... and this image can be compared to the table in the OP to see the difference from the original font encoding. Also, if the bash script is re-run via pdflatex -shell-escape, the .tfm file will be reconstructed too, so the inclusion of the glyph in-between normal font should be visible immediately with this code.

\documentclass{article}

% use pdflatex -shell-escape test.tex to run these commands:
% (tested in bash shell, Linux)
% expect:   Warning: ttf2afm (file /tmp/aa/Desconsolata.ttf): glyph endash' not found ... ;
% note: cannot do bash globbing here, like: echo $(ls *.\string\{afm,map,tfm,fd,log,aux\string\}) ; %\def\tname{Desconsolata} % NOTE: cannot use Desconsolata as name here, because of f@tstripsize #1at#2pt->#1 -> will cut the name at 'at' in Desconsol'at'a; so: kpathsea: Running mktexmf Desconsol -> ! Font \f@ttestfont=Desconsol at 4.99947pt not loadable: Metric (TFM) file not found. ; so... just call it Desconsol: % ENCFILE="T1-WGL4.enc" ; \def\tname{Desconsol} \immediate\write18{% set -x ; rm -v *.afm *.map *.tfm *.fd *.log *.aux *.enc ; ls ; wget -nc http://oracc.museum.upenn.edu/downloads/Desconsolata-ttf.zip ; unzip -n Desconsolata-ttf.zip ; ENCFORG="T1-WGL4.enc" ; ENCFILE="T1-WGL4-D.enc" ; PENCFILE="$PWD/$ENCFILE" ; cp$(kpsewhich $ENCFORG)$PENCFILE ;
sed -i 's_/cedilla_/uniE025_g' $PENCFILE ; TNAME="\tname" ; ttf2afm -e "$PENCFILE" -o $PWD/$TNAME.afm $PWD/Desconsolata.ttf ; maplineorig=$(afm2tfm $PWD/$TNAME.afm -T "$PENCFILE") ; mapline="$TNAME $TNAME "'" T1Encoding ReEncodeFont " <'"$PWD/Desconsolata.ttf $PENCFILE" ; echo$maplineorig ; echo $mapline ; echo "$mapline" > $TNAME.map ; echo "\string\ProvidesFile{T1$TNAME.fd}[$TNAME Font name/desc/comment]" > T1$TNAME.fd ;
echo >> T1$TNAME.fd ; echo "\string\DeclareFontFamily{T1}{$TNAME}{}" >> T1$TNAME.fd ; echo "\string\DeclareFontShape{T1}{$TNAME}{m}{n}{ <-> $PWD/$TNAME}{}" >> T1$TNAME.fd ; echo >> T1$TNAME.fd ;
echo '\string\endinput' >> T1\$TNAME.fd ;
}

\usepackage[utf8]{inputenc}
\usepackage[T1]{fontenc}
\usepackage{lmodern}
\usepackage{trace}

\pdfmapfile{=\tname.map}

\usepackage{fonttable}
\makeatletter
\def\myfontinfo{font: encoding \f@encoding, family: \f@family, series: \f@series, shape: \f@shape, size: \f@size, baselineskip: \f@baselineskip}

\def\insertMyGlyphEZeroTwoFive{%
\let\old@ffamily\f@family%
\fontencoding{T1}\fontfamily{\tname}\selectfont%
\char"0B% glyph 11=0x0B (was /cedilla, now /uniE025)
\let\f@family\old@ffamily\selectfont%
}
\makeatother

\begin{document}

\myfontinfo
\vspace{-2em}
%% \xfonttable{ encoding }{ family }{ series }{ shape }
\makeatletter
\xfonttable{\f@encoding}{\f@family}{\f@series}{\f@shape}
\makeatother

\clearpage

Just testing normal font, and my glyph \insertMyGlyphEZeroTwoFive{} in between;\\
\footnotesize% just to make small enough to fit on one page
\fontencoding{T1}\fontfamily{\tname}\selectfont
\myfontinfo
\vspace{-2em}
\makeatletter
\xfonttable{\f@encoding}{\f@family}{\f@series}{\f@shape}
\makeatother

\end{document}