I think I found an answer (following everyone's valuable suggestions): below is what I understand of the information contained in various kinds of files, and the path from TeX source to rendered bitmap.
TeX file
We start with a TeX file, like (say) a hello.tex
that contains just:
hello
\bye
DVI file
Compiling the TeX file with tex hello.tex
produces hello.dvi
, which contains only the name of the font and instructions where to place the characters from the font. For example, this is what hello.dvi
contains (output of hexdump -v -C hello.dvi
):
00000000 f7 02 01 83 92 c0 1c 3b 00 00 00 00 03 e8 1b 20 |.......;....... |
00000010 54 65 58 20 6f 75 74 70 75 74 20 32 30 31 37 2e |TeX output 2017.|
00000020 30 33 2e 32 34 3a 31 32 35 35 8b 00 00 00 01 00 |03.24:1255......|
00000030 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 |................|
00000040 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 |................|
00000050 00 00 00 ff ff ff ff 8d 9f f2 00 00 8e a0 02 83 |................|
00000060 33 da 8d a0 fd 86 cc 26 8d 91 14 00 00 f3 00 4b |3......&.......K|
00000070 f1 60 79 00 0a 00 00 00 0a 00 00 00 05 63 6d 72 |.`y..........cmr|
00000080 31 30 ab 68 65 6c 6c 6f 8e 8e 9f 18 00 00 8d 92 |10.hello........|
00000090 00 e8 60 a3 31 8e 8c f8 00 00 00 2a 01 83 92 c0 |..`.1......*....|
000000a0 1c 3b 00 00 00 00 03 e8 02 9b 33 da 01 d5 c1 47 |.;........3....G|
000000b0 00 02 00 01 f3 00 4b f1 60 79 00 0a 00 00 00 0a |......K.`y......|
000000c0 00 00 00 05 63 6d 72 31 30 f9 00 00 00 97 02 df |....cmr10.......|
000000d0 df df df df |....|
A human-readable version of this can be got with dvitype
; the relevant part is:
109: fntdef1 0: cmr10
130: fntnum0 current font is cmr10
131: setchar104 h:=1310720+364090=1674810, hh:=106
132: setchar101 h:=1674810+291271=1966081, hh:=124
133: setchar108 h:=1966081+182045=2148126, hh:=136
134: setchar108 h:=2148126+182045=2330171, hh:=148
135: setchar111 h:=2330171+327681=2657852, hh:=169
I think these are instructions defining font 0 as "cmr10", to use font 0, and to set characters (104, 101, 108, 108, 111) (corresponding to "hello") at certain positions. The hh
values vary based on the -dpi
value passed to dvitype
(default 300), but the rest are fixed. (We can also use "dviasm hello.dvi" or "dv2dt hello.dvi" for human-readable output in other formats.) So as we can see, there is still not any font shape information in the dvi file. A DVI driver is what takes these instructions, looks up the font ("cmr10"), and puts the specified characters at the specified positions.
PS file with Type 1 fonts
Calling dvips hello.dvi
uses (on my system at least) the Type 1 (vector) fonts like texmf-dist/fonts/type1/public/amsfonts/cm/cmr10.pfb
. The comments in cmr10.pfb
say %Copyright: Copyright (c) 1997, 2009 American Mathematical Society
. I think this corresponds to this page on the AMS site about the Type 1 fonts, which mentions that the (outline version of the) fonts were produced by Blue Sky Research and Y&Y Inc. (See also README.bluesky in fonts/cm/ps-type1 directory, "Blue Sky Research and Computer Modern fonts" on Nelson Beebe's site, and Karel Píška's comparison (draft PDF) of these with another Type 1 version of Computer Modern, namely that of BaKoMa (by Basil K. Malyshev).)
There is some binary stuff in this file cmr10.pfb
, which you can turn into human-readable form with t1disasm
. That shows that, for example, the glyph for the letter e
is defined in cmr10.pfb
with code like:
/e {
28 444 hsbw
67 4 callsubr
84 252 rmoveto
6 149 84 25 34 0 rrcurveto
103 10 -135 -39 hvcurveto
closepath
-238 -21 rmoveto
279 hlineto
22 3 0 21 hvcurveto
99 -54 97 -125 vhcurveto
-116 -92 -103 -125 hvcurveto
68 4 callsubr
-134 105 -97 115 vhcurveto
122 45 111 19 hvcurveto
10 -8 2 -5 vhcurveto
-9 0 -2 -6 -2 -8 rrcurveto
-35 -103 -90 0 -10 0 rrcurveto
-50 0 -40 30 -23 37 rrcurveto
-30 48 0 66 0 36 rrcurveto
closepath
endchar
} ND
This is the kind of font that most users want and use. It's a vector font rather than a raster font, meaning that it specifies the "shape" of the character rather than the bitmap for it. The way it ultimately becomes pixels on your screen (or paper) is that when you open the PostScript (or PDF) file in an interpreter/viewer (like Ghostscript), that program (the viewer/renderer) takes the shape specified above, and tries to approximate that shape into the available number of pixels at your viewing zoom level. Thus the same shape description "works" at all zoom levels, meaning that the renderer recomputes the pixels at whatever zoom level you choose. It can also use some hinting information specified with the font. (I learned a bit about all this (font rasterization etc.) when writing this answer, about a situation that is an example of how this can be subtle.)
To get a bitmap out of a PostScript file like this, we can use Ghostscript to produce a PBM file which is basically a matrix of 0s and 1s: cat hello.ps | gs -sDEVICE=pbm -sOutputFile=hello.pbm -r600 -
. Note the "-r600" which specifies a resolution of 600 dpi: whatever resolution you specify, gs
will "render" (rasterize) the PostScript file to that resolution, producing the bitmap.
As I said, these fonts are what most users want (see this highly upvoted answer by Heiko Oberdiek, the author of the brilliant pkfix
), but here I want to see the bitmaps generated by Metafont (and positioned by TeX), not the ones generated by the viewer software, so moving on...
PS file with bitmap fonts
Instead of dvips hello.dvi
which ended up using Type 1 fonts, we can use the -V
option, documented as “Download non-resident PostScript fonts as bitmaps”. So calling dvips -V hello.dvi
produces hello.ps
, and the log says it's using bitmap fonts like texmf-var/fonts/pk/ljfour/public/cm/cmr10.600pk
(from a directory inside our home directory).
This is also where we can specify a different DPI and mode from modes.mf, like dvips -V -D100 -mode amiga foo.dvi
or dvips -V -D1200 -mode cgnszz foo.dvi
. (The two must match, or it prints a warning like mktexpk: Mismatched mode […] and resolution […]; ignoring mode
.) For each mode we specify, it either picks up the pk
file from our local directory, or, if one doens't exist, automatically invokes Metafont. For example, if we call dvips -V -D406 -mode agfafzz hello.dvi
(and these options are being used for the first time), the logs say:
kpathsea: Running mktexpk --mfmode agfafzz --bdpi 406 --mag 1+0/406 --dpi 406 cmr10
mktexpk: Running mf-nowin -progname=mf \mode:=agfafzz; mag:=1+0/406; nonstopmode; input cmr10
This is METAFONT, Version 2.7182818 (TeX Live 2015) (preloaded base=mf)
...
Font metrics written on cmr10.tfm.
Output written on cmr10.406gf (128 characters, 16756 bytes).
Transcript written on cmr10.log.
mktexpk: /Users/username/Library/texlive/2015/texmf-var/fonts/pk/agfafzz/public/cm/cmr10.406pk: successfully generated.
and this generated pk
file is what is used in the PS file. See this excellent answer by David Carlisle for a bit on gf
and pk
files, and on modes.mf
.
When such pk
files are used, the ps
file contains a section like (for our hello.dvi
above, which needs only five characters):
%DVIPSBitmapFont: Fa cmr10 10 5
/Fa 5 112 df<0001C0000003C0000007C000001FC00000FFC000FFFFC000FFFFC000FF
1FC000001FC000001FC000001FC000001FC000001FC000001FC000001FC000001FC00000
1FC000001FC000001FC000001FC000001FC000001FC000001FC000001FC000001FC00000
1FC000001FC000001FC000001FC000001FC000001FC000001FC000001FC000001FC00000
1FC000001FC000001FC000001FC000001FC000001FC000001FC000001FC000001FC00000
1FC000001FC000001FC000001FC000001FC000001FC000001FC000001FC000001FC00000
3FE0007FFFFFF07FFFFFF07FFFFFF01C3879B72A>49 D<0007F800001FFF00007C0FC001
F803E003F001F007E001F80FC000F81F80007C1F80007C3F00007E3F00003E7F00003E7F
00003F7E00003FFE00003FFE00003FFE00003FFFFFFFFFFFFFFFFFFE000000FE000000FE
000000FE000000FE0000007E0000007E0000007F0000007F0000003F0000033F8000031F
8000070FC0000607C0000E07E0001C01F0003800F80070007E03E0001FFF800003FC0020
277EA525>101 D<03F000000000FFF000000000FFF000000000FFF0000000000FF00000
000003F00000000003F00000000003F00000000003F00000000003F00000000003F00000
000003F00000000003F00000000003F00000000003F00000000003F00000000003F00000
000003F00000000003F00000000003F00000000003F00000000003F00FF0000003F03FFC
000003F0F03F000003F1C01F800003F3800FC00003F7000FC00003FE000FC00003FC0007
E00003FC0007E00003F80007E00003F80007E00003F80007E00003F00007E00003F00007
E00003F00007E00003F00007E00003F00007E00003F00007E00003F00007E00003F00007
E00003F00007E00003F00007E00003F00007E00003F00007E00003F00007E00003F00007
E00003F00007E00003F00007E00003F00007E00003F00007E00003F00007E00003F00007
E00003F00007E00007F8000FF000FFFFC1FFFF80FFFFC1FFFF80FFFFC1FFFF80293A7EB9
2E>104 D<03F000FFF000FFF000FFF0000FF00003F00003F00003F00003F00003F00003
F00003F00003F00003F00003F00003F00003F00003F00003F00003F00003F00003F00003
F00003F00003F00003F00003F00003F00003F00003F00003F00003F00003F00003F00003
F00003F00003F00003F00003F00003F00003F00003F00003F00003F00003F00003F00003
F00003F00003F00003F00003F00003F00003F00003F00007F800FFFFC0FFFFC0FFFFC012
3A7EB917>108 D<0003FE0000000FFF8000003E03E00000F800F80001F0007C0003E000
3E0007C0001F000F80000F801F80000FC01F000007C03F000007E03F000007E07E000003
F07E000003F07E000003F07E000003F0FE000003F8FE000003F8FE000003F8FE000003F8
FE000003F8FE000003F8FE000003F8FE000003F8FE000003F87E000003F07E000003F07F
000007F03F000007E03F000007E01F80000FC00F80000F800FC0001F8007E0003F0003F0
007E0000F800F800007E03F000001FFFC0000003FE000025277EA52A>111
D E
%EndDVIPSBitmapFont
And this changes with mode: for mode amiga
which is at only 100 dpi, there is less to the font:
%DVIPSBitmapFont: Fa cmr10 10 5
/Fa 5 112 df<20E02020202020207004097F8807>49 D<7088F88088700506808506>
101 D<C04040407844444444EE070A808908>104 D<C04040404040404040E0030A8089
04>108 D<7884848484780606808507>111 D E
%EndDVIPSBitmapFont
I haven't yet manged to reverse-engineer this information directly (how to get from <7088F88088700506808506>
to the glyph for e
). But we can use pktype
on the generated pk
file to see it in a human-readable form. This is the section for Character 101 (lowercase e
) from texmf-var/fonts/pk/amiga/public/cm/cmr10.100pk
:
635: Flag byte = 224 Character = 101 Packet length = 15
Dynamic packing variable = 14
TFM width = 466035 dx = 393216
Height = 6 Width = 5 X-offset = 0 Y-offset = 5
.***.
*...*
*****
*....
*...*
.***.
(Except at very low resolutions like the above, it doesn't print the pixel images like this: for the 600 DPI texmf-var/fonts/pk/ljfour/public/cm/cmr10.600pk
the corresponding output looks like
3304: Flag byte = 192 Character = 101 Packet length = 77
Dynamic packing variable = 12
TFM width = 466035 dx = 2424832
Height = 39 Width = 32 X-offset = -2 Y-offset = 37
(13)8(22)13(17)5(6)6(13)6(9)5(11)6(11)5(9)6(12)6(7)6(14)5(6)[1]6(16)5(4)6
(17)6(3)6(18)5(2)7(18)5(2)7(18)6(1)6(19)13[2](19)77[4](26)[1]6(26)[1]7(26)6
(22)2(2)7(21)2(3)6(20)3(4)6(19)2(6)5(18)3(6)6(16)3(9)5(14)3(11)5(12)3(13)6
(7)5(16)14(21)8(10)
instead.)
We can also call pktogf
on that pk
file, and then use gftype
on the gf
file. At maximum output, namely with the -images -mnemonics
options, gftype
has output like:
1160: beginning of char 101: 0<=m<=5 0<=n<=5
(initially n=5) paint (1)3
1168: newrow 0 (n=4) paint 1(3)1
1172: newrow 0 (n=3) paint 5
1174: newrow 0 (n=2) paint 1
1176: newrow 0 (n=1) paint 1(3)1
1180: newrow 1 (n=0) paint 3
1182: eoc
.<--This pixel's lower left corner is at (0,6) in METAFONT coordinates
***
* *
*****
*
* *
***
.<--This pixel's upper left corner is at (0,0) in METAFONT coordinates
which fully spells out the bitmap for each character (glyph).
(Not sure whether the strange usage of "lower left" and "upper left" in the output is a bug in gftype
(written by David Fuchs and Knuth), or an artefact/joke of some sort.)
Bitmap (e.g. PBM file)
I first learned of the PBM file format from this blog post. It is very convenient, and is a way of avoiding having to deal with every image format. From the PostScript file as above, Ghostscript can produce a PBM file as mentioned earlier:
cat hello.ps | gs -sDEVICE=pbm -sOutputFile=hello.pbm -r100 -
The resulting hello.pbm
file starts like this:
P1
# Image generated by GPL Ghostscript (device=pbm)
850 1100
0000000000000000000000000000000000000000000000000000000000000000
0000000000000000000000000000000000000000000000000000000000000000
0000000000000000000000000000000000000000000000000000000000000000
0000000000000000000000000000000000000000000000000000000000000000
0000000000000000000000000000000000000000000000000000000000000000
0000000000000000000000000000000000000000000000000000000000000000
0000000000000000000000000000000000000000000000000000000000000000
0000000000000000000000000000000000000000000000000000000000000000
0000000000000000000000000000000000000000000000000000000000000000
0000000000000000000000000000000000000000000000000000000000000000
0000000000000000000000000000000000000000000000000000000000000000
0000000000000000000000000000000000000000000000000000000000000000
0000000000000000000000000000000000000000000000000000000000000000
000000000000000000
0000000000000000000000000000000000000000000000000000000000000000
0000000000000000000000000000000000000000000000000000000000000000
0000000000000000000000000000000000000000000000000000000000000000
0000000000000000000000000000000000000000000000000000000000000000
0000000000000000000000000000000000000000000000000000000000000000
0000000000000000000000000000000000000000000000000000000000000000
0000000000000000000000000000000000000000000000000000000000000000
0000000000000000000000000000000000000000000000000000000000000000
0000000000000000000000000000000000000000000000000000000000000000
0000000000000000000000000000000000000000000000000000000000000000
0000000000000000000000000000000000000000000000000000000000000000
0000000000000000000000000000000000000000000000000000000000000000
0000000000000000000000000000000000000000000000000000000000000000
000000000000000000
0000000000000000000000000000000000000000000000000000000000000000
0000000000000000000000000000000000000000000000000000000000000000
0000000000000000000000000000000000000000000000000000000000000000
0000000000000000000000000000000000000000000000000000000000000000
and so on. Each "block" in the above (a few lines of length 64 each, plus a bit more on another line) has 850 digits (0 or 1). There are 1100 such blocks. (As the paper size is 8.5 inches x 11 inches, and we've specified 100 dots per inch, the whole image is 850 dots x 1100 dots.) After unwrapping those lines and zooming in to the area which has the bits corresponding to "hello", we get the following bitmap for that region:
00000000000000000000000000000000000000000000000000000000000
00000000000000000000000000000000000000000000000000000000000
00000000000000000000000000000000000000000000000000000000000
00000000000000110000000000001100110000000000000000000000000
00000000000000010000000000000100010000000000000000000000000
00000000000000010000000000000100010000000000000000000000000
00000000000000010000000000000100010000000000000000000000000
00000000000000011110000111000100010001111000000000000000000
00000000000000010001001000100100010010000100000000000000000
00000000000000010001001111100100010010000100000000000000000
00000000000000010001001000000100010010000100000000000000000
00000000000000010001001000100100010010000100000000000000000
00000000000000111011100111001110111001111000000000000000000
00000000000000000000000000000000000000000000000000000000000
00000000000000000000000000000000000000000000000000000000000
00000000000000000000000000000000000000000000000000000000000
(If that doesn't say "hello" to you, try moving a good distance from your screen. Of course, using those bits to control which pixels are on and off is the "real" way of visualzing them.)
Anyway, it appears that this way we can get a bitmap of the typeset page, at whatever resolution / Metafont mode.
For what it's worth, the following is the bitmap of the same region that we get from the PostScript file that had the vector (Type 1) fonts, at the same resolution (100 dpi):
00000000000000000000000000000000000000000000000000000000000
00000000000000000000000000000000000000000000000000000000000
00000000000000000000000000000000000000000000000000000000000
00000000000000111000000000011101110000000000000000000000000
00000000000000011000000000001100110000000000000000000000000
00000000000000011000000000001100110000000000000000000000000
00000000000000011000000110001100110001100000000000000000000
00000000000000011111001101001100110011011000000000000000000
00000000000000011001011001101100110110001100000000000000000
00000000000000011001011111101100110110001100000000000000000
00000000000000011001011000001100110110001100000000000000000
00000000000000011001011000101100110110001100000000000000000
00000000000000111111101101111111111011011000000000000000000
00000000000000000000000000000000000000000000000000000000000
00000000000000000000000000000000000000000000000000000000000
00000000000000000000000000000000000000000000000000000000000
As you can see, it's considerably darker. Of course, it's meaningless to compare rasterizations at such ridiculously low resolutions; the point is just that they are different. There can be many reasons why they differ; for example there are philosophical and pragmatic considerations such as whether to optimize for accuracy of font shape, or for legibility.
Unanswered questions
What is the format of the bitmap font information in the PostScript file, surrounded by %DVIPSBitmapFont:
comments? I couldn't find any documentation, but a little more experimentation will probably throw some light here.
Is it guaranteed that if we ask Ghostscript to render the PostScript file at exactly the same resolution as the one the bitmap was generated for, then it will use exactly the same bitmap (the one from the gf
or pk
file) for each character? I verified this for each of the five distinct characters in hello.ps
, but I'm not sure whether that's strictly guaranteed. (Note that Ghostscript will happily generate a PBM at whatever resolution for whatever font—that is its purpose—so this is a natural question to ask.) If not, then the last part of this answer would be invalid, and we should probably find an alternative. However, it seems plausible that gs
will use the same bitmap if the resolutions match. More experimentation should confirm this.
gs
rather thandvipng
. PNG does not seem likely at all. – cfr Mar 24 '17 at 4:12gs
seems to throw away the resolution and/or possibly do something weird. I still think there must be a way to get there from the dvi directly. Or even from the ps but with a particularly "dumb" rasterizer. But maybe one doesn't exist. – ShreevatsaR Mar 24 '17 at 15:05.mf
file. but the machinery has to be in place with information about the resolution of the intended printing device. at ams, we always pre-generated the.pk
files to the correct resolution, and it's been so long since we've done that -- and on different hardware -- that the procedures are no longer in place. i'm pretty sure i can come up with the resolution-specific information needed bygftopk
(that's been published in tugboat), but i'm not so sure anyone has agftopk
binary for a current os. – barbara beeton Mar 25 '17 at 2:25