How to generate a bitmap using TeX (with bitmap fonts)?

Question

This may be a strange question, as it is something most users do not want.

I would like to typeset some text with TeX, and get the description of the typeset page as a bitmap: a sequence of 0s and 1s representing where the ink should go (assuming a certain resolution of course). Moreover I'd like to use the raster (Metafont) version of the Computer Modern font, and not a vector version like cm-super or Latin Modern or whatever. Ideally even be able to vary the resolution (number of dots per inch) and other printer characteristics, as Metafont can do.

How could I achieve this? As I understand it, it should be possible :-) (Knuth's goal when developing TeX and Metafont was to produce output for a digital typesetter: to produce the pattern of dots that would go onto the page, for output devices like the XGP (180–200 dpi), Dover (384 dpi) and Alphatype CRS (5333 dpi). But perhaps he never generated explicit bitmaps; just output in whatever format was suitable for those devices.)

If I understand correctly, the DVI file contains instructions like "put character 72 at position (h, w)". (We can view the DVI file in a textual form with dvitype, also written by Knuth.) Then a DVI driver (like dvips or dvipdfmx) will take these instructions, and separately the font information, and produce the actual output. This is the part where I want it to use the “original” Metafont (raster) fonts rather than their vector equivalents.

There seems to exist a dvipng on my system (part of TeX Live), which may be something I could use. It even has options like --mode and --bdpi which seem relevant. The problems are:

• I don't know how to use this :-) (how to specify resolution for example)
• The PNG format is not trivial and I'm not sure how to get a plain sequences of 0s and 1s out of it. In fact, the dvipng documentation mentions things like anti-aliasing and --gamma “interpolation of colors”, which makes me fear that it's not strictly generating a bitmap, but a more sophisticated raster image.

So perhaps there is an alternative: what is a good way one can get a plain and simple bitmap for a page?

Answer 1

I think I found an answer (following everyone's valuable suggestions): below is what I understand of the information contained in various kinds of files, and the path from TeX source to rendered bitmap.

TeX file

We start with a TeX file, like (say) a hello.tex that contains just:

hello
\bye

DVI file

Compiling the TeX file with tex hello.tex produces hello.dvi, which contains only the name of the font and instructions where to place the characters from the font. For example, this is what hello.dvi contains (output of hexdump -v -C hello.dvi):

00000000  f7 02 01 83 92 c0 1c 3b  00 00 00 00 03 e8 1b 20  |.......;....... |
00000010  54 65 58 20 6f 75 74 70  75 74 20 32 30 31 37 2e  |TeX output 2017.|
00000020  30 33 2e 32 34 3a 31 32  35 35 8b 00 00 00 01 00  |03.24:1255......|
00000030  00 00 00 00 00 00 00 00  00 00 00 00 00 00 00 00  |................|
00000040  00 00 00 00 00 00 00 00  00 00 00 00 00 00 00 00  |................|
00000050  00 00 00 ff ff ff ff 8d  9f f2 00 00 8e a0 02 83  |................|
00000060  33 da 8d a0 fd 86 cc 26  8d 91 14 00 00 f3 00 4b  |3......&.......K|
00000070  f1 60 79 00 0a 00 00 00  0a 00 00 00 05 63 6d 72  |.`y..........cmr|
00000080  31 30 ab 68 65 6c 6c 6f  8e 8e 9f 18 00 00 8d 92  |10.hello........|
00000090  00 e8 60 a3 31 8e 8c f8  00 00 00 2a 01 83 92 c0  |..`.1......*....|
000000a0  1c 3b 00 00 00 00 03 e8  02 9b 33 da 01 d5 c1 47  |.;........3....G|
000000b0  00 02 00 01 f3 00 4b f1  60 79 00 0a 00 00 00 0a  |......K.`y......|
000000c0  00 00 00 05 63 6d 72 31  30 f9 00 00 00 97 02 df  |....cmr10.......|
000000d0  df df df df                                       |....|

A human-readable version of this can be got with dvitype; the relevant part is:

109: fntdef1 0: cmr10
130: fntnum0 current font is cmr10
131: setchar104 h:=1310720+364090=1674810, hh:=106
132: setchar101 h:=1674810+291271=1966081, hh:=124
133: setchar108 h:=1966081+182045=2148126, hh:=136
134: setchar108 h:=2148126+182045=2330171, hh:=148
135: setchar111 h:=2330171+327681=2657852, hh:=169

I think these are instructions defining font 0 as "cmr10", to use font 0, and to set characters (104, 101, 108, 108, 111) (corresponding to "hello") at certain positions. The hh values vary based on the -dpi value passed to dvitype (default 300), but the rest are fixed. (We can also use "dviasm hello.dvi" or "dv2dt hello.dvi" for human-readable output in other formats.) So as we can see, there is still not any font shape information in the dvi file. A DVI driver is what takes these instructions, looks up the font ("cmr10"), and puts the specified characters at the specified positions.

PS file with Type 1 fonts

Calling dvips hello.dvi uses (on my system at least) the Type 1 (vector) fonts like texmf-dist/fonts/type1/public/amsfonts/cm/cmr10.pfb. The comments in cmr10.pfb say %Copyright: Copyright (c) 1997, 2009 American Mathematical Society. I think this corresponds to this page on the AMS site about the Type 1 fonts, which mentions that the (outline version of the) fonts were produced by Blue Sky Research and Y&Y Inc. (See also README.bluesky in fonts/cm/ps-type1 directory, "Blue Sky Research and Computer Modern fonts" on Nelson Beebe's site, and Karel Píška's comparison (draft PDF) of these with another Type 1 version of Computer Modern, namely that of BaKoMa (by Basil K. Malyshev).)

There is some binary stuff in this file cmr10.pfb, which you can turn into human-readable form with t1disasm. That shows that, for example, the glyph for the letter e is defined in cmr10.pfb with code like:

    /e {
            28 444 hsbw
            67 4 callsubr
            84 252 rmoveto
            6 149 84 25 34 0 rrcurveto
            103 10 -135 -39 hvcurveto
            closepath
            -238 -21 rmoveto
            279 hlineto
            22 3 0 21 hvcurveto
            99 -54 97 -125 vhcurveto
            -116 -92 -103 -125 hvcurveto
            68 4 callsubr
            -134 105 -97 115 vhcurveto
            122 45 111 19 hvcurveto
            10 -8 2 -5 vhcurveto
            -9 0 -2 -6 -2 -8 rrcurveto
            -35 -103 -90 0 -10 0 rrcurveto
            -50 0 -40 30 -23 37 rrcurveto
            -30 48 0 66 0 36 rrcurveto
            closepath
            endchar
            } ND

This is the kind of font that most users want and use. It's a vector font rather than a raster font, meaning that it specifies the "shape" of the character rather than the bitmap for it. The way it ultimately becomes pixels on your screen (or paper) is that when you open the PostScript (or PDF) file in an interpreter/viewer (like Ghostscript), that program (the viewer/renderer) takes the shape specified above, and tries to approximate that shape into the available number of pixels at your viewing zoom level. Thus the same shape description "works" at all zoom levels, meaning that the renderer recomputes the pixels at whatever zoom level you choose. It can also use some hinting information specified with the font. (I learned a bit about all this (font rasterization etc.) when writing this answer, about a situation that is an example of how this can be subtle.)

To get a bitmap out of a PostScript file like this, we can use Ghostscript to produce a PBM file which is basically a matrix of 0s and 1s: cat hello.ps | gs -sDEVICE=pbm -sOutputFile=hello.pbm -r600 -. Note the "-r600" which specifies a resolution of 600 dpi: whatever resolution you specify, gs will "render" (rasterize) the PostScript file to that resolution, producing the bitmap.

As I said, these fonts are what most users want (see this highly upvoted answer by Heiko Oberdiek, the author of the brilliant pkfix), but here I want to see the bitmaps generated by Metafont (and positioned by TeX), not the ones generated by the viewer software, so moving on...

PS file with bitmap fonts

Instead of dvips hello.dvi which ended up using Type 1 fonts, we can use the -V option, documented as “Download non-resident PostScript fonts as bitmaps”. So calling dvips -V hello.dvi produces hello.ps, and the log says it's using bitmap fonts like texmf-var/fonts/pk/ljfour/public/cm/cmr10.600pk (from a directory inside our home directory).

This is also where we can specify a different DPI and mode from modes.mf, like dvips -V -D100 -mode amiga foo.dvi or dvips -V -D1200 -mode cgnszz foo.dvi. (The two must match, or it prints a warning like mktexpk: Mismatched mode […] and resolution […]; ignoring mode.) For each mode we specify, it either picks up the pk file from our local directory, or, if one doens't exist, automatically invokes Metafont. For example, if we call dvips -V -D406 -mode agfafzz hello.dvi (and these options are being used for the first time), the logs say:

kpathsea: Running mktexpk --mfmode agfafzz --bdpi 406 --mag 1+0/406 --dpi 406 cmr10
mktexpk: Running mf-nowin -progname=mf \mode:=agfafzz; mag:=1+0/406; nonstopmode; input cmr10
This is METAFONT, Version 2.7182818 (TeX Live 2015) (preloaded base=mf)
...
Font metrics written on cmr10.tfm.
Output written on cmr10.406gf (128 characters, 16756 bytes).
Transcript written on cmr10.log.
mktexpk: /Users/username/Library/texlive/2015/texmf-var/fonts/pk/agfafzz/public/cm/cmr10.406pk: successfully generated.

and this generated pk file is what is used in the PS file. See this excellent answer by David Carlisle for a bit on gf and pk files, and on modes.mf.

When such pk files are used, the ps file contains a section like (for our hello.dvi above, which needs only five characters):

%DVIPSBitmapFont: Fa cmr10 10 5
/Fa 5 112 df<0001C0000003C0000007C000001FC00000FFC000FFFFC000FFFFC000FF
1FC000001FC000001FC000001FC000001FC000001FC000001FC000001FC000001FC00000
1FC000001FC000001FC000001FC000001FC000001FC000001FC000001FC000001FC00000
1FC000001FC000001FC000001FC000001FC000001FC000001FC000001FC000001FC00000
1FC000001FC000001FC000001FC000001FC000001FC000001FC000001FC000001FC00000
1FC000001FC000001FC000001FC000001FC000001FC000001FC000001FC000001FC00000
3FE0007FFFFFF07FFFFFF07FFFFFF01C3879B72A>49 D<0007F800001FFF00007C0FC001
F803E003F001F007E001F80FC000F81F80007C1F80007C3F00007E3F00003E7F00003E7F
00003F7E00003FFE00003FFE00003FFE00003FFFFFFFFFFFFFFFFFFE000000FE000000FE
000000FE000000FE0000007E0000007E0000007F0000007F0000003F0000033F8000031F
8000070FC0000607C0000E07E0001C01F0003800F80070007E03E0001FFF800003FC0020
277EA525>101 D<03F000000000FFF000000000FFF000000000FFF0000000000FF00000
000003F00000000003F00000000003F00000000003F00000000003F00000000003F00000
000003F00000000003F00000000003F00000000003F00000000003F00000000003F00000
000003F00000000003F00000000003F00000000003F00000000003F00FF0000003F03FFC
000003F0F03F000003F1C01F800003F3800FC00003F7000FC00003FE000FC00003FC0007
E00003FC0007E00003F80007E00003F80007E00003F80007E00003F00007E00003F00007
E00003F00007E00003F00007E00003F00007E00003F00007E00003F00007E00003F00007
E00003F00007E00003F00007E00003F00007E00003F00007E00003F00007E00003F00007
E00003F00007E00003F00007E00003F00007E00003F00007E00003F00007E00003F00007
E00003F00007E00007F8000FF000FFFFC1FFFF80FFFFC1FFFF80FFFFC1FFFF80293A7EB9
2E>104 D<03F000FFF000FFF000FFF0000FF00003F00003F00003F00003F00003F00003
F00003F00003F00003F00003F00003F00003F00003F00003F00003F00003F00003F00003
F00003F00003F00003F00003F00003F00003F00003F00003F00003F00003F00003F00003
F00003F00003F00003F00003F00003F00003F00003F00003F00003F00003F00003F00003
F00003F00003F00003F00003F00003F00003F00003F00007F800FFFFC0FFFFC0FFFFC012
3A7EB917>108 D<0003FE0000000FFF8000003E03E00000F800F80001F0007C0003E000
3E0007C0001F000F80000F801F80000FC01F000007C03F000007E03F000007E07E000003
F07E000003F07E000003F07E000003F0FE000003F8FE000003F8FE000003F8FE000003F8
FE000003F8FE000003F8FE000003F8FE000003F8FE000003F87E000003F07E000003F07F
000007F03F000007E03F000007E01F80000FC00F80000F800FC0001F8007E0003F0003F0
007E0000F800F800007E03F000001FFFC0000003FE000025277EA52A>111
D E
%EndDVIPSBitmapFont

And this changes with mode: for mode amiga which is at only 100 dpi, there is less to the font:

%DVIPSBitmapFont: Fa cmr10 10 5
/Fa 5 112 df<20E02020202020207004097F8807>49 D<7088F88088700506808506>
101 D<C04040407844444444EE070A808908>104 D<C04040404040404040E0030A8089
04>108 D<7884848484780606808507>111 D E
%EndDVIPSBitmapFont

I haven't yet manged to reverse-engineer this information directly (how to get from <7088F88088700506808506> to the glyph for e). But we can use pktype on the generated pk file to see it in a human-readable form. This is the section for Character 101 (lowercase e) from texmf-var/fonts/pk/amiga/public/cm/cmr10.100pk:

635:  Flag byte = 224  Character = 101  Packet length = 15
  Dynamic packing variable = 14
  TFM width = 466035  dx = 393216
  Height = 6  Width = 5  X-offset = 0  Y-offset = 5
  .***.
  *...*
  *****
  *....
  *...*
  .***.

(Except at very low resolutions like the above, it doesn't print the pixel images like this: for the 600 DPI texmf-var/fonts/pk/ljfour/public/cm/cmr10.600pk the corresponding output looks like

3304:  Flag byte = 192  Character = 101  Packet length = 77
  Dynamic packing variable = 12
  TFM width = 466035  dx = 2424832
  Height = 39  Width = 32  X-offset = -2  Y-offset = 37
  (13)8(22)13(17)5(6)6(13)6(9)5(11)6(11)5(9)6(12)6(7)6(14)5(6)[1]6(16)5(4)6
  (17)6(3)6(18)5(2)7(18)5(2)7(18)6(1)6(19)13[2](19)77[4](26)[1]6(26)[1]7(26)6
  (22)2(2)7(21)2(3)6(20)3(4)6(19)2(6)5(18)3(6)6(16)3(9)5(14)3(11)5(12)3(13)6
  (7)5(16)14(21)8(10)

instead.)

We can also call pktogf on that pk file, and then use gftype on the gf file. At maximum output, namely with the -images -mnemonics options, gftype has output like:

1160: beginning of char 101: 0<=m<=5 0<=n<=5
(initially n=5) paint (1)3
1168: newrow 0 (n=4) paint 1(3)1
1172: newrow 0 (n=3) paint 5
1174: newrow 0 (n=2) paint 1
1176: newrow 0 (n=1) paint 1(3)1
1180: newrow 1 (n=0) paint 3
1182: eoc
.<--This pixel's lower left corner is at (0,6) in METAFONT coordinates
 ***
*   *
*****
*
*   *
 ***
.<--This pixel's upper left corner is at (0,0) in METAFONT coordinates

which fully spells out the bitmap for each character (glyph).

(Not sure whether the strange usage of "lower left" and "upper left" in the output is a bug in gftype (written by David Fuchs and Knuth), or an artefact/joke of some sort.)

Bitmap (e.g. PBM file)

I first learned of the PBM file format from this blog post. It is very convenient, and is a way of avoiding having to deal with every image format. From the PostScript file as above, Ghostscript can produce a PBM file as mentioned earlier:

cat hello.ps | gs -sDEVICE=pbm -sOutputFile=hello.pbm -r100 -

The resulting hello.pbm file starts like this:

P1
# Image generated by GPL Ghostscript (device=pbm)
850 1100
0000000000000000000000000000000000000000000000000000000000000000
0000000000000000000000000000000000000000000000000000000000000000
0000000000000000000000000000000000000000000000000000000000000000
0000000000000000000000000000000000000000000000000000000000000000
0000000000000000000000000000000000000000000000000000000000000000
0000000000000000000000000000000000000000000000000000000000000000
0000000000000000000000000000000000000000000000000000000000000000
0000000000000000000000000000000000000000000000000000000000000000
0000000000000000000000000000000000000000000000000000000000000000
0000000000000000000000000000000000000000000000000000000000000000
0000000000000000000000000000000000000000000000000000000000000000
0000000000000000000000000000000000000000000000000000000000000000
0000000000000000000000000000000000000000000000000000000000000000
000000000000000000
0000000000000000000000000000000000000000000000000000000000000000
0000000000000000000000000000000000000000000000000000000000000000
0000000000000000000000000000000000000000000000000000000000000000
0000000000000000000000000000000000000000000000000000000000000000
0000000000000000000000000000000000000000000000000000000000000000
0000000000000000000000000000000000000000000000000000000000000000
0000000000000000000000000000000000000000000000000000000000000000
0000000000000000000000000000000000000000000000000000000000000000
0000000000000000000000000000000000000000000000000000000000000000
0000000000000000000000000000000000000000000000000000000000000000
0000000000000000000000000000000000000000000000000000000000000000
0000000000000000000000000000000000000000000000000000000000000000
0000000000000000000000000000000000000000000000000000000000000000
000000000000000000
0000000000000000000000000000000000000000000000000000000000000000
0000000000000000000000000000000000000000000000000000000000000000
0000000000000000000000000000000000000000000000000000000000000000
0000000000000000000000000000000000000000000000000000000000000000

and so on. Each "block" in the above (a few lines of length 64 each, plus a bit more on another line) has 850 digits (0 or 1). There are 1100 such blocks. (As the paper size is 8.5 inches x 11 inches, and we've specified 100 dots per inch, the whole image is 850 dots x 1100 dots.) After unwrapping those lines and zooming in to the area which has the bits corresponding to "hello", we get the following bitmap for that region:

00000000000000000000000000000000000000000000000000000000000
00000000000000000000000000000000000000000000000000000000000
00000000000000000000000000000000000000000000000000000000000
00000000000000110000000000001100110000000000000000000000000
00000000000000010000000000000100010000000000000000000000000
00000000000000010000000000000100010000000000000000000000000
00000000000000010000000000000100010000000000000000000000000
00000000000000011110000111000100010001111000000000000000000
00000000000000010001001000100100010010000100000000000000000
00000000000000010001001111100100010010000100000000000000000
00000000000000010001001000000100010010000100000000000000000
00000000000000010001001000100100010010000100000000000000000
00000000000000111011100111001110111001111000000000000000000
00000000000000000000000000000000000000000000000000000000000
00000000000000000000000000000000000000000000000000000000000
00000000000000000000000000000000000000000000000000000000000

(If that doesn't say "hello" to you, try moving a good distance from your screen. Of course, using those bits to control which pixels are on and off is the "real" way of visualzing them.)

Anyway, it appears that this way we can get a bitmap of the typeset page, at whatever resolution / Metafont mode.

For what it's worth, the following is the bitmap of the same region that we get from the PostScript file that had the vector (Type 1) fonts, at the same resolution (100 dpi):

00000000000000000000000000000000000000000000000000000000000
00000000000000000000000000000000000000000000000000000000000
00000000000000000000000000000000000000000000000000000000000
00000000000000111000000000011101110000000000000000000000000
00000000000000011000000000001100110000000000000000000000000
00000000000000011000000000001100110000000000000000000000000
00000000000000011000000110001100110001100000000000000000000
00000000000000011111001101001100110011011000000000000000000
00000000000000011001011001101100110110001100000000000000000
00000000000000011001011111101100110110001100000000000000000
00000000000000011001011000001100110110001100000000000000000
00000000000000011001011000101100110110001100000000000000000
00000000000000111111101101111111111011011000000000000000000
00000000000000000000000000000000000000000000000000000000000
00000000000000000000000000000000000000000000000000000000000
00000000000000000000000000000000000000000000000000000000000

As you can see, it's considerably darker. Of course, it's meaningless to compare rasterizations at such ridiculously low resolutions; the point is just that they are different. There can be many reasons why they differ; for example there are philosophical and pragmatic considerations such as whether to optimize for accuracy of font shape, or for legibility.

Unanswered questions

1. What is the format of the bitmap font information in the PostScript file, surrounded by %DVIPSBitmapFont: comments? I couldn't find any documentation, but a little more experimentation will probably throw some light here.

2. Is it guaranteed that if we ask Ghostscript to render the PostScript file at exactly the same resolution as the one the bitmap was generated for, then it will use exactly the same bitmap (the one from the gf or pk file) for each character? I verified this for each of the five distinct characters in hello.ps, but I'm not sure whether that's strictly guaranteed. (Note that Ghostscript will happily generate a PBM at whatever resolution for whatever font—that is its purpose—so this is a natural question to ask.) If not, then the last part of this answer would be invalid, and we should probably find an alternative. However, it seems plausible that gs will use the same bitmap if the resolutions match. More experimentation should confirm this.