I wrote the project that cloutiy mentioned, but I am actually currently in the process of moving away from Pango/Cairo, so I thought it might be worth explaining why.
Pango and Cairo worked very well for basic typesetting use; I produced a couple of books using SILE and a Pango/Cairo backend, and they look good. However, there were a number of problems with them which made them unsuitable in the long run.
The most obvious is that Cairo's PDF surface is pretty basic. You can draw pictures and text on a page, which is most of what you need for typesetting, but people want the ability to add PDF outlines links, etc. which means that Cairo is not a general solution.
Another problem is with the shaper that Pango uses on Linux.
A fundamental part of TeX-like typesetting is putting text and glyphs onto boxes, measuring the boxes and calculating how many boxes fit nicely onto a line. Unfortunately, I can only make Pango's basic font shaping engine measure its boxes to the nearest point, which is not accurate enough; if you're laying out whole strings on text it works fine, because Pango/Cairo also handles the inter-glyph kerning, but if you want to put individual glyphs into boxes, then having the box size be inaccurate by about 12% leads to pretty horrible output. (The obvious solution is not to box up individual glyphs but to box up whole words; then the inaccuracy only affects the inter-word spacing. The problem with the obvious solution is that the TeX algorithm inserts discretionary nodes where words can be hyphenated. So you always end up with a few characters in each box, not a whole word.) The CoreText shaper on OS X works fine.
You can get a decent enough result with Pango/Cairo, and it will save a lot of work, but I think if you're aiming for something more general-purpose, it is worth putting in the work to talk to lower-level shaping libraries and PDF libraries.
If you haven't read it, State of Text Rendering is a good document which explains Cairo/Pango/Harfbuzz/etc. and how they all fit together.