The spacing above and below math blocks is also controlled by
\belowdisplayshortskip. Being skips, they can have some amount of stretch, which sometimes explains the (to my eye) surprising gaps between text paragraphs and math --- there's extra vertical stretchability that I didn't expect. Also, since the above and below versions may not be equal, they may stretch differently, leading to asymmetric spaces around the math.
In my current thesis style file, the default values are set at
Above: 10.0pt plus 2.0pt minus 5.0pt,
Aboveshort: 0.0pt plus 3.0pt,
Below: 10.0pt plus 2.0pt minus 5.0pt,
Belowshort: 6.0pt plus 3.0pt minus 3.0pt
(Note that Aboveshort != Belowshort; I have no idea why this is typographically right...) I don't quite understand when the "short" versions are used instead of the full versions, but I have found that always setting both pairs in tandem works well for me.