I noticed that after I symlinked the 09-texlive-fonts.conf file from /etc/fonts/conf.d it interfered with fonts system wide, notably firefox. On https://reddit.com/r/aww all the fonts in the header were showing 'squares' and in other applications font glyphs were not even rendering.
Is there any way to install the TexLive fonts without interfering with the system? Currently I'm doing:
sudo apt install lmodern
Which seems to work without any side effects, but this solution won't work on every Linux distribution.