I recently asked a related question about how often one should update LaTeX packages. Supposing that I update a bunch of packages in one go, or regularly update with only a few packages changing, how can I test that my TeX documents aren't changing substantially?
In programming, this would be akin to unit or regression testing. Maybe I am bringing too much experiences with programming bugs to the LaTeX realm, but it is a concern of mine that, over time, different versions of packages will lead to different PDF output.
I am aware of
latexdiff, and use it routinely for manual inspection of differences in tex files, but that doesn't address two issues: it won't easily capture differences attributable to different package versions, and it isn't clear to me that it's easily automated to identify those files that have differences. It seems to me that the former issue requires that I do a lot to manually segregate different versions of packages, and the latter issue requires doesn't seem easily automated at all.
Is there another method for checking that the output is visually the same, for files compiled with different versions of the same dependencies?
Update 1: A use case: Suppose you have a lot of published documents or a large manuscript, and want to be sure that their compiled versions haven't changed due to a package update. It may be the case that you want to change the output, hence the reason for a package update, but it's most important that you know if anything has changed, and you can subsequently discern how and why they changed.