The answers depend a lot on what kind of document you are producing.
LaTeX's float placement algorithm is a lot better and has a lot more parameters than many users believe. See Frank's description One of the most common problems of course is that when things don't go quite as people expect they use
[h] which makes it much more likely that the figure can not be positioned, and that it floats to the end of the document.
That said, it is just an algorithm, coded in TeX, using the memory constraints of the 1980's (and to a lesser extent updated for latex2e in the 1990's). If you are writing a thesis, that is, a single document which may be worked on over a period of months or years, you could contemplate taking more manual control over the figure positioning. If you are prepared to rewrite text and adjust figure scaling or order, in order to make things fit while maintaining good page breaks, then you should expect to be able to make a better job than an automatic placement. However this does introduce a very high additional overhead in the document production, that after potentially any edit you need to adjust the figure placement by hand.
If you are setting up typesetting rules for multiple documents, manually placing all figures is rarely an option and allowing latex to place them is usually the best choice.
If you are positioning things by hand there are two possible styles.
You may explicitly word the document on the basis that images (like equations) are non floating and are part of the text narrative. So the text can refer to "the figure below" etc, in this case figures do not need captions (just as equations do not) and a center environment is probably a good choice.
Or, you may want a look that, to the reader, looks like a document with captioned, floating, figures, but where you have taken control of the float positioning by hand just as a technical production detail. Here you do not just want
center as it will not keep the image and caption together, you need a
minipage or the float package
H option to produce an unbreakable box into which you can place the image and caption. Arranging that such captioned boxes come in sensible places such as top of page by hand would be a lot of work, but not impossible, if you want absolutely full control over your document.