I am the author of TeXShop. TeXShop always saves the source before typesetting, whether you use Auto Save or not. The "dirty file" indicator is unreliable in recent systems, and seems to have been completely removed in Sierra. The Apple setting "Ask to keep changes when closing documents" DOES NOT turn off Auto Saving, at least in Sierra. It merely puts up a dialog when you close a document asking if you want to revert to its startup state. If you do this, your source and output will not agree, since the output contains all the changes you made before the last typeset.
Some users have found a hidden Apple preference to turn off Auto Save for all applications. I spent several hours yesterday helping a user who turned it off, forgot about it, and then ran into trouble with recent systems. Eventually the user created a new account and ran TeXShop in the new account, where it ran perfectly.
Sierra fixes a number of Apple bugs which affected TeXShop, and I rather strongly recommend that users upgrade to it. I recommend leaving Auto Save on. In particular, TeXShop is NEVER tested with Auto Save off, since the Preference to turn it off is hidden from ordinary users.