The theorem count doesn't update when I move to another section. I.e. I'm currently in section 6, and LyX has already in theorem 30, when theorem 1 is in section 1. How can I make LyX start counting the theorems and lemmas from "1" in each new section?
If you add the Theorems (Numbered by Section) module your theorems will be numbered 1.x for the first section, 2.x for the second, etc.
If you for some reason don't want the section number in the theorem number (which you should, if you're cross referencing theorems), you can add to the preamble (Document --> Settings --> LaTeX preamble) the following.
(Assuming you're using the Theorems (AMS) module, I didn't check if other theorem modules have other names for that counter.)
Note that the LyX view will not have the correct numbers with this last method, but the PDF will be correct.