You ask,
Is there a general rule I can follow to decide whether the punctuation between math mode content should be in math mode or text mode?"
Short answer: Yes.
Slightly longer answer: The commas should be rendered in math mode if they're a fundamental part of the math expression(s), and in text mode otherwise.
But how to decide in whether the commas are fundamental parts of a math expression? Let's consider your first example:
Therefore the solutions to the given equation are $2$, $4$, $6$, $8$, [and~]$10$.
One way to make progress is to ask if one can either replace the math components with non-mathy words, say
Therefore the solutions to the given equation are red, orange, yellow, green, blue, indigo, and violet.
This sentence parses fine; the commas are mainly needed to provide visual separators between objects (here: nouns). Another way is to ask if changing the ordering of the math components would change the message fundamentally. E.g.,
Therefore the solutions to the given equation are $8$, $2$, $4$, $10$, and~$6$.
The fact the numbers that form the solution set belong to a short sequence of even numbers does not require them to be listed in ascending order in order for the sentence to form a valid statement, right? By either of these semantic criteria, then, the commas are part of the sentence -- and therefore should be typeset in text mode. (Aside: Do note that the sentence-ending period should also be rendered in text mode.)
On to your second example:
Let us consider numbers of the form $4k + 1$ where $k = 0, 1, 2, 3, \dots$.
Here, the commas separate elements of a (very simple) arithmetic sequence, and it's obviously meaningful to list the numbers in ascending order. Since they're part of a math expression, they should be typeset in math mode.
By the way, I recommend omitting the .
(period) at the end of this sentence; the output of \dots
will easily be understood to denote both a typographic ellipsis and the end of the sentence it's a part of.