There is a bug in gloss-french (or more precisely it hasn't been updated to the new xetex yet). The space/boundary has in the newer xetex versions another class. This avoids the break:
\documentclass[12pt]{memoir}
\usepackage{polyglossia}
\setdefaultlanguage{french}
\makeatletter
\XeTeXinterchartoks 4095 \french@punctthin = {\xpg@unskip\nobreak\thinspace}%
\XeTeXinterchartoks 4095 \french@punctthick = {\xpg@unskip\nobreakspace}%
\makeatother
\begin{document}
Bonjour tout le monde. Si je vais au bout de la ligne, je vais avoir cela : non !
\end{document}
A patch that would allow you to run the document with luatex too would be:
\documentclass[12pt]{memoir}
\usepackage{polyglossia}
\setdefaultlanguage{french}
\usepackage{etoolbox}
\makeatletter
\patchcmd\french@punctuation{255}{4095}{}{}
\patchcmd\french@punctuation{255}{4095}{}{}
\makeatother
\begin{document}
Bonjour tout le monde. Si je vais au bout de la ligne, je vais avoir cela : non !
\end{document}
Beside this you don't have to add the space in the input as the french module will do it for you
\documentclass[12pt]{memoir}
\usepackage{polyglossia}
\setdefaultlanguage{french}
\begin{document}
Bonjour tout le monde. Si je vais au bout de la ligne, je vais avoir cela: non!
\end{document}
(Imho if you also write in other languages it is a good idea to get used to an input without spaces as it will give the wrong output e.g. in an english or german context.)
