The problem is that kWh is a combined unit of watt and hours with a kilo prefix. So if you wirte \kWh it expands to \kilo\watt\hour and the \per will be applied to the \watt part only.
Solution 1
Use the sticky-per option to get \per\hour to.
\documentclass{article}
\usepackage{siunitx}
\begin{document}
\SI[per-mode=symbol,sticky-per]{0.1}[\$]{\per\kWh}
\end{document}

Adding bracket-unit-denominator = false will suppress the parens.
\documentclass{article}
\usepackage{siunitx}
\begin{document}
\SI[%
per-mode=symbol,sticky-per,
bracket-unit-denominator=false,
]{0.1}[\$]{\per\kWh}
\end{document}

Solution 2
Overwrite the definition of \kWh:
\documentclass{article}
\usepackage{siunitx}
\AtBeginDocument{\DeclareSIUnit{\kWh}{kWh}}
\begin{document}
\SI[per-mode=symbol]{0.1}[\$]{\per\kWh}
\end{document}

Conclusion
Decide wether you think kWh is a combine unit or not. If you think it is, use Solution 1 if you see it as a single unit use solution 2.
\SI[per-mode=symbol]{0.1}{\$\per\kWh}? – Claudio Fiandrino May 15 '12 at 7:53dollar-cent 10 per kilowatt-hour? The comment from @ClaudioFiandrino looks like the correct solution to me – matth May 15 '12 at 8:21