soul package command \hl can not work with escaped space

Question

It seems that \hl command in the soul package can not work with escaped space. A minimal non-working example is

\documentclass{article}
\usepackage{xcolor, soul}

\colorlet{mycolor}{red!30}
\sethlcolor{mycolor}

\begin{document}

\hl{can\ we\ highlight}

\end{document}

The error message is

Package soul Error: Reconstruction failed.

Some posts (for example, here and here) suggest using \mbox(or \hbox) to wrap the text or using \protect. While these methods work, they have some drawbacks:

• If \mbox is used, long text will not wrap normally.
• If \protect is used, the background color for text will disappear.

How to solve this problem more elegantly?

Edit

The solution shared by Skillmon works for English, but not for Chinese. The following MWE will illustrate that.

\documentclass{ctexart}
\usepackage{xcolor, soul}
\usepackage{xparse}

\let\hlORIG\hl

\colorlet{mycolor}{red!30}
\sethlcolor{mycolor}

\ExplSyntaxOn
\tl_new:N \l_jdhao_hlx_tl
\RenewDocumentCommand \hl { +m }
  {
    \tl_set:Nn \l_jdhao_hlx_tl { #1 }
    \tl_replace_all:Nnn \l_jdhao_hlx_tl { \  } { ~ }
    \exp_args:NV \hlORIG \l_jdhao_hlx_tl
  }
\ExplSyntaxOff


\begin{document}

\hl{test\ test}  %% pass!
\hl{测试\ 测试} %% fail!

\end{document}

It is interesting that English test passed, but the Chinese test failed. The error message is the same.

Answer 1

With a helper macro which converts all the explicit spaces (\) into a normal space, this is possible (I redefined the \hl macro so that you don't have to change anything else):

\documentclass{article}
\usepackage{xcolor, soul}
\usepackage{xparse}
\let\hlORIG\hl

\colorlet{mycolor}{red!30}
\sethlcolor{mycolor}

\ExplSyntaxOn
\tl_new:N \l_jdhao_hlx_tl
\RenewDocumentCommand \hl { +m }
  {
    \tl_set:Nn \l_jdhao_hlx_tl { #1 }
    \tl_replace_all:Nnn \l_jdhao_hlx_tl { \  } { ~ }
    \exp_args:NV \hlORIG \l_jdhao_hlx_tl
  }
\ExplSyntaxOff

\begin{document}

\hl{can\ we\ highlight}

\end{document}

EDIT: Working with Chinese input, but breaking automatic hyphenation (you can still provide manual hyphenation with \-):

\documentclass{ctexart}
\usepackage{xcolor, soul}
\usepackage{xparse}
%\usepackage[chinese,main=english]{babel}
\let\hlORIG\hl

\colorlet{mycolor}{red!30}
\sethlcolor{mycolor}

\ExplSyntaxOn
\tl_new:N \l_jdhao_hlx_tl
\RenewDocumentCommand \hl { m }
  {
    \tl_set:Nn \l_jdhao_hlx_tl { #1 }
    \tl_replace_all:Nnn \l_jdhao_hlx_tl { \  } { ~ }
    \regex_replace_all:nnN { [^\s\c{-}] } { \c{hbox} \0 } \l_jdhao_hlx_tl
    \exp_args:NV \hlORIG \l_jdhao_hlx_tl
  }
\ExplSyntaxOff

\begin{document}
% for comparison of hyphenating 
\rule{.9\textwidth}{1pt}possibility\ to\ highlight

\rule{.9\textwidth}{1pt}\hl{possibility\ to\ highlight}

\rule{.9\textwidth}{1pt}\hl{pos\-sibility\ to\ highlight}

\rule{.9\textwidth}{1pt}\hlORIG{possibility to highlight}

\hl{汤面}
\hl{汤面}
\hl{测试\ 测试}
%\hlORIG{汤面}

\end{document}

EDIT2:

Yet another version which allows you to use braces and other stuff in the argument of \hl. Note that both this version and the second version break maths in the argument of \hl. You can change the regular expression used as the first argument of \regex_replace_all:nnN to fix more issues you're facing, I'm not willing to put more time into fiddling with it.

\documentclass{ctexart}
\usepackage{xcolor, soul}
\usepackage{xparse}
%\usepackage[chinese,main=english]{babel}
\let\hlORIG\hl

\colorlet{mycolor}{red!30}
\sethlcolor{mycolor}

\ExplSyntaxOn
\tl_new:N \l_jdhao_hlx_tl
\RenewDocumentCommand \hl { m }
  {
    \tl_set:Nn \l_jdhao_hlx_tl { #1 }
    \tl_replace_all:Nnn \l_jdhao_hlx_tl { \  } { ~ }
    \regex_replace_all:nnN { \c[^CBEMTPUDA]\S } { \c{hbox} \0 } \l_jdhao_hlx_tl
    \exp_args:NV \hlORIG \l_jdhao_hlx_tl
  }
\ExplSyntaxOff

\begin{document}
% for comparison of hyphenating 
\rule{.9\textwidth}{1pt}possibility\ to\ highlight

\rule{.9\textwidth}{1pt}\hl{possibility\ to\ highlight}

\rule{.9\textwidth}{1pt}\hl{pos\-sibility\ \textbackslash{} to\ highlight}

\rule{.9\textwidth}{1pt}\hlORIG{possibility\textbackslash{} to highlight}

\hl{汤面}
\hl{汤面}
\hl{测试\ 测试}

\end{document}