\DTLifinlist crashes when used with \dtlexpandnewvalue & the character 'é'

Question

The following code gives the error

! You can't use `macro parameter character #' in horizontal mode. \DTLifinlist ...4->\def \@dtl@doifinlist ##1,#1,## 2\end@dtl@doifinlist {\def... l.18 }

\documentclass{article}

\usepackage{datatool}

\begin{document}

\DTLnewdb{dtOne}%
\dtlexpandnewvalue
\DTLnewrow{dtOne}%
\DTLnewdbentry{dtOne}{Place}{Marché}%

\DTLforeach*{dtOne}%
{\Place=Place}%
{%
    \expandafter\DTLifinlist\expandafter{\Place}{Car,Man,Ferry}%
    {\Place: Found!\\}%
    {\Place: Not found!\\}%
}

\end{document}

If I %\dtlexpandnewvalue or if I change the db entry to \DTLnewdbentry{dtOne}{Place}{Marche}% ('é' changed in 'e'), the error goes away.

Is there a way to use \DTLifinlist with \dtlexpandnewvalue and accented characters?

Answer 1

You might protect the entry against expansion and then compare the entries as strings.

\documentclass{article}

\usepackage{datatool}

\newcommand{\DTLstringifinlist}[4]{%
  \begingroup\edef\x{\endgroup
    \noexpand\DTLifinlist{\detokenize\expandafter{#1}}{\detokenize{#2}}%
  }\x{#3}{#4}%
}
\newcommand{\DTLnewdbprotectedentry}[3]{%
  \DTLnewdbentry{#1}{#2}{\unexpanded{#3}}%
}

\begin{document}

\DTLnewdb{dtOne}
\dtlexpandnewvalue
\DTLnewrow{dtOne}
\DTLnewdbprotectedentry{dtOne}{Place}{Marché}

\DTLforeach*{dtOne}
  {\Place=Place}%
  {%
    \DTLstringifinlist{\Place}{Car,Man,Ferry}%
    {\Place: Found!}%
    {\Place: Not found!}%
  }

\DTLforeach*{dtOne}
  {\Place=Place}%
  {%
    \DTLstringifinlist{\Place}{Car,Man,Ferry,Marché}%
    {\Place: Found!}%
    {\Place: Not found!}%
  }

\DTLforeach{dtOne}{\Place=Place}{\Place}

\end{document}

Works with pdflatex, xelatex and lualatex.

Answer 2

Yes! Just use XeLaTeX to compile your code and no errors will be generated!

Alternatively, if you insist on using pdfLaTeX, you can replace Marché with March\'e, and no errors will be generated.

---

I shall quote from the datatool documentation:

Page 84: As from version 2.24, the predefined handlers now have limited support for UTF-8 characters. This is still experimental.

and

Page 85: With regular (pdf)LaTeX (as opposed to XeLaTeX) each UTF-8 character is actually treated as two tokens that represent the first and second octet of the UTF-8 character. This means that it’s not possible for TeX to obtain a character code using the usual backtick method…