# Why does this protection not work?

The answers to this question explain in detail what protection is so I'm not wondering about that. My question is why a certain command definition with expl3 does not give a protected command.

Consider this LaTeX2e example:

\documentclass{article}

\DeclareRobustCommand\foo[1]{#1}
\DeclareRobustCommand\?{??}
\expandafter\DeclareRobustCommand\expandafter\-\expandafter{\-}

\begin{document}

\tableofcontents
% this is <jobname>.toc:
% \contentsline {section}{\numberline {1}\foo {bar}}{1}
% \contentsline {section}{\numberline {2}foo\?}{1}
% \contentsline {section}{\numberline {3}foo\-bar}{1}

\section{\foo{bar}}

\section{foo\?}

\section{foo\-bar}

\end{document}


Everything is working as expected.

However, when I try to do the same with expl3 the \- is not protected:

\documentclass{article}

\usepackage{expl3}
\ExplSyntaxOn
\cs_new_protected:Npn \foo #1 { #1 }          % works
\cs_new_protected:Npn \? { ?? }               % works
\exp_args:NNo \cs_set_protected:Npn \- { \- } % does not work
\ExplSyntaxOff

\begin{document}

\tableofcontents
% this is <jobname>.toc:
% \contentsline {section}{\numberline {1}\foo {bar}}{1}
% \contentsline {section}{\numberline {2}foo\?}{1}
% \contentsline {section}{\numberline {3}foo\discretionary {-}{}{}bar}{1}
% \contentsline {section}{\numberline {4}foo\-bar}{1}

\section{\foo{bar}}

\section{foo\?}

\section{foo\-bar}

\section{foo\protect\-bar}

\end{document}


Am I doing something wrong or why does \exp_args:NNo \cs_set_protected:Npn \- { \- } not leave the desired result? The only command where I could find this behaviour is \-. Did I not look hard enough or is there something special to \-?

-

You are being caught out not by expl3 but by LaTeX2e! The redefinition you use is correct, which you will see if you \show\-:

> \-=\protected\long macro:
->\discretionary {-}{}{}.


However, LaTeX2e redefines \- during the output routine, which is where the .toc entries are created:

\@arrayparboxrestore ->\let \if@nobreak \iffalse \let \if@noskipsec \iffalse \l
et \par \@@par \let \-\@dischyph \let \'\@acci \let \\@accii \let \=\@acciii \
parindent \z@ \parskip \z@skip \everypar {}\linewidth \hsize \@totalleftmargin
\z@ \leftskip \z@skip \rightskip \z@skip \@rightskip \z@skip \parfillskip \@flu
shglue \lineskip \normallineskip \baselineskip \normalbaselineskip \sloppy
{\let}
{reassigning \if@nobreak=\iffalse}
{\let}
{reassigning \if@noskipsec=\iffalse}
{\let}
{reassigning \par=\par}
{\let}
{changing \-=\protected\long macro:->\discretionary {-}{}{}}
{into \-=macro:->\discretionary {-}{}{}}
...


So you may wonder why we see a difference between the behaviour with a protected command and a LaTeX2e 'robust' one: after all, both have been ignored by the output routine! It comes down to how the different protection works. With the LaTeX3 method, it's down to the engine, so each time an \edef or similar is applied to \- it stays totally unchanged. However, the protection is removed by LaTeX2e.

On the other hand, the LaTeX2e robust mechanism results in \- turning into \protect\- inside an \edef. That means that even if we change the definition of \- later, the protection 'hangs about'.

You see this where TeX writes to the .toc when \tracingall. With LaTeX2e's robust mechanism:

\write->\@writefile{toc}{\protect \contentsline {section}{\protect \numberline
{1}foo\protect \-bar}{\thepage }}


but with LaTeX3's native engine system

\write->\@writefile{toc}{\protect \contentsline {section}{\protect \numberline
{1}foo\-bar}{\thepage }}


As egreg has already commented, the best solution at the moment is to make \@dischyph engine-protected by setting it equal to \-. (Of course, longer-term a LaTeX3 format will have everything protected or expandable, and this sort of issue should go away.)

By the way, I'd use

\cs_set_protected_nopar:Npx \- { \exp_not:o { \- } }

-
I never would have catched that. I need to use \tracingall more often. Is there a reason you'd use \cs_set_protected_nopar:Npx \- { \exp_not:o { \- } } ? What about a \cs_set_protected_nopar:Npo variant? –  cgnieder Jan 26 '12 at 21:39
@cgnieder I find the x-type expansion easier to read here than using a (hidden) expansion chain. That's a general LaTeX3 thing: if we can use x-type expansion plus some \exp_not:o stuff then it's usually easier to read. We've not got \cs_set_protected_nopar:Npo principally because it's not really needed, given the way you can use x-type expansion. –  Joseph Wright Jan 26 '12 at 21:41
Curious. For some reason \exp_not:o and its brothers seem to always slip my mind. I only thought of \exp_not:N and thus disregarded x-type expansion. Thanks. –  cgnieder Jan 26 '12 at 21:51

Among what LaTeX does during the output routine there is \@arrayparboxrestore which emits, among other things,

\-\@dischyph


where \@dischyph is defined by the kernel with

560 \def\-{\discretionary{-}{}{}}
561 \let\@dischyph=\-


(I've left the line numbers). This means that when the .aux file is written out, \- has again the value it had before your redefinition (and is not protected, of course). With \DeclareRobustCommand what's passed to the .aux file is not \-, but something different (it's really \protect\- that, since \protect is \noexpand, becomes the unexpanded \-). Thus the problem does not show.

\makeatletter
\let\@dischyph\-
\makeatother


after having redefined \-.

-
I originally thought it would be the \-  business with the space at the end of the name: it took a careful reading of the log to reveal that this is not the case. –  Joseph Wright Jan 26 '12 at 21:24
Yes, it's a control symbol, so it uses \x@protect. –  egreg Jan 26 '12 at 21:34
Thanks for the clarification. I always wonder where one get's to know this kind of information. I accepted Joseph's answer because he also answered the expl3 part of my question. But you've got my upvote. –  cgnieder Jan 26 '12 at 21:56
@cgnieder Note that the redefinition of \- in \@arrayparboxrestore is due to the fact that the output routine might be invoked inside a tabbing environment, where \- is redefined. –  egreg Jan 26 '12 at 22:01
I see. It all begins to make some sense now. I was completely unaware of the tabbing` environment, though. –  cgnieder Jan 26 '12 at 22:29