Why doesn't \xspace always insert a following space when that is expected?

Question

In the MWE, \xspace correctly inserts a space in the first test of \foo, but not in the second test when it is followed by an en-dash.

Is there a fix for this?

\documentclass{article}
\usepackage{xspace}
\newcommand{\foo}{FOO\xspace}

\begin{document}
\foo BAR

\foo -- why no space here?

\end{document}

Output:

Answer 1

The hyphen character - is in xspace's exception list¹, where the space isn't added. You can remove it with \xspaceremoveexception{-}:

\documentclass{article}
\usepackage{xspace}
\newcommand{\foo}{FOO\xspace}

\begin{document}
\foo BAR

\foo -- why no space here?

\xspaceremoveexception{-}
\foo -- why no space here?

\end{document}

---

As David said in the comments, xspace doesn't detect the space. As far as it knows \foo--bar and \foo --bar are exactly the same, because at the time \xspace gets to do its thing, TeX has already tokenized the macro and the next character, and any space in between was ignored.

xspace looks ahead to see if the next character is supposed to be preceded by a space and re-inserts that space if it is needed.

With the solution above, we changed \xspace's exception list, so it will always, from now on, add a space before a -, even when you don't want it to.

Still not satisfied?

If you want to hack into \xspace's rules, you can start by the example given in the manual:

\documentclass{article}
\usepackage{xspace}
\newcommand{\foo}{FOO\xspace}

\begin{document}
\foo BAR

\foo -- why no space here?

\xspaceremoveexception{-}
\foo -- why no space here?

\xspaceremoveexception{-}% repeated, for the sake of copy-pasting
\makeatletter
\renewcommand*\@xspace@hook{%
  \ifx\@let@token-%
    \expandafter\@xspace@dash@i
  \fi
}
\def\@xspace@dash@i-{\futurelet\@let@token\@xspace@dash@ii}
\def\@xspace@dash@ii{%
  \ifx\@let@token-%
  \else
    \unskip
  \fi
  -%
}
\makeatother

\foo - why no space here?

\foo -- why no space here?

\end{document}

There David shows how to use \@xspace@hook to check if the next two characters are -- and, if not, remove the space inserted by \xspace. This can be expanded to check more characters², but is it worth it?

My 2 cents: I used xspace for a few months, when I was writing my bachelor's thesis. It helps you not forget to type \foo{} or \foo\ when you are in the habit of forgetting it. But with time, you need different things (as you did here), and it gets more time consuming than it should. My opinion (and David's, apparently :P).

---

¹ The exception list, by default, contains:

,.'/?;:!~-)\ \/\bgroup\egroup\@sptoken\space\@xobeysp\footnote\footnotemark
%        ↑ Here it is :)

² Apparently I've got nothing better to do :)

The \@xspace@hook below checks:

1. If what follows \foo is a single -. If it is, it removes the space inserted by \xspace and returns;

2. If what follows \foo is --. If it is, it looks ahead for a space. If the space is found, the space inserted by \xspace is kept, otherwise it is removed. That is:

   \foo-bar  % prints FOO-bar
   \foo-- bar% prints FOO -- bar
   \foo--bar % prints FOO--bar
   

(note that either way, the space after \foo does not matter because of TeX's parsing rules).

\documentclass{article}
\usepackage{xspace}
\newcommand{\foo}{FOO\xspace}

\begin{document}
\pagenumbering{gobble}
\foo BAR

\foo -- why no space here?

\xspaceremoveexception{-}
\foo -- why no space here?

\xspaceremoveexception{-}
\makeatletter
\renewcommand*\@xspace@hook{%
  \begingroup
  \ifx\@let@token-%
    \obeyspaces
    \expandafter\@xspace@dash@i
  \fi
}
\def\@xspace@dash@i-{\futurelet\@let@token\@xspace@dash@ii}
\def\@xspace@dash@ii{%
  \ifx\@let@token-%
    \expandafter\@xspace@dash@sp@i
  \else
    \unskip
    -%
    \endgroup
  \fi
}
\def\@xspace@dash@sp@i-{\futurelet\@let@token\@xspace@dash@sp@ii}
\def\@xspace@dash@sp@ii{%
  \ifx\@let@token\space%
  \else
    \unskip
  \fi
  --%
  \endgroup
}
\makeatother

\foo - why no space here?

\foo -- why no space here?

\foo--why no space here?

\end{document}