Infinite loop on a non-macro token

Question

The following document infinite-loops:

\documentclass{article}
\usepackage{breqn}
\begin{document}

\newcommand{\allowbreaks}[2][,]{\begingroup
  \catcode`#1=\active
  \scantokens{%
    \edef#1{%
      \show\detokenize{#1}%
      \detokenize{#1}%
      \noexpand\allowbreak
    \show0%
    }%
    #2%
  }
\endgroup}

\allowbreaks{$,$}

\end{document}

LaTeX displays

> the character ,.
,->\show ,
          ,\allowbreak \show 0
l.113 \allowbreaks{$,$}

?

repeatedly.

If I type I\show into it's prompt, it gives me

> the character ,.
,->\show ,,
           \allowbreak \show 0
l.113 \allowbreaks{$,$}

That is, the token that it's expanding it a character, not a macro. How is this possible?

If I comment out the breqn line, it works fine. I'm using pdfTeX, Version 3.1415926-1.40.10 (TeX Live 2009/Debian).

Answer 1

When you say \detokenize{,}, the comma becomes a character of category code 12, so \show, will show the meaning of this character.

The \edef will define the active comma to expand to

\show,,\allowbreak\show0

where the two commas have category code 12.

But when breqn is loaded, the comma has mathcode "8000 and you are in math mode! So TeX follows its rules: after having shown the meaning of the comma it finds another comma and, having it category code 12 and mathcode "8000, TeX replaces it with the meaning it currently has as an active character. That is, to show the comma and to use a comma, which has category code 12 and mathcode "8000 …

Oops, infinite loop. :-)