# hanging package chokes on apostrophes

I'm writing a file in LaTeX2e with TeXshop in which I'd like some paragraphs where all lines of a paragraph after the first line are intended (as with dictionary entries). I found out about the hanging package -- such paragraph are called hanging paragraphs, which I hadn't known before -- and tried to use it. Alas, I got a "Fatal error" message.

Eventually I determined the problem is that the hanging package chokes if there is an apostrophe in math mode, such as you would find in $f'(x)$. I isolated the problem in the following short file:

\documentclass[11pt]{amsart}
\usepackage{hanging}

\begin{document}
$f'(x)$
\end{document}


That is the whole file. When I try to compile this, I get the following error message:

! TeX capacity exceeded, sorry [input stack size=5000].
'->\futurelet
\next \h@ngrqtest
l.7 $f'( x)$
!  ==> Fatal error occurred, no output PDF file produced!
Transcript written on test.log.


Can anyone tell me how to fix this problem?

I looked online to see if someone else wrote about this, and all I found was something from 2010 at http://handyfloss.wordpress.com/2006/11/30/tex-capacity-exceeded-error/, where in reply 13 Joshua Sasmor reports he found this problem. But no solution that permits usage of the hanging package was forthcoming.

-
I came across this problem myself today and was very glad to find this here. I think the kindest thing you can say about this is that it's an extremely bad design decision by the package author, along with some extremely poor software engineering in terms of reporting the error. – Ben Crowell Aug 21 '14 at 0:16
Kind words indeed! :) – KCd Aug 21 '14 at 1:06

The problem is the “hanging punctuation” (whatever that is?) feature of the hanging package. It makes all kinds of punctuation characters active, without checking whether they are used in math-mode or text-mode. Fortunately, there are options to disable that behavior:

\documentclass[11pt]{amsart}
\usepackage[notquote]{hanging} % possibly you need to add notcomma, ...
% (see the package manual)

\begin{document}
$f'(x)$
\end{document}

-
Thanks, this worked perfectly both on my little test file and, more importantly, on my main file! – KCd Jun 30 '12 at 4:33

If you want to keep the "hanging quote" (in the hangpunct environment), then this code in the preamble should work:

\usepackage{hanging}
\makeatletter
\def\latex@@rquote{^\bgroup\prim@s} % kernel meaning of math active quote
\let\hang@@h@ngrquote\h@ngrquote
\renewcommand{\h@ngrquote}{%
\relax\ifmmode
\expandafter\latex@@rquote
\else
\expandafter\hang@@h@ngrquote
\fi}
\makeatother


The change should be straightforward: we check whether we are in math mode; if so we do what the active quote should do according to the LaTeX kernel, otherwise we let hanging perform its duty.

-