# Ignoring Punctuation during Comparison

I am trying to compare two strings and ignore punctuation during the comparison. I thought that catcode redefinition would be a good way to proceed. But, obviously, the use of \catcode has a great deal of nuance.

I make my MWE below by comparing two strings, one "x" and the other "x,". In the MWE, you will see three separate places where I could invoke the catcode to ignore commas (catcode 9). Only the invocation before the definition of the comma'd string makes the comparisons equal. Obviously, that is not what I'm looking for. When I put the catcode change in my routine, the comma remains in the earlier defined string.

So, I thought maybe if I re-edef the comma'd string within my routine, following the catcode change, the comma would be removed from the updated \edef. Not so. As you can see, I tried several different ways of redefining the comma'd string, employing various combinations of \xdef, \edef and \expandafter.

I'm not tied to a catcode approach. If there is a way to redefine the comma temporarily to nothing with a \def command, I'd be more than happy to use that approach.

The key is that that the two strings come to the check routine with their commas intact, and I have to develop a way to discard the comma on the fly.

\documentclass{article}
\usepackage{ifthen}
\parindent 0in
\begin{document}
\edef\x{x}
%  \catcode,=9 % ONLY THIS EXTERIOR INVOCATION GIVES EQUALITY
\edef\xcomma{x,}
%  \catcode,=9 % THIS EXTERIOR INVOCATION DOES NOT GIVE EQUALITY
\def\testequalignorecomma#1#2{%
\catcode,=9 % THIS INTERIOR INVOCATION DOES NOT GIVE EQUALITY
\ifthenelse{\equal{#1}{#2}}{#1~equals #2}{#1~does not equal #2}\\%
\edef\xx{#1}\expandafter\edef\expandafter\xxc\expandafter{#2}%
\ifthenelse{\equal{\xx}{\xxc}}{\xx~equals \xxc}{\xx~does not equal \xxc}\\%
\xdef\xx{#1}\xdef\xxc{#2}%
\ifthenelse{\equal{\xx}{\xxc}}{\xx~equals \xxc}{\xx~does not equal \xxc}%
\catcode,=12 %
}
\testequalignorecomma{\x}{\xcomma}
\end{document}

• You can't change catcode of tokens when they have already been absorbed. – egreg Mar 28 '13 at 17:39
• You should be able to do this with regular expressions: ctan.mackichan.com/macros/latex/contrib/l3experimental/…. – Ethan Bolker Mar 28 '13 at 18:02
• @EthanBolker Thanks. I can also use stringstrings package \Treatment{1}{1}{0}{1}{1}{1}\substring[q]{\xcomma}{1}{\$} to remove ALL the punctuation, but I was hoping to do it simply without loading other packages. – Steven B. Segletes Mar 28 '13 at 18:07

catcode changes are really the wrong tool for manipulating tokens, they are only used when converting from characters in the file to character tokens, and have no affect normally on token lists. The etex \scantokens primitive does however let you treat a list of tokens as if it were a list of characters from a file, so having got this far you could complete like so:

\documentclass{article}
\usepackage{ifthen}
\parindent 0in
\begin{document}
\edef\x{x}
%  \catcode,=9 % ONLY THIS EXTERIOR INVOCATION GIVES EQUALITY
\edef\xcomma{x,}

\makeatletter

\def\testequalignorecomma#1#2{%
\catcode,=9 % THIS INTERIOR INVOCATION DOES NOT GIVE EQUALITY
\edef\tmp{\noexpand\scantokens{\def\noexpand\tmp{\noexpand\ifthenelse{\noexpand\equal{#1}{#2}}}}}%
\tmp\ifhmode\unskip\fi\tmp
{#1~equals #2}{#1~does not equal #2}%
\catcode,=12 %
}

\makeatother
\testequalignorecomma{\x}{\xcomma}
\end{document}

• That solution is magical, and I guarantee I would never have stumbled across it. I will try to avoid catcode abuse in the future. – Steven B. Segletes Mar 28 '13 at 19:04
• There is only one slight drawback, I found. If you stick a printing marker about both sides of \testequalignorecomma, you will find that its invocation introduces some white space in the output. – Steven B. Segletes Mar 28 '13 at 19:40
• @StevenB.Segletes ooh \scantokens is a tricky beast to tame, code updated. – David Carlisle Mar 28 '13 at 19:49

Without category code shuffling, here's a general solution with regular expressions:

\documentclass{article}
\usepackage{xparse,l3regex}
\ExplSyntaxOn
\NewDocumentCommand{\compareignorepunct}{mmmm}
{
\segletes_compare_ignorepunct:nnnn {#1}{#2}{#3}{#4}
}

\regex_const:Nn \c_punctuation_regex { [\.\,\:\;\?\!]+ }

\cs_new_protected:Npn \segletes_compare_ignorepunct:nnnn #1 #2 #3 #4
{
\tl_if_single:nTF { #1 }
{ \tl_set:No \l_tmpa_tl { #1 } }
{ \tl_set:Nn \l_tmpa_tl { #1 } }
\tl_if_single:nTF { #2 }
{ \tl_set:No \l_tmpb_tl { #2 } }
{ \tl_set:Nn \l_tmpb_tl { #2 } }
\regex_replace_all:NnN \c_punctuation_regex { } \l_tmpa_tl
\regex_replace_all:NnN \c_punctuation_regex { } \l_tmpb_tl
\tl_if_eq:NNTF \l_tmpa_tl \l_tmpb_tl { #3 } { #4 }
}
\ExplSyntaxOff

\begin{document}
\def\x{x}
\def\xcomma{x,}
\compareignorepunct{\x}{\xcomma}{\typeout{EQUAL}}{\typeout{NOT EQUAL}}
\compareignorepunct{x?}{\xcomma}{\typeout{EQUAL}}{\typeout{NOT EQUAL}}
\end{document}


If the argument is a single token, we assume it's a control sequence to be expanded. Variations are possible.

The terminal output is:

This is pdfTeX, Version 3.1415926-2.4-1.40.13 (TeX Live 2012)
restricted \write18 enabled.
entering extended mode
(./ignorepunct.tex
LaTeX2e <2011/06/27>
[...irrelevant lines omitted...]
EQUAL
EQUAL
(./ignorepunct.aux) )
No pages of output.
Transcript written on ignorepunct.log.
`