Active character can be defined, but not within \def?

Question

This is a follow up to my earlier question here: Detect catcode of next character?

What I want to do: Count the number of " used in the document body. I am not using " for purposes such as math, babel shortcuts, or Unicode characters. Just dialog quotations (fiction writing, English).

I hunted around various packages, and see that polyglossia has gloss-dutch, which sets some accented characters to be active. This is done withing a \def macro. It looked like just what I need. To my surprise, it works when I write the code not within a \def, but fails when I write the code within a \def (inaccessible). MWE, which works as-is:

\documentclass{article} % Compile using lualatex.
\usepackage{fontspec} % Less-minimal exaple would need this.
\defaultfontfeatures{} % Reset, to avoid tlig for this MWE.
\newcounter{dblq}
\def\shazam{} % Placeholder.

% To see the problem, uncomment/comment:

% Uncomment these 4 lines:
%\def\shazam{
%  \catcode`\"=13
%  \def "{”\stepcounter{dblq}}
%}

% And apply comment to these two lines:
\catcode`\"=13
\def "{”\stepcounter{dblq}}

\begin{document}
\shazam

"What in the world," he asked, "is happening here?"

\typeout{DBLQ=\arabic{dblq}}
\end{document}

Reason for lualatex is that I use only utf-8 with fontspec.

I have tried various random changes such as removing space, using \let, etc. No joy. Also fails with \AtBeginDocument.

Answer 1

setting \catcode affects the conversion of characters in the input to character tokens but it has no effect at all on character tokens.

In

\def\shazam{
  \catcode`\"=13
  \def "{”\stepcounter{dblq}}
}

the body of the command definition has been tokenized with a catcode 12 " so the \catcode setting does not affect this (but would affect following text) and \def " is a syntax error.

\catcode`\"=13
\def\shazam{
  \def "{”\stepcounter{dblq}}
}

defines the command with active " (even if " is not active at the point \shazam is used.

Answer 2

After digesting what David wrote, and re-inspecting the code for parts of babel and polyglossia, I was still mystified. Their code defined active characters within a macro, but in a manner that did not work for me. Understand that for reasons unrelated to the original question, I need to activate and define the character \AtBeginDocument or perhaps \AfterEndPreamble (etoolbox).

Then it dawned on me: The other code was loaded using \usepackage or \RequirePackage. Maybe those macros did something magical? Yes! So here is something that works, and may be useful to others:

1. Create file dblq.sty containing this line:

\ProvidesFile{dblq.sty}[2023/10/01 v0.01 LaTeX file (double quotes).]
\catcode`\"=13
\def "{”\stepcounter{dblq}}
\AtEndDocument{
  \catcode`\"=12
  \typeout{You used \arabic{dblq}\string ".}
}
\endinput

Then put it next to the document, or command-line mktexlsr to find it.

2. In the document class (or Preamble}:

\newcounter{dblq}
\AtBeginDocument{\RequirePackage{dblq}}

3. Here is another MWE that illustrates its usage. This assumes that file dblq.sty is already there:

\documentclass{article}
% \usepackage{fontspec} % If building with lualatex.
\def\doesNotCount{"Hello World," they said.}
\newcounter{dblq}
\AtBeginDocument{
  \def\uni#1{No\obeyspaces} % This MWE only, pdflatex.
  % \def\uni#1{{\char"#1}\obeyspaces} % uncomment for lualatex
  \RequirePackage{dblq}
}
\begin{document}
\def\doesCount{"Hello Again," they said.}
\doesNotCount\par
\doesCount\par
"Hello to you, too," they replied.\par
"Deise is sch\"one," said Hans.\par
Finally, "\uni{2116} kidding!" they shouted.\par
\end{document}

That correctly counts 8 doublequote. The ones in \doesNotCount are not counted, because they are written prior to activation. The active character does not affect \"o and is not counted. Although \uni might not be necessary, I use it for my own purposes, so I wrote it here.

I do not use babel/polyglossia shortcuts, so I do not know if this interferes with them.

EDIT: Modified per suggestion in comment.