# An environment which parses periods differently to redefine their behavior

I have a collection of documents that I want to read into a LaTeX environment. The problem is that these documents are not exactly formatted the way I would like. I would like to create an environment which for the duration of the environment changes the way LaTeX parses . at the end of a sentence. Essentially, I would like something which looks like:

\begin{mycontent}
random content.  more random content.  some more random content.
\end{mycontent}


and parses it as though I had written

\begin{mycontent}
random content.\formattingcommand  more random content.\formattingcommand  some more random content.\formattingcommand
\end{mycontent}


I don't have control over the content of the files getting passed to me, but I am guaranteed that the files are all structured the same. I could write a Perl script to handle this, and normally I would. But this is something that needs some kind of LaTeX wrapper that others can use without invoking Perl and works seamlessly for them.

It would be nice if I could write something like the following:

 \let\oldperiod{.}
\def.{\oldperiod\formattingcommand}%


But . is not a command. I've looked in the Knuth's The TeXbook in the section on sentences, but I don't see anything there I can immediately use.

I suppose I could do something that changes the cat code for . for the duration of the environment, but I need the old period to show up within this new environment. I'm not really sure how to go this direction and would prefer not to. But, I'm willing to be flexible on this point (particularly if someone's got something really cool to show.)

Edit 1

On the subject of catcodes: I've never really worked with catcodes before. So I'm kind of blind going down that alley on my own right now.

Edit 2

If I follow JosephWright's suggestion, I might worry that some other package I load might now have a clash in how an active period will be treated. Is there a reason the following code has not been suggested:

\documentclass{article}
\newcommand{\formattingcommand}{DEMO}
\newenvironment{mycontent}
{%
\let\normalperiod=.%
\catcode\.=\active
\def.{\normalperiod\formattingcommand}%
}
{}
\begin{document}
\begin{mycontent}
random content.  more random content.  some more random content.
\end{mycontent}
\end{document}


I ask, because it would seem to me that this version should work, but when I try it I get an error:

! Missing control sequence inserted.
<inserted text>
\inaccessible
l.11 \begin{mycontent}

?


Edit 3

Based upon everyone's feedback below, would the following be safe?

\documentclass{article}
\newcommand{\formattingcommand}{DEMO}
\let\normalperiod=.%
\chardef\periodcatcode=\catcode\.
\catcode\.=\active
\newenvironment{mycontent}
{%
\catcode\.=\active%
\def.{\normalperiod\formattingcommand}%
}
{}
\catcode\.=\periodcatcode
\begin{document}
\begin{mycontent}
random content.  more random content.  some more random content.
\end{mycontent}
\end{document}


By safe I mean:

1. \let saves the correct version of period (period with the expected catcode): assuming I haven't done anything else wonky with catcodes earlier.
2. I've saved the current catcode before making it \active
3. I've made . have an \active cat code to ensure that solely within the scope of my environment it has the \active definition I'm giving it. In other words, my definition is local in scope and won't effect definitions outside of its scope.
4. I've restored the catcode of . to what it was before creating my new environment without having to second-guess what that might have been.
5. If other packages wish to redefined an active ., my tinkering is invisible to them (assuming I'm not using any code related to those packaged within my environment).
-
As I (imperfectly) understand it, this is exactly what catcodes are for. – Mohan Dec 30 '12 at 21:27
But if I change the catcode, I still nevertheless need the period to show up. How do I get the period there? – A.Ellett Dec 30 '12 at 21:28
In exactly the way you suggest in your examples -- you save the old period using let. (I'm not sure the syntax is quite right, though.) You should read this: tex.stackexchange.com/questions/258/… – Mohan Dec 30 '12 at 21:44
@Mohan In this case it shouldn't make any difference; although I too would use \def\normalperiod{.}; the syntax \let\oldperiod{.} is definitely wrong and it should be \let\oldperiod=. (the = is optional). – egreg Dec 30 '12 at 22:11
The code in "edit 3" is safe, but I wouldn't recommend it. I believe that the \lowercase method is to be preferred, if you don't want to assign a global meaning to the active period, but it's personal preference. – egreg Dec 30 '12 at 23:22

You are on the right track with catcode changes. The key is to save the old definition of . with another name before applying the change. The standard approach would look something like

\documentclass{article}
\begingroup
\catcode\.=\active
\gdef.{\normalperiod\formattingcommand}%
\endgroup
\newcommand{\formattingcommand}{DEMO}
\newenvironment{mycontent}
{%
\let\normalperiod=.%
\catcode\.=\active
}
{}
\begin{document}
\begin{mycontent}
random content.  more random content.  some more random content.
\end{mycontent}
\end{document}


where I've used a group to avoid needing to know what catcode . has more generally.

You might be surprised by the global definition of . here. The 'trick' is that the category code of . is only changed locally, and the definition only gets applied where where . is active.

As with anything relying on catcode changes, this will only work correctly if the 'content' has not already been read by TeX ('tokenized'). As such, don't use it if you content will have been used in the argument of some command. Also note that babel uses active characters a lot, and there is a risk of a clash if it is in use.

The reason that

\newenvironment{mycontent}
{%
\let\normalperiod=.%
\catcode\.=\active
\def.{\normalperiod\formattingcommand}%
}
{}


fails is that when TeX reads the code, . is not active, and so \def. is incorrect (\def must be followed either by an active character or a macro name). Th \newenvironment command has read all of the 'start of environment' code but not executed it, so the fact it contains a \catcode line has no influence on the \def that follows.

You could avoid this in a couple of ways, for example

\catcode\.=\active
\newenvironment{mycontent}
{%
\let\normalperiod=.%
\catcode\.=\active
\def.{\normalperiod\formattingcommand}%
}
\catcode\.=12 % Assuming the usual set up: could be risky


or

\newenvironment{mycontent}
{%
\let\normalperiod=.%
\catcode\.=\active
\lccode\~=.%
\lowercase{%
\def~{\normalperiod\formattingcommand}%
}%
}


which uses the fact that ~ is active in LaTeX documents, and that \lowercase preserves catcodes.

Of the two alternative approaches above, I would prefer the second (using \lowercase) over the one making . active for the entire definition of mycontent. If another package makes . active in some limited context, it would be possible for

\catcode\.=\active
\newenvironment{mycontent}
{%
\let\normalperiod=.%


to pick up a definition for an active . which was not actually correct (if in the general document . is not active).

-
Isn't \gdef redefining . globally? I don't want . to change it's behavior outside of these environments. – A.Ellett Dec 30 '12 at 21:32
@A.Ellett That defines globally the active period. When it has category code 12 that meaning is irrelevant. – egreg Dec 30 '12 at 21:33
@egreg. Wow! I'm really in over my head. The way I read this code, it looks like . has been made active for my entire document. How it that this is not happening? – A.Ellett Dec 30 '12 at 21:35
@A.Ellett the catcode assignment is local, only the definition of the active . is global. – clemens Dec 30 '12 at 21:38
@A.Ellett It is activated after \begingroup, so the category code assignment will end with \endgroup. But the definition of the active period is global, so it survives the group and it will be used whenever an active period is found (for instance in the mycontent environment). – egreg Dec 30 '12 at 21:38

There are a few problems to be aware of when dealing with category codes and active characters in particular.

1. When character tokens have already entered TeX, their category code is fixed (unless \scantokens is used, but this would open a can of worms).

2. If an active character somehow sneaks into a list for \write, there must be a definition for it.

3. babel makes some characters active, but delays doing so at \begin{document}

Assuming that making . active and defining its meaning to be "print a period and execute \formatcommand" solves your problem (and you haven't anything that could sneak in a \write such as a label containing a period), let's see a couple of alternatives.

### Method 1

\begingroup
\catcode\.=\active
\gdef.{\normalperiod\formatcommand}
\endgroup
\newcommand\normalperiod{.}
\newcommand\formatcommand{whatever}
\newenvironment{mycontent}
{\catcode\.=\active}
{}


This method is the easiest one. But it would break if babel is loaded with a language that makes the period active, or if some other package does the same.

### Method 2

\newcommand\formatcommand{whatever}
\edef\normalperiod{\string.}
\newenvironment{mycontent}
{\begingroup\lccode~=\.
\lowercase{\endgroup\def~}{\normalperiod\formatcommand}%
\catcode\.=\active}
{}


You still can't use the period inside mycontents for things that go into a \write, but this is a common limitation with every method.

This method is quite safe with babel, but of course it would override the meaning it may give to the active period. The \edef used for \normalperiod ensures that its replacement text is a normal category code 12 period, so if some other previously loaded package changes the category code, you're safe against it.

-
@JosephWright and egreg I find this \lowercase route interesting, but also perplexing. It looks more like a hack to me. Can someone point me to other answers on this site or references I can read to better appreciate this approach? – A.Ellett Dec 30 '12 at 23:59
@A.Ellett I forgot the main assignment in method 2. :( – egreg Dec 31 '12 at 0:02
YEAH! Works! Perfect. Now to read up on this solution and learn its magic. :) – A.Ellett Dec 31 '12 at 0:03
You can look here for an explanation of the \lowercase trick. – egreg Dec 31 '12 at 0:07
I'm not sure where to put this comment. My understanding of \lowercase{\endgroud\def~} is that \lowercase parses its argument and sends \endgroup\def. to the input stream. This is somehow done while TeX sees an active token after \def to stop TeX from complaining about . being currently not active. But the token \endgroup has not yet been acted upon essentially allowing us to smuggle in . to where it normal wouldn't be accepted and appropriately limiting the scope of \lccode. Am I essentially correct? – A.Ellett Dec 31 '12 at 0:41