Typesetting hashtags with natural syntax

Question

I'm using an online conversation in my document and I've made an environment to use for it. The thing is, there are many #hashtags in each post (that I'd like to color appropriately). To make it more readable, I want to make # an active character and take everything from the usage to either the next hash (#) or a space.

Help me, I'm #lost.

Help me, I'm #lost!

Best ever #tex #latex #plain_text

What? #confused#lost#savingspace

I suppose a regular expression that would match a hashtag could be as simple as #[\-_A-Za-z], so the complement would denote a break.

The 'either-or' aspect as too much to get my head around when trying to figure this out, but that's the ideal. I just decided to make # active. But when you think about it, there's no way you can have a macro argument for this. You don't have any characters with the appropriate catcode (6). So, use a different character, right? Unfortunately this doesn't work:

\documentclass{article}
\usepackage{xcolor}
\begin{document}
\newcommand\maketag[1]{\textcolor{cyan}{\##1}}
\newenvironment{post}[2]{%
  #1 said on #2:\quote
  \catcode`"=6
  \catcode`\#=13
  \def\#""1 {\maketag{""1}}%
}{%
  \catcode`"=12 % edit: wasn't needed at all..
  \catcode`\#=6 % #late_night_mistakes
  \endquote
}
\begin{post}{Sean Allred}{yesterday}
  #hello
\end{post}
\end{document}

ERROR: Undefined control sequence.

--- TeX said ---
l.16       #
            hello
--- HELP ---
TeX encountered an unknown command name. You probably misspelled the
name. If this message occurs when a LaTeX command is being processed,
the command is probably in the wrong place---for example, the error
can be produced by an \item command that's not inside a list-making
environment. The error can also be caused by a missing \documentclass
command.

How can I get the behavior I want?

I'm more than open to expl3 answers :)

Answer 1

\documentclass{article}
\usepackage{xcolor}
\usepackage[T1]{fontenc}
\begin{document}

\def\hashtag{%
\textcolor{cyan}{\#}%
\begingroup
\color{red}%
\xhashtag}

\def\xhashtag{\futurelet\tmp\xxhashtag}

\def\xyhashtag#1{\xhashtag}

\def\xxhashtag{%
\ifcat a\noexpand\tmp 
 \tmp\expandafter\xyhashtag
\else
  \endgroup
\fi}

\newenvironment{post}[2]{%
  #1 said on #2:\quote
  \catcode`\#\active
  \catcode`\-11 %
  \catcode`\_11 %
  \lccode`\~`\#%
  \lowercase{\let~}\hashtag
}{%
  \endquote
}
\begin{post}{Sean Allred}{yesterday}
Help me, I'm #lost.

Help me, I'm #lost!

Best ever #tex #latex #plain_text

What? #confused#lost#saving-space
\end{post}
\end{document}

Answer 2

There are at least a couple of ways you could do this: catcode changes or the \lowercase 'trick'. As you've started off with catcodes, I'll stick with that. What you need to remember is you are tokenizing material when you do \newenviroment. As such, you need to change catcodes before the definition:

\documentclass{article}
\newcommand\maketag[1]{\##1}
\catcode`\"=6 %
\catcode`\#=\active
\newenvironment{post}{%
  \def#""1 {\maketag{""1}}%
  \catcode`\#=\active
  \quote
}{%
  \endquote
}
\catcode`\"=12 %
\catcode`\#=6 %

\begin{document}
\begin{post}
  #hello
\end{post}
\end{document}

Notice that inside the environment we only need to change the catcode of ": we need the definition to have " as the parameter char, but not to read the environment. It's only # that has special handling here. Also notice that as environments form groups there is no need to worry about resetting catcodes in the \endpost macro (end-of-environment argument).

For contrast, the \lowercase approach would look like

\documentclass{article}
\newcommand\maketag[1]{\##1}
\newenvironment{post}{%
  \begingroup
    \lccode`\~=\#
    \lowercase{%
      \endgroup
      \def~##1}{\maketag{##1}}%
  \catcode`\#=\active
  \quote
}{%
  \endquote
}

\begin{document}
\begin{post}
  #hello
\end{post}
\end{document}

Here, I don't have to mess with catcodes beyond making # active inside the environment. The idea here is that ~ is active anyway, so I can lower-case it into a # (which remains active) while using a 'normal' # for setting up the definition.

By the way, notice that in both cases we need to double the parameter char here as #1 (standard catcodes) refers to any argument for the environment itself. I didn't understand why you'd got [2] for arguments to that, so I dropped it!

Answer 3

Customizable at will (but of course rather slow):

\documentclass{article}
\usepackage{xcolor,environ,xparse,l3regex}

\ExplSyntaxOn
\NewDocumentEnvironment{post}{mm}
 {
  \char_set_catcode_other:N \# % change the catcode not to confuse \innerpost
  \char_set_catcode_other:N \_
  #1~said~on~#2\tl_to_str:n {:}\quote
  \innerpost % absorb the contents
 }
 {
  \endinnerpost\endquote
 }
\NewEnviron{innerpost}
 {
  \regex_replace_all:nnN
   { \#([[:word:]\-]*) }
   { \c{texttt}\cB\{\c{coloredhash}\1\cE\} }
   \BODY

  \BODY
 }
\ExplSyntaxOff

\newcommand\coloredhash{\textcolor{cyan}{\#}}

\begin{document}

\begin{post}{Sean Allred}{yesterday}
Help me, I'm #lost.

Help me, I'm #lost!

Best ever #tex #latex #plain_text

What? #confused#lost#saving-space
\end{post}

\end{document}

An example of further customization, suppose you want to accept accented characters in the hashtags, with \usepackage[utf8]{inputenc}; it's sufficient to change the search regular expression into

{ \#((\c[LA][^\~]|\_|\-)*) }

which matches any combination of letters, active characters (except for ~), the hyphen and the underscore.

A solution based on \ifcat would need much more tests.

Answer 4

There are two problems here. The catcode changing (which was solved in previous answers) and your wish that the argument of hash ends by "space OR hash". Your example:

What? #confused#lost#savingspace

I guess that this second problem isn't solved in previous answers. It could be solved in general, see this thread. But we needn't so complicated code in this special case. We suppose, that the space definitely occurs after the sequence of #aa#bb#cc#etc. The whole sequence would be the argument of the first hash. We need to deactivate the internal hashes in such parameter only.

Now, the implementation:

\def\sethash#1{\lccode`\~=`\#\lowercase{\let~=#1}}
\def\maketag#1 {{\color{cyan}\sethash\#\##1}} % macro parameter separated by space

\newenvironment{post}{%
     \sethash\maketag \catcode`\#=\active     
  \quote
}{%
  \endquote
}

I hope, you can add another LaTeX specific code (like \documentclass etc).

Answer 5

This short code sets “#” active inside the environment, and defines # to be (in this case) equal to \# (chek the handy functions \char_set_active:Npn or \char_set_eq:NN).

\documentclass{scrartcl}

\usepackage{xparse}

\ExplSyntaxOn
\NewDocumentEnvironment { post } { }
  { \quote \char_set_active_eq:NN \# \# }
  { \endquote }
\ExplSyntaxOff

\begin{document}
\begin{post}
  #hello
\end{post}
\end{document}

Now the deeper question on how to handle the “arguments”. One option was swallowing tokens until finding a non-letter (finally answered by David). The other option was l3regex (egreg made a similar approach). In this case, since _ is non-letter (subscript), this doesn't handle #something_like_this, but you can change the catcode whenever you want.

\documentclass{scrartcl}

\usepackage[T1]{fontenc}
\usepackage{environ,l3regex,xparse,xcolor}

\ExplSyntaxOn
\NewEnviron { post }
  {
   \tl_set:No \l_tmpa_tl { \BODY }
   \regex_replace_all:nnN
    { (\#)([\w\-]+) } { \c{textcolor}\cB\{cyan\cE\}\cB\{\c{\#}\2\cE\} }
    \l_tmpa_tl
   \tl_use:N \l_tmpa_tl
  } 
\ExplSyntaxOff

\begin{document}
\begin{post}
  #hello #ratata And not a hashtag, and a
  #hashtag#another#another with #hyphens-but
  (\string_ doesn't work)
\end{post}
\end{document}

Answer 6

simply using the backslash, the hash sign, and the word in curly brackets worked for me:

\#{MeToo}

produced

#MeToo