# Typesetting hashtags with natural syntax

I'm using an online conversation in my document and I've made an environment to use for it. The thing is, there are many #hashtags in each post (that I'd like to color appropriately). To make it more readable, I want to make # an active character and take everything from the usage to either the next hash (#) or a space.

Help me, I'm #lost.

Help me, I'm #lost!

Best ever #tex #latex #plain_text

What? #confused#lost#savingspace

I suppose a regular expression that would match a hashtag could be as simple as #[\-_A-Za-z], so the complement would denote a break.

The 'either-or' aspect as too much to get my head around when trying to figure this out, but that's the ideal. I just decided to make # active. But when you think about it, there's no way you can have a macro argument for this. You don't have any characters with the appropriate catcode (6). So, use a different character, right? Unfortunately this doesn't work:

\documentclass{article}
\usepackage{xcolor}
\begin{document}
\newcommand\maketag[1]{\textcolor{cyan}{\##1}}
\newenvironment{post}[2]{%
#1 said on #2:\quote
\catcode"=6
\catcode\#=13
\def\#""1 {\maketag{""1}}%
}{%
\catcode"=12 % edit: wasn't needed at all..
\catcode\#=6 % #late_night_mistakes
\endquote
}
\begin{post}{Sean Allred}{yesterday}
#hello
\end{post}
\end{document}

ERROR: Undefined control sequence.

--- TeX said ---
l.16       #
hello
--- HELP ---
TeX encountered an unknown command name. You probably misspelled the
name. If this message occurs when a LaTeX command is being processed,
the command is probably in the wrong place---for example, the error
can be produced by an \item command that's not inside a list-making
environment. The error can also be caused by a missing \documentclass
command.


How can I get the behavior I want?

I'm more than open to expl3 answers :)

## 6 Answers

\documentclass{article}
\usepackage{xcolor}
\usepackage[T1]{fontenc}
\begin{document}

\def\hashtag{%
\textcolor{cyan}{\#}%
\begingroup
\color{red}%
\xhashtag}

\def\xhashtag{\futurelet\tmp\xxhashtag}

\def\xyhashtag#1{\xhashtag}

\def\xxhashtag{%
\ifcat a\noexpand\tmp
\tmp\expandafter\xyhashtag
\else
\endgroup
\fi}

\newenvironment{post}[2]{%
#1 said on #2:\quote
\catcode\#\active
\catcode\-11 %
\catcode\_11 %
\lccode\~\#%
\lowercase{\let~}\hashtag
}{%
\endquote
}
\begin{post}{Sean Allred}{yesterday}
Help me, I'm #lost.

Help me, I'm #lost!

Best ever #tex #latex #plain_text

What? #confused#lost#saving-space
\end{post}
\end{document}

• Awesome! I don't have a TeX environment to test with, but I from what I understand from reading the code (it's a little bit, ehhh....advanced ;)), this solution would handle This is a #test hashtag => This is a \magic{#test} hashtag, yes? – Sean Allred Sep 17 '14 at 16:23
• @SeanAllred probably:-) – David Carlisle Sep 17 '14 at 18:25

There are at least a couple of ways you could do this: catcode changes or the \lowercase 'trick'. As you've started off with catcodes, I'll stick with that. What you need to remember is you are tokenizing material when you do \newenviroment. As such, you need to change catcodes before the definition:

\documentclass{article}
\newcommand\maketag[1]{\##1}
\catcode\"=6 %
\catcode\#=\active
\newenvironment{post}{%
\def#""1 {\maketag{""1}}%
\catcode\#=\active
\quote
}{%
\endquote
}
\catcode\"=12 %
\catcode\#=6 %

\begin{document}
\begin{post}
#hello
\end{post}
\end{document}


Notice that inside the environment we only need to change the catcode of ": we need the definition to have " as the parameter char, but not to read the environment. It's only # that has special handling here. Also notice that as environments form groups there is no need to worry about resetting catcodes in the \endpost macro (end-of-environment argument).

For contrast, the \lowercase approach would look like

\documentclass{article}
\newcommand\maketag[1]{\##1}
\newenvironment{post}{%
\begingroup
\lccode\~=\#
\lowercase{%
\endgroup
\def~##1}{\maketag{##1}}%
\catcode\#=\active
\quote
}{%
\endquote
}

\begin{document}
\begin{post}
#hello
\end{post}
\end{document}


Here, I don't have to mess with catcodes beyond making # active inside the environment. The idea here is that ~ is active anyway, so I can lower-case it into a # (which remains active) while using a 'normal' # for setting up the definition.

By the way, notice that in both cases we need to double the parameter char here as #1 (standard catcodes) refers to any argument for the environment itself. I didn't understand why you'd got [2] for arguments to that, so I dropped it!

• You could of course simply use a verbatim environment: I'm guessing that's not what you are after! – Joseph Wright Sep 17 '14 at 6:34
• Is it possible to define \makehashtagon (like \makeatletter) and define \newenvironment{post}{\makehashtagon}{\makehashtagoff}? – Symbol 1 Sep 17 '14 at 6:43
• @Symbol1 It's of course a bit more complex as \makeatletter only alters the catcode of @ but as it's not active there is no definition. You could wrap up either version of the above, with the result that # would grab an argument and apply \maketag, for example. As with the above, you wouldn't need \makehashtagoff as the environment forms a group. – Joseph Wright Sep 17 '14 at 6:49
• Forget \makehashtagoff :P I realize that I have to change catcode of # before I define \makeatletter, right? – Symbol 1 Sep 17 '14 at 6:52
• You forgot a third way: \let#\maketag would avoid the acrobatics with \catcode\"=6 in the first solution (and would shorten the second one). – egreg Sep 17 '14 at 9:06

Customizable at will (but of course rather slow):

\documentclass{article}
\usepackage{xcolor,environ,xparse,l3regex}

\ExplSyntaxOn
\NewDocumentEnvironment{post}{mm}
{
\char_set_catcode_other:N \# % change the catcode not to confuse \innerpost
\char_set_catcode_other:N \_
#1~said~on~#2\tl_to_str:n {:}\quote
\innerpost % absorb the contents
}
{
\endinnerpost\endquote
}
\NewEnviron{innerpost}
{
\regex_replace_all:nnN
{ \#([[:word:]\-]*) }
{ \c{texttt}\cB\{\c{coloredhash}\1\cE\} }
\BODY

\BODY
}
\ExplSyntaxOff

\newcommand\coloredhash{\textcolor{cyan}{\#}}

\begin{document}

\begin{post}{Sean Allred}{yesterday}
Help me, I'm #lost.

Help me, I'm #lost!

Best ever #tex #latex #plain_text

What? #confused#lost#saving-space
\end{post}

\end{document}


An example of further customization, suppose you want to accept accented characters in the hashtags, with \usepackage[utf8]{inputenc}; it's sufficient to change the search regular expression into

{ \#((\c[LA][^\~]|\_|\-)*) }


which matches any combination of letters, active characters (except for ~), the hyphen and the underscore.

A solution based on \ifcat would need much more tests.

• A doubt, what does [[:word:]] do? Is it the same as \w? – Manuel Sep 17 '14 at 14:20
• @Manuel I think it's the same, but clearer: letters, digits and underscore. – egreg Sep 17 '14 at 14:28

There are two problems here. The catcode changing (which was solved in previous answers) and your wish that the argument of hash ends by "space OR hash". Your example:

What? #confused#lost#savingspace


I guess that this second problem isn't solved in previous answers. It could be solved in general, see this thread. But we needn't so complicated code in this special case. We suppose, that the space definitely occurs after the sequence of #aa#bb#cc#etc. The whole sequence would be the argument of the first hash. We need to deactivate the internal hashes in such parameter only.

Now, the implementation:

\def\sethash#1{\lccode\~=\#\lowercase{\let~=#1}}
\def\maketag#1 {{\color{cyan}\sethash\#\##1}} % macro parameter separated by space

\newenvironment{post}{%
\sethash\maketag \catcode\#=\active
\quote
}{%
\endquote
}


I hope, you can add another LaTeX specific code (like \documentclass etc).

• I wouldn't leave the \lccode setting for ~ in the wild. – egreg Sep 17 '14 at 13:53

This short code sets “#” active inside the environment, and defines # to be (in this case) equal to \# (chek the handy functions \char_set_active:Npn or \char_set_eq:NN).

\documentclass{scrartcl}

\usepackage{xparse}

\ExplSyntaxOn
\NewDocumentEnvironment { post } { }
{ \quote \char_set_active_eq:NN \# \# }
{ \endquote }
\ExplSyntaxOff

\begin{document}
\begin{post}
#hello
\end{post}
\end{document}


Now the deeper question on how to handle the “arguments”. One option was swallowing tokens until finding a non-letter (finally answered by David). The other option was l3regex (egreg made a similar approach). In this case, since _ is non-letter (subscript), this doesn't handle #something_like_this, but you can change the catcode whenever you want.

\documentclass{scrartcl}

\usepackage[T1]{fontenc}
\usepackage{environ,l3regex,xparse,xcolor}

\ExplSyntaxOn
\NewEnviron { post }
{
\tl_set:No \l_tmpa_tl { \BODY }
\regex_replace_all:nnN
{ (\#)([\w\-]+) } { \c{textcolor}\cB\{cyan\cE\}\cB\{\c{\#}\2\cE\} }
\l_tmpa_tl
\tl_use:N \l_tmpa_tl
}
\ExplSyntaxOff

\begin{document}
\begin{post}
#hello #ratata And not a hashtag, and a
#hashtag#another#another with #hyphens-but
(\string_ doesn't work)
\end{post}
\end{document}

• +1 for the cleanest solution. \maketag can be easily defined with u{~} as the argspec (if you disregard the alternate #/~ requirement) :) – Sean Allred Sep 17 '14 at 11:43
• I think [A-Za-z_]* is the actual regex for most applications, so the complement would indicate a break. – Sean Allred Sep 17 '14 at 11:47
• Well yes, I'm making certain assumptions to avoid the recursion, but I think there is a way (at least in e-TeX) to scan ahead looking for a token (potentially a group of tokens), so there might be an expl3 analog. You could then have something like \NewDocumentCommand\maketag{m}{…}\cs_new:Npn\hashtags_scan_and_make:{…\maketag{…}} – Sean Allred Sep 17 '14 at 11:50
• @SeanAllred Sorry, I don't understand :( – Manuel Sep 17 '14 at 11:51
• See the edit to my comment. I'll try to figure a solution during lunch today and edit your post if that's okay with you :). – Sean Allred Sep 17 '14 at 11:53

simply using the backslash, the hash sign, and the word in curly brackets worked for me:

\#{MeToo}


produced

#MeToo

• Thanks for your answer and welcome to the site :-) This is the trivial approach and does work, but note though that in my original question, "I want to make # an active character and take everything from the usage to either the next hash (#) or a space." – Sean Allred Jun 12 '18 at 17:41
• I'll note that the braces should be unnecessary. You may need to have a space like so: \# MeToo given TeX's tokenization rules. Brackets effectively do the same thing, but look a little awkward to my biased eyes. – Sean Allred Jun 12 '18 at 17:42