# Command to display free-text answers from a survey, eliminating ascii control chars

I'd like to create a report from a survey with free-text answers that is collected in a database. For each question there are typically many answers which I could present in an itemize environment, one answer per item. The problem I have is that an individual answer (in Latin1 encoding) might contain double spaces, returns and other control characters that should not matter in the presentation. Is there a suitable environment or command (implemented probably using catcode trickery) that smoothes these out, resulting in a nice even presentation?

\answer{Code with  double spaces, returns, linefeeds and other control character that should not matter here}


An alternative would be to filter all these out using the software that fetches the data from a database and generates the LaTeX code. However, I'd prefer a TeX/LaTeX solution. I'd be also interested in other creative solutions.

-
Not really an answer since I do not know enough to attempt the code, but LuaTeX might help here. It has the formatting of TeX plus the programming capabilities of a scripted language. – Matthew Leingang Feb 2 '11 at 18:46
Sorry for misunderstand the question. (I'm too uncareful and I'm not native English speaker.) So I deleted that wrong answer. It might be easier to write a program (even in C) other than TeX to do this. Is it needed to convert & to \&, % to \% and so on? – Leo Liu Feb 2 '11 at 20:57

I would suggest to simply set all ASCII control characters (0-31) to catcode 9 (ignored) etc.:

\def\mycatcodes{%
% Control characters ASCII #0-31:
% Catcode 9 = "ignore"
\catcode0=9
\catcode1=9
\catcode2=9
\catcode3=9
...
\catcode30=9
\catcode31=9
% and DEL:
\catcode127=9
%
% You might want the line-end character to be a space:
\catcode13=10
% Might be also required:
\endlinechar=-1
%
% Verbatimise all special TeX characters:
\let\do\@makeother
\dospecials
%
% Let Spaces be spaces again:
\catcode\ =10
}

\begingroup
\mycatcodes
}{}%
\begingroup
\catcode\|=0
\catcode$$=1 \catcode$$=2
\@makeother\{
\@makeother\}
\@makeother\\
|endgroup % end catcode changes
)
|endgroup
% Now a third macro is used which doesn't has the hassle with catcode changes
% Does the typesetting and formatting of the answer, e.g.
\texttt{#1}%
% Note, that not all verbatim character (like _) are correctly displayed
% by all fonts
}


In the document:

\begin{answer}
Test with possible control characters (ignored) and LaTeX special characters
incl. { and }.


As you already said, a filter script (e.g. Perl) might be also a good possibility.

Edit: Turned it into a environment

-
Thanks! Two questions: (1) should I use #1 before the \endgroup? I'd guess so, unlike what the comment suggests. (2) It's not possible to simply inline the body of @answer into \answer? Or put in another way: the split up into two commands is deliberate? – Christian Lindig Feb 2 '11 at 18:42
@Christian: (1) NO! The \endgroup should be the first thing. The group doesn't affect the substitution of arguments. The #1 is inserted before the macro is executed. (2) No again, the split is required because otherwise the text would be already read before the catcode changes are in effect! AFAIK the code above is "the standard way" to do catcode macros. – Martin Scharrer Feb 2 '11 at 18:55
Thanks again for this valuable explanation! +1 – Christian Lindig Feb 2 '11 at 19:06
@Christian: I turned it into an environment, so everything except "\end{answer}" can now appear in the text. – Martin Scharrer Feb 2 '11 at 20:49

Here is how this could be done in ConTeXt.

\def\startignore
{\dostartbuffer[ignore][startignore][stopignore]}

\def\stopignore
{\pushcatcodetable
\setcatcodetable\nilcatcodes
\getbuffer[ignore]
\popcatcodetable}

\setupbodyfont[cambria]
%Any unicode font that contains greek used in the example will do

\starttext

\startignore
Line
feeds
and multiple   spaces are ignored. The text may contain \undefined \TeX
commands or  or ' or _ or ^ or ωθατενερ
\stopignore

\stoptext
`

In principle, the same idea will work in LaTeX: capture the content of the environment verbatim, set nil catcodes (basically just set the catcode of space, end of line, form feed, tab, and end of file to their usual values; everything else is set to other), and retrieve the contents of the environment.

-