The package inputenc makes it possible, for example, to write ° to print a degree symbol. Simplifying a bit, it obtains that by making it an active character (catcode 13) and defining the corresponding macro. If its category is changed to other (catcode 12) and, then, changed back with \scantokens, it keeps printing the degree symbol.
Since I need to print a string, a character at a time, spaces included, my first idea was of using \let:
\documentclass{article}
\usepackage[ansinew]{inputenc}
\usepackage[T1]{fontenc}
\begin{document}
\setlength\parindent{0pt}
A°@ 1
{\catcode`\°12 \xdef\str{A°@ 1}}
\str
\scantokens\expandafter{\str\empty}
\def\aaa{\afterassignment\bbb\let\ccc= }
\def\bbb{%
\ifx\ccc\nil
END
\let\next\relax
\else
[\scantokens\expandafter{\ccc\empty}]\let\next\aaa
\fi
\next
}
\expandafter\aaa\str\nil
\end{document}
whose output is:
Unfortunately, as jfbu pointed out, \scantokens does nothing on the \let character and so it prints [ř] instead of [°]. To make it apparent, consider the following:
\documentclass{article}
\usepackage[ansinew]{inputenc}
\usepackage[T1]{fontenc}
\begin{document}
\setlength\parindent{0pt}
{\catcode`\°12 \gdef\ddd{°} \global\let\eee°}
[\ddd] - [\scantokens\expandafter{\ddd\empty}]
[\eee] - [\scantokens\expandafter{\eee\empty}]
\end{document}
whose output is:
Of course, in the first example, one possible solution would be to substitute the \scantokens in square brackets with \ifx\ccc\ddd°\else\ccc\fi, having taken care to add {\catcode`\°12 \global\let\ddd°} at the beginning. The problem with this approach is that it is not very 'scalable'.
My real goal here is to parse some (CP-1252 encoded) files, which can contain any characters, not just °. To put this in context and for the sake of simplicity, let us just suppose I am coding a hex viewer in (La)TeX (which, by the way, would not be such a bad idea).
So, for the textual part, I first 'load' the file with something like \edef\fff#1{\pdfunescapehex{\pdffiledump length \pdffilesize{#1}{#1}}}, which gives all character tokens with catcode 12, except the space (that has catcode 10); then I scan such a token list and act as needed, printing it or else.
So, is there a way to scan a token list, including spaces, without using \let? Assigning catcode 12 also to spaces would suffice, can I do it? How? Alternatively, is there a way of changing the catcode after a \let, which does not involve a lot of conditional expressions? If not, what would be the best (compact and/or elegant) way to do it?
°is not a single token? – Manuel Nov 23 '17 at 20:21ansinew(cp1252` input encoding. This is notutf8. – user4686 Nov 23 '17 at 21:48[ansinew](which doesn't refer to any standard encoding) if the encoding of the text changes (for example as posted above it is in UTF-8) then the code will fail as ° may be multiple tokens. – David Carlisle Nov 23 '17 at 21:54