# Use the catchfile package to preprocess an input file character by character?

Is it possible to read a file using the \CatchFileDef or \CatchFileEdef commands from the catchfile package and then output its exact copy to another file? What about input and output encoding? Do catchfile and filehook play well with each other?

If yes: The intention behind that is to convert spaces to newlines and add auxiliary information (moving towards Forward-inverse search using SyncTeX with the precison of a single word). How would a (La)TeX implementation of such a char-by-char analysis look like, and would it scale well for large files?

I am looking for a solution that can be used with current pdflatex, perhaps --shell-escape with a small Perl script would perform just as well? I'm open to suggestions.

-
What if the input file contains a verbatim environment? – egreg Jun 14 '12 at 12:50
The contents of the input file should not be interpreted at all by LaTeX. They are to be copied, well, almost verbatim, to another file. – krlmlr Jun 14 '12 at 12:57
I understand what you'd like to do; the problem is that if you rewrite a file to have one word per line, a verbatim environment in an input file will be rendered incorrectly. – egreg Jun 14 '12 at 13:03
@egreg: Right, I missed that point. Care to edit the CW in the linked question? – krlmlr Jun 14 '12 at 13:23

Is it possible to read a file using the \CatchFileDef or \CatchFileEdef commands from the catchfile package and then output its exact copy to another file?

No. Package catchfile uses TeX's \input that reads the file line by line:

• Line end markers are removed.
• Space characters at the end of the line are removed (regardless of catcode).
• If \endlinechar contains a valid character code, then that character is added as end line marker. The valid range is 0 to 255 in TeX or pdfTeX. The range is greater in XeTeX or LuaTeX.

Because of the changes at the line ends, exact copies are not possible in general.

Also writing is a problem. \write adds an end of line marker. Depending on the TeX compiler and its setup some characters might be written in ^^xx-notation with xx as lowercase hexadecimal digits.

If pdfTeX is used, binary files can at least be read and analyzed. The expandable primitive \pdffiledump reads some bytes at the specified offset and length and expands to a hex dump.

What about input and output encoding?

Package stringenc provides conversions from one encoding to another. For example, you can read a file in latin-1 and write as UTF-8.

Do catchfile and filehook play well with each other?

They have no common playground. Package catchfile needs and uses the expandable primitive \input that is saved as \@@input in LaTeX. Package filehook does not redefine \@@input. This would be impossible in TeX without restrictions. A file name can be terminated by a space token or an non-expandable token that is not a character. The file name cannot read by delimited argument, because it is not known, how the file name is terminated. Undilimited parameters cannot catch space tokens, because TeX ignores them in the process of finding the parameters. The remaining peek ahead feature \futurelet destroys the expandability. Therefore package filehook hooks into the many higher level input macros of LaTeX.

The packages catchfile and filehook do not disturb or know each other.

The intention behind that is to convert spaces to newlines and add auxiliary information (moving towards Forward-inverse search using SyncTeX with the precison of a single word). How would a (La)TeX implementation of such a char-by-char analysis look like, and would it scale well for large files?

If you are just converting text files, then I do not see that you need exact copies. Then TeX's changes when reading a line might be acceptable. An alternative for package catchfile is reading line by line with \openin and \read. In the former case the whole file is read into the memory, the latter case reads the file line by line and does scale better for larger files.

-