I'm working on a python script that will compile a series of latex documents according to a work list. Basically, the script reads the file name and title of a document, adds a couple of things that should always be included and sends a pdflatex call to command line. The command has the following form:

>pdflatex -job-name="The title" "\somestuff\input{thefile}"

(I'm using a couple of options for output directories as well, but they have been ommitted here for brevity...)

Now, I have a couple of documents in the list that need to have special characters in the title. When that happens, the file names are corrupted. If I loop through the file names with python, reading them as utf8 shows the same thing as in my Windows Explorer window, but reading them as latin1 gives the correct names.

Currently ÅÄÖåäö are the only characters in question, but I'd like support for the entire utf8 charset. Is there any way I can make pdflatex encode these file names correctly?

I am moving the entire system to a server system with another OS when I'm done with it, so it's possible I'll need to encode these things in some other encoding then. Therefore, a solution that is encoding generic gets bonus points =)

  • 2
    Did you check the encoding used by your Python script? TeX see the jobname as a bunch of bytes and doesn't reencode it in any way. So, if the name you pass it is correctly encoded, the output name should be correctly encoded too. – mpg Nov 13 '10 at 1:42
  • You could use a dummy name and then rename the file using Python. – Aditya Nov 15 '10 at 1:13

What you want is currently not possible. Microsoft's implementation of the C standard library (which is what pdfTeX ultimately uses) never uses UTF-8 for the standard C functions. Do as Aditya suggests and use only ASCII characters for all file names when calling TeX.

Reading the file names in Python works because the Python runtime circumvents the C standard library on Windows. Working with Unicode file names on Windows always requires either a compatibility layer such as Cygwin or Windows-specific code, and I'm not aware that any of those is implemented by any engine (but it could be a valuable suggestion for LuaTeX).


There are four pieces of software whose character encoding might be relevant here:

  1. Windows Explorer you seem to have nailed down;
  2. Python, which should assume its input is UTF8. But in any case, if all it does is output strings as it reads them, its character encoding shouldn't matter;
  3. Pdftex. As mpg says, this should also pass the strings to the filesystem uninterpreted, but you can check to see if using Luatex (which does assume UTF8 everywhere; use --jobname=whatever), and see if anything changes;
  4. Which leaves, as the most likely suspect, the encoding used by the terminal you use to enter the pdflatex command: can you see if this is Latin-1?
  • Problem is, Windows has neither a filesystem encoding nor a terminal encoding in the Unix sense, it always uses UTF-16, but the standard C library functions aren't aware of that. – Philipp Dec 6 '10 at 13:20

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.