TeXcount cannot fine \input files with a command in the path?

Question

UPDATE: The problem I am having is due to having Python code inside a PythonTeX pycode environment. Seems this causes a parsing error with the TeXcount Perl script.

---

I want to use TeXcount to count the words in my thesis. However, I have either run into a limitation of TeXcount or it's difficult to find an explanation (via documentation, here, forums, etc.) detailing how to alleviate my issue.

I have all the chapters in my thesis in a subfolder from the main .tex document. I call these via the following line (or similar, depending on the chapter name, obviously):

\include{\chaptersPath/background_information}

Within many of those chapter files, I have additional files stored a directory deeper for major topics/themes of that chapter, such as:

\input{\chaptersPath/background_information/literature}

The document compiles fine. And honestly, I'm not here currently to debate whether my file and directory structure is proper, the most efficient, etc.

I have tried defining \chaptersPath variously as:

\newcommand{\buildFilesPath}{_build_files}%

or

\edef\buildFilesPath{./_build_files}%

Same behavior for both - The problem I have is that having this command or definition in the \input doesn't seem to expand for texcount. TeXcount complains that the file is not found:

[1;31m!!! File not found or not readable: \chaptersPath/background_information !!![0m

I tried using -auxdir and -dir, but these seem to not make TeXcount happy because those commands are still in the \input calls.

I also tried including \immediate\write18{texcount \jobname.tex -inc -out=\buildFilesPath/\jobname.sum} in the preamble thinking perhaps it would recognize those commands (replaced with plain text path details). I was wrong; it doesn't. Funny thing here is that \buildFilesPath and \jobname do get recognized.

Lastly, I've tried playing around with including \expandafter and this is totally not useful as well. It seems the texcount.exe (even within the \immediate\write18 method simply cannot expand those file path definitions).

Question: Is it possible to resolve this problem somehow, while maintaining those commands defining the file/chapter paths?

I realize I can just go through and hardcode all the paths manually. But this is rather crappy for flexibility, if I want to create a new document using much of the same (La)TeX preamble and base .tex document but with a different file/folder structure. The current way requires just one change rather than finding and replacing a bunch of stuff every time.

Answer 1

As daleif has pointed out in a comment, TeXcount does not actually expand LaTeX commands, and so it does not expand the path in \chaptersPath when trying to locate the included files.

There is a trick that might work. TeXcount can take instructions embedded in the LaTeX document as comments on the form %TC:.... One such instruction is %TC:subst which does a simple substitution of a macro with an arbitrary text string. Eg, you could try including somewhere at the top of the document (eg right after the definition of \chaptersPath) the command %TC:subst \chaptersPath /my/chapters/path and see if that does the job. This basically does a simple search and replace. You may have to fiddle around a bit to get the path right (absolute path or relative path?). I'm not entirely sure how it will handle tex files included inside included tex files, but I guess you'll find out if that works out.

---

Update on bug in TeXcount that causes problems

The TeXcount solution, as well as other uses file inclusion using \include, may fail due to a bug in TeXcount.

When using \include to include a TeX file, the .tex file type is not required. TeXcount was supposed to add .tex to the file name if not already there, but this seems to have failed and was the root cause of the difficulty of getting the solution to work.

This bug can be bypassed by ensuring all file inclusion using \include contain the .tex file type.

An alternative is to fix the code of TeXcount which is quite simple. Just find the function definition sub _parse_include_file and change the line if (!$file=~/\.tex$/i) {$file.='.tex';} to if ($file!~/\.tex$/i) {$file.='.tex';} and TeXcount should append .tex to the file name when required.

Answer 2

As @daleif said in the comments, TeXcount can't parse TeX, being a Perl script.

I don't do Perl so faced with a similar issue I wrote somethign in Python to pre-process the master tex file.

Here's the python script adapted for a minmal version of your case. I call it tc-paths.py:

import fileinput # see https://docs.python.org/2/library/fileinput.html
import os
chaptersPath=None
chaptersPathMacro=r'\chaptersPath'
chaptersPathDef=r'\def'+chaptersPathMacro
outfilename=None

for line in fileinput.input():
    if outfilename==None:
        outfilename=os.path.splitext(fileinput.filename())[0]+'-temp'+os.path.splitext(fileinput.filename())[1]
        print ('Writing to: '+outfilename)
        outfile=open(outfilename,'w')
    if chaptersPathDef in line:
        startpos=line.find('{',line.find(chaptersPathDef)+len(chaptersPathDef))+1
        endpos=line.find('}',startpos)
        chaptersPath=line[startpos:endpos]
        print (r'\chaptersPath='+chaptersPath)
        outline=line
    elif r'\input{'+chaptersPathMacro in line:
        print ('Found '+chaptersPathMacro+' in line %d'%fileinput.lineno())  
        outline=line.replace(chaptersPathMacro,chaptersPath)
    else:
        outline=line
    outfile.write(outline)

run it with python tc-paths.py my-master-file.tex, and it will output to my-master-file-temp.tex

A sample input tex file:

\documentclass{article}
\def\chaptersPath{../Chris}
\begin{document}
\input{\chaptersPath/chap1.tex}
\end{document}

And the output:

\documentclass{article}
\def\chaptersPath{../Chris}
\begin{document}
\input{../Chris/chap1.tex}
\end{document}

Then you run texcount on the -temp.tex file.