# Extract document statistics? - How many pages has chapter xy? Count fixmes?

I have a large pdflatex document where each chapter is located in an extra text file and included with \include{chapter3.tex} ...

• How can I extract the page numbers of each chapter and write them into a text file?
I want to know how many pages each chapter has (at a certain moment) and get a list with e. g.
(Can this be done with the included text files or would it be better to define labels at the start of each chapter and then read out the page numbers of those labels)?

chapter 1: 5 pages
chapter 2: 10 pages
chapter 3: 4 pages

document: 19 pages

• Furthermore I'd like to count the occurences of certain commands like e. g. \N{note} or \NK{note} (which I have defined for creating notes in the document with the fixme package per chapter and write them also in the text file like:

chapter \N \NK
chapter 1: 20 3
chapter 2: 3 5

document: 23 8

-
You can extract chapter pages from the generated .toc file easily, with a few regexps. I did it once with python, splitting the output PDF file in per-chapter PDF's including meaningful titles. I will post the script for inspiration when I get to my other computer later. –  eudoxos Jul 29 '11 at 14:11
One caveat: this will be more work if you want the chapter titles too and if you use inputenc. Having said that, I'm waiting for your solution, too (it would be also useful for me;)). –  mbork Jul 29 '11 at 16:41

I assume that you do not want to modify your source file (otherwise it is much easier - you add some smart command after each \chapter).

You could do something like this:

\let\origchapter=\chapter
\def\chapter{\label{chap:#1}\origchapter}


This has a disadvantage that the label is associated with the previous page. To do it better you'd have to take care for the syntax of \chapter (optional star, optional arguments), which is a bit more time-consuming to implement, but well possible (and not that difficult, either).

Then just parse the .aux file with some perl/python/lua/whatever script.

Alternatively, you could use something like

\write\somefilehandler{Chapter: \value{chapter}\thepage}


instead of \label; then you would have to open \somefilehandler at the beginning of your document and close it \AtEndDocument.

As for your second question, a simple idea is:

\newcounter{foocount}
\let\origfoo=\foo
\def\foo{\stepcounter{foocounter}\origfoo}
\AtEndDocument{\message{\string\foo: \value{foocounter}}}


This would give you totals for the whole document in your log file and on the terminal. If you want totals per chapter, you could do \messages (or \writes) in \chapter, redefining it in the above spirit.

-
thanks for your suggestion! As it is about one very special document of mine, there would be no problem to modify the source file (by adding some lines of code somewhere, as long as they do not change the typesetting of the document itself) –  Martin Jul 29 '11 at 11:10
thanks again - I tried to make it work and I wonder how I could write the title text of the chapter to the external file. With \value{chapter} I only get a string like c@chapter. –  Martin Dec 11 '11 at 22:53

as mentioned in my comment above, this is python script I used to split resulting PDF file by sections. You might have to adapt it for your needs, I hope it is will be useful for you. It calls pdftk to do the actual splitting.

Maybe there is a more "standard" solution, I hope someone can add a comment.

#!/usr/bin/python

class breakPt:
def __init__(self,pg,title,num):
self.pg,self.title,self.num=pg,title,num
import re,os
pts=[]
for l in file('master.toc'):
#if (not '{part}' in l) and (not '{section}' in l): continue
m=re.match(r'^\\contentsline\W*{(section|part|chapter)}{(.*)}{([0-9]+)}({[^}]*})?$',l) if not m: continue #print 'Match:',m.group(1,2,3) type,raw,pg=m.group(1,2,3) if type=='section': m=re.match(r'^\\numberline\W*{([0-9]+)}(.*)$',raw)
num,raw=m.group(1,2)
raw=re.sub(r'\\FN@sf@gobble@opt .*\$','',raw) # strip footnote
raw=re.sub(r'\\IeC\W*{.*?([a-zA-Z]) ?}',r'\1',raw) # remove accents
raw=re.sub(r'\\emph\W*{(.*?)}',r'\1',raw) # remove \\emph
raw=re.sub(r'(:|\W*\\&|\W*$$).*','',raw) # take just the "first part" as name raw=re.sub(r' a ',r' ',raw) # remove 'a' as conjunction raw=re.sub(r'[^a-zA-Z]+','_',raw) # remove commans raw=raw.lower() pts.append(breakPt(int(pg),raw,int(num) if type=='section' else -1)) #print 'added',pts[-1].num,pts[-1].title,pts[-1].pg for i,pt in enumerate(pts): bgPg,endPg=pt.pg,(pts[i+1].pg-1 if i+1<len(pts) else -1) if pt.num<0: continue #print pt.num,bgPg,endPg,pt.title pgSpec='%02d-%02d'%(bgPg,endPg) if endPg>0 else '%02d-end'%bgPg out='%02d-%s.pdf'%(pt.num,pt.title) print pgSpec,out os.system('pdftk master.pdf cat %s output %s'%(pgSpec,out))  - thank you for your script - sorry for the late reply! I tried to make it work, but I'm getting an error line 16, in <module> num,raw=m.group(1,2) AttributeError: 'NoneType' object has no attribute 'group' Unfortunately, I'm not (yet) familiar with python at all and I have no idea what it means. I tried to execute the script in a directory with the files master.toc and master.pdf – Martin Dec 11 '11 at 0:08 It means the \\numberline regular expression is not matched; it only comes on the section line; perhaps your format of the .toc file is different from the mine. What does that line look like? (The re.match returns None if the expression does not match the text) – eudoxos Dec 12 '11 at 14:49 I finally used a solution based on a bash script. As it might be interesting for someone else, I'm sharing it here. However I have to say that this is experimental and might contain some very nasty hacks, as I'm not very experienced with bash scripting. #!/bin/bash # script overwrites file DocStat.txt with recent statistics of latex writing project: # Block 1: date, number of pages and file size of PDF # Block 2: All chapters with title and number of pages # Block 3: All chapters and sectinos with title and number of pages # Block 4: word count statistics using textcount.pl for all chapters # (during script runtime, a temporary file Docstat.tmp is created for collecting the output) # the script scans the aux files to extract the page numbers, where sections begin date > Docstat.txt grep "Output written on" Diss.log >> Docstat.txt grep "contentsline {chapter}" Diss.toc | sed 's/\\contentsline //g' | sed 's/\\numberline //g' >> Docstat.tmp NEin=(grep "newlabel{anf:Kap}\|newlabel{end:Kap}" 1_Introduction.aux | awk 'BEGIN { FS="[{}]+" } { if (2=="anf:Kap") KapAnf=4 if (2=="end:Kap") KapEnd=4 } END { # print KapAnf print KapEnd-KapAnf+1 }') NGru=(grep "newlabel{anf:Kap}\|newlabel{end:Kap}" 2_Theory.aux | awk 'BEGIN { FS="[{}]+" } { if (2=="anf:Kap") KapAnf=4 if (2=="end:Kap") KapEnd=4 } END { # print KapAnf print KapEnd-KapAnf+1 }') NExp=(grep "newlabel{anf:Kap}\|newlabel{end:Kap}" 3_Experimental.aux | awk 'BEGIN { FS="[{}]+" } { if (2=="anf:Kap") KapAnf=4 if (2=="end:Kap") KapEnd=4 } END { # print KapAnf print KapEnd-KapAnf+1 }') NEuD=(grep "newlabel{anf:Kap}\|newlabel{end:Kap}" 5_ResultsAndDiscussion.aux | awk 'BEGIN { FS="[{}]+" } { if (2=="anf:Kap") KapAnf=4 if (2=="end:Kap") KapEnd=4 } END { # print KapAnf print KapEnd-KapAnf+1 }') NZus=(grep "newlabel{anf:Kap}\|newlabel{end:Kap}" 7_Conclusion.aux | awk 'BEGIN { FS="[{}]+" } { if (2=="anf:Kap") KapAnf=4 if (2=="end:Kap") KapEnd=4 } END { # print KapAnf print KapEnd-KapAnf+1 }') NLit=(awk 'BEGIN { FS="[{}]+" } { if (3=="References") #manuelle Anpassung, weil Lit. keine Nummer hat KapA=4 if (4=="Publications") # KapB=5-5 # manual correction by 5 pages!! # KapB= KapB-4 # manuelle Korrektur um 4 Seite!! } END { print KapB-KapA+1 }' Docstat.tmp) #&& echo DIFFERENZ NGes=(awk 'BEGIN { FS="[{}]+" } { if (4=="Introduction") KapA=5 if (3=="References") #manuelle Anpassung, weil Lit. keine Nummer hat KapB=4 } END { print KapB-KapA+1 }' Docstat.tmp) #&& echo DIFFERENZ NGesLit=((NGes + NLit)) echo " " >> Docstat.txt echo "==== page numbers" >>Docstat.txt echo "total_(withoutRefs): NGes S." >> Docstat.txt echo "total: NGesLit S." >> Docstat.txt echo " " >>Docstat.txt echo "Introduction: NEin S." >> Docstat.txt echo "Theory: NGru S." >> Docstat.txt echo "Experimental: NExp S." >> Docstat.txt echo "Results: NEuD S." >> Docstat.txt echo "Conclusions: NZus S." >> Docstat.txt echo "References: NLit S." >> Docstat.txt head -14 Docstat.txt echo '==== number of lines' grep Zeilenzahl Diss.log # must be grep linenumber *.aux | sed 's/\\setcounter{linenumber}/ /g' growlnotify -t "Diss Statistik: NGes / NGesLit S." -m "Ein NEin, Gru NGru, Erg NEuD" echo " " >> Docstat.txt echo '==== Chapters' >> Docstat.txt cat Docstat.tmp >> Docstat.txt echo "" >> Docstat.txt echo '==== Details' >> Docstat.txt grep "contentsline \({chapter}\|{section}$$"  Diss.toc | sed 's/\\contentsline //g' | sed 's/\\numberline //g' >> Docstat.txt

echo '==== Word count' >> Docstat.txt
texcount  1_Introduction.tex >> Docstat.txt
texcount  2_Theory.tex >> Docstat.txt
texcount  3_Experimental.tex >> Docstat.txt
texcount  5_Results.tex >> Docstat.txt
texcount  7_Conclusion.tex >> Docstat.txt

echo '==== Number of lines' >> Docstat.txt
grep Zeilenzahl Diss.log >> Docstat.txt
echo 'by chapter' >> Docstat.txt
grep linenumber *.aux | sed 's/\\setcounter{linenumber}/ /g' >> Docstat.txt

rm Docstat.tmp

-