Extract document statistics? - How many pages has chapter xy? Count fixmes?

Question

I have a large pdflatex document where each chapter is located in an extra text file and included with \include{chapter3.tex} ...

• How can I extract the page numbers of each chapter and write them into a text file?
  I want to know how many pages each chapter has (at a certain moment) and get a list with e. g.
  (Can this be done with the included text files or would it be better to define labels at the start of each chapter and then read out the page numbers of those labels)?

chapter 1: 5 pages
chapter 2: 10 pages
chapter 3: 4 pages

document: 19 pages

• Furthermore I'd like to count the occurences of certain commands like e. g. \N{note} or \NK{note} (which I have defined for creating notes in the document with the fixme package per chapter and write them also in the text file like:

chapter \N \NK
chapter 1: 20 3
chapter 2: 3 5

document: 23 8

Answer 1

I assume that you do not want to modify your source file (otherwise it is much easier - you add some smart command after each \chapter).

You could do something like this:

\let\origchapter=\chapter
\def\chapter{\label{chap:#1}\origchapter}

This has a disadvantage that the label is associated with the previous page. To do it better you'd have to take care for the syntax of \chapter (optional star, optional arguments), which is a bit more time-consuming to implement, but well possible (and not that difficult, either).

Then just parse the .aux file with some perl/python/lua/whatever script.

Alternatively, you could use something like

\write\somefilehandler{Chapter: \value{chapter}\thepage}

instead of \label; then you would have to open \somefilehandler at the beginning of your document and close it \AtEndDocument.

As for your second question, a simple idea is:

\newcounter{foocount}
\let\origfoo=\foo
\def\foo{\stepcounter{foocounter}\origfoo}
\AtEndDocument{\message{\string\foo: \value{foocounter}}}

This would give you totals for the whole document in your log file and on the terminal. If you want totals per chapter, you could do \messages (or \writes) in \chapter, redefining it in the above spirit.

Answer 2

as mentioned in my comment above, this is python script I used to split resulting PDF file by sections. You might have to adapt it for your needs, I hope it is will be useful for you. It calls pdftk to do the actual splitting.

Maybe there is a more "standard" solution, I hope someone can add a comment.

#!/usr/bin/python

class breakPt:
    def __init__(self,pg,title,num):
        self.pg,self.title,self.num=pg,title,num
import re,os
pts=[]
for l in file('master.toc'):
    #if (not '{part}' in l) and (not '{section}' in l): continue
    m=re.match(r'^\\contentsline\W*{(section|part|chapter)}{(.*)}{([0-9]+)}({[^}]*})?$',l)
    if not m: continue
    #print 'Match:',m.group(1,2,3)
    type,raw,pg=m.group(1,2,3)
    if type=='section':
        m=re.match(r'^\\numberline\W*{([0-9]+)}(.*)$',raw)
        num,raw=m.group(1,2)
        raw=re.sub(r'\\FN@sf@gobble@opt .*$','',raw) # strip footnote
        raw=re.sub(r'\\IeC\W*{.*?([a-zA-Z]) ?}',r'\1',raw) # remove accents
        raw=re.sub(r'\\emph\W*{(.*?)}',r'\1',raw) # remove \\emph
        raw=re.sub(r'(:|\W*\\&|\W*\().*$','',raw) # take just the "first part" as name
        raw=re.sub(r' a ',r' ',raw) # remove 'a' as conjunction
        raw=re.sub(r'[^a-zA-Z]+','_',raw) # remove commans
        raw=raw.lower()
    pts.append(breakPt(int(pg),raw,int(num) if type=='section' else -1))
    #print 'added',pts[-1].num,pts[-1].title,pts[-1].pg

for i,pt in enumerate(pts):
    bgPg,endPg=pt.pg,(pts[i+1].pg-1 if i+1<len(pts) else -1)
    if pt.num<0: continue
    #print pt.num,bgPg,endPg,pt.title
    pgSpec='%02d-%02d'%(bgPg,endPg) if endPg>0 else '%02d-end'%bgPg
    out='%02d-%s.pdf'%(pt.num,pt.title)
    print pgSpec,out
    os.system('pdftk master.pdf cat %s output %s'%(pgSpec,out))

Answer 3

I finally used a solution based on a bash script.

As it might be interesting for someone else, I'm sharing it here. However I have to say that this is experimental and might contain some very nasty hacks, as I'm not very experienced with bash scripting.

#!/bin/bash

# script overwrites file DocStat.txt with recent statistics of latex writing project:
# Block 1: date, number of pages and file size of PDF
# Block 2: All chapters with title and number of pages
# Block 3: All chapters and sectinos with title and number of pages 
# Block 4: word count statistics using textcount.pl for all chapters
# (during script runtime, a temporary file Docstat.tmp is created for collecting the output)
# the script scans the aux files to extract the page numbers, where sections begin

date > Docstat.txt
grep "Output written on" Diss.log >> Docstat.txt

grep "contentsline {chapter}"  Diss.toc | sed 's/\\contentsline //g' | sed 's/\\numberline //g' >> Docstat.tmp

NEin=$(grep "newlabel{anf:Kap}\|newlabel{end:Kap}"  1_Introduction.aux | awk 'BEGIN {
  FS="[{}]+"
 } {
  if ($2=="anf:Kap")
   KapAnf=$4
  if ($2=="end:Kap")
   KapEnd=$4
 } END {
#   print KapAnf
   print KapEnd-KapAnf+1
 }')

NGru=$(grep "newlabel{anf:Kap}\|newlabel{end:Kap}"  2_Theory.aux | awk 'BEGIN {
  FS="[{}]+"
 } {
  if ($2=="anf:Kap")
   KapAnf=$4
  if ($2=="end:Kap")
   KapEnd=$4
 } END {
#   print KapAnf
   print KapEnd-KapAnf+1
 }')

 NExp=$(grep "newlabel{anf:Kap}\|newlabel{end:Kap}"  3_Experimental.aux | awk 'BEGIN {
  FS="[{}]+"
 } {
  if ($2=="anf:Kap")
   KapAnf=$4
  if ($2=="end:Kap")
   KapEnd=$4
 } END {
#   print KapAnf
   print KapEnd-KapAnf+1
 }')

  NEuD=$(grep "newlabel{anf:Kap}\|newlabel{end:Kap}"  5_ResultsAndDiscussion.aux | awk 'BEGIN {
  FS="[{}]+"
 } {
  if ($2=="anf:Kap")
   KapAnf=$4
  if ($2=="end:Kap")
   KapEnd=$4
 } END {
#   print KapAnf
   print KapEnd-KapAnf+1
 }')

   NZus=$(grep "newlabel{anf:Kap}\|newlabel{end:Kap}"  7_Conclusion.aux | awk 'BEGIN {
  FS="[{}]+"
 } {
  if ($2=="anf:Kap")
   KapAnf=$4
  if ($2=="end:Kap")
   KapEnd=$4
 } END {
#   print KapAnf
   print KapEnd-KapAnf+1
 }')


NLit=$(awk 'BEGIN {
  FS="[{}]+"
 } {
  if ($3=="References") #manuelle Anpassung, weil Lit. keine Nummer hat
   KapA=$4
  if ($4=="Publications")
#   KapB=$5-5 # manual correction by 5 pages!!
#   KapB= $KapB-4 # manuelle Korrektur um 4 Seite!!
 } END {
   print KapB-KapA+1
 }' Docstat.tmp) #&& echo $DIFFERENZ

NGes=$(awk 'BEGIN {
  FS="[{}]+"
 } {
  if ($4=="Introduction")
   KapA=$5
  if ($3=="References") #manuelle Anpassung, weil Lit. keine Nummer hat
   KapB=$4
 } END {
   print KapB-KapA+1
 }' Docstat.tmp) #&& echo $DIFFERENZ


NGesLit=$(($NGes + $NLit)) 

echo " " >> Docstat.txt
echo "==== page numbers" >>Docstat.txt
echo "total_(withoutRefs): $NGes S." >> Docstat.txt
echo "total:             $NGesLit S." >> Docstat.txt
echo " " >>Docstat.txt
echo "Introduction: $NEin S." >> Docstat.txt
echo "Theory:       $NGru S." >> Docstat.txt
echo "Experimental: $NExp S." >> Docstat.txt
echo "Results:      $NEuD S." >> Docstat.txt
echo "Conclusions:  $NZus S." >> Docstat.txt
echo "References:   $NLit S." >> Docstat.txt

head -14 Docstat.txt

echo '==== number of lines'
grep Zeilenzahl Diss.log # must be 

grep linenumber *.aux | sed 's/\\setcounter{linenumber}/ /g'

growlnotify -t "Diss Statistik: $NGes / $NGesLit S." -m "Ein $NEin, Gru $NGru, Erg $NEuD"

echo " " >> Docstat.txt
echo '==== Chapters' >> Docstat.txt
cat Docstat.tmp >> Docstat.txt

echo "" >> Docstat.txt


echo '==== Details' >> Docstat.txt
grep "contentsline \({chapter}\|{section}\)"  Diss.toc | sed 's/\\contentsline //g' | sed 's/\\numberline //g' >> Docstat.txt

echo '==== Word count' >> Docstat.txt
texcount  1_Introduction.tex >> Docstat.txt
texcount  2_Theory.tex >> Docstat.txt 
texcount  3_Experimental.tex >> Docstat.txt
texcount  5_Results.tex >> Docstat.txt
texcount  7_Conclusion.tex >> Docstat.txt

echo '==== Number of lines' >> Docstat.txt
grep Zeilenzahl Diss.log >> Docstat.txt
echo 'by chapter' >> Docstat.txt
grep linenumber *.aux | sed 's/\\setcounter{linenumber}/ /g' >> Docstat.txt

rm Docstat.tmp