Take the 2-minute tour ×
TeX - LaTeX Stack Exchange is a question and answer site for users of TeX, LaTeX, ConTeXt, and related typesetting systems. It's 100% free, no registration required.

How do you count the number of word occurrences in your tex files? The reason I do this is to easier recognize words I use too much in a text. At the moment I use the following one-liner in the bash.

cat *.tex | sed 's/[[:space:]|[:punct:]]\+/\n/g' | sort | uniq -c | sort -n

What it does is, output all .tex-Files with cat, substitute the whitespaces and punctuations with a line-break using sed, sort the output, count the unique words and sort it again after the number output by uniq -c.

One of the problems I have with that approach is, that words that belong together but are divided by a whitespace are counted separately. So for example "New York" you get k occurrences of New and n occurrences of York, mixing with other occurrences of New and York.

EDIT: Another problem is of course, how do you recognize word inflection such as declension and conjugation? But that's probably something way out of scope of a one-liner, or does anyone have an idea how to cope with that?

EDIT2: As Hendrik and Joseph pointed out, that's not really TeX-related, but perhaps somebody finds it useful :)

share|improve this question
3  
I think your question isn't really TeX related, so I'm voting to close it as "off-topic". My point is somehow that a satisfactory answer wouldn't involve any TeX knowledge but rather access to a huge database. –  Hendrik Vogt Mar 26 '11 at 23:02
    
I'd agree that the question as posed does not really seem to be TeX-related. I'd vote to close if I were not a moderator. –  Joseph Wright Mar 27 '11 at 8:44
1  
Alternatively, look at tex.stackexchange.com/questions/534 for a TeX-specific set of answers. –  Joseph Wright Mar 27 '11 at 8:44
    
Thanks for the comments, you're probably right that it's not only related to TeX but text-processing as a whole. I've thought that maybe other people had the same issue when writing their thesis in TeX, so I just asked. –  Bruno Mar 27 '11 at 12:40

1 Answer 1

Cannot say much about the run latex, and then use dvi2tty on the output .dvi file. This would take better care of macro expansion. I suggest therefore:

dvi2tty 00.dvi | sed  's/[[:space:]|[:punct:]|[:digit:]]\+/\n/g' | sed '/^$/d' | tr "A-Z" "a-z" | sort | uniq -c | sort -nr | sed "/ 1 /d" 

which is similar to your pipe, except that

  • It treats digits as separators.
  • It ignores spaces
  • It eliminates empty lines
  • It eliminates words that occur only once.
share|improve this answer

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.