I'm using (a slightly modified version of) RefBase to organise my references these days and that allows me to add keywords and to group references. I've just started using instiki as a place to take notes on these references and that allows me to add "categories" to the pages, which is somewhat akin to keywords, I guess.
Since most of the stuff I index already has MSC and/or arXiv classification (I'm a mathematician), adding my own keywords based on those seems a little pointless. The scheme I'm currently using is to add keywords based on the paper or idea I was working on when I looked for that reference. Sometimes if a couple of papers are very closely related then I might use the same keyword for both (usually the name of the first paper).
So corresponding to a couple of my papers, I have articles tagged "hopf" and "comparative smootheology". If I don't have an actual paper, I'll still try to tag them by the answer to the question "What was I thinking about when I went looking for this?". So I have papers tagged "loop spaces", even though that covers a wide spectrum of my work. Lastly, I read the "new on the arXiv" and have a way of automatically importing references from that into my database; these, therefore, aren't articles that I've gone looking for so I tag them by the first connection that came into my head when I read the abstract. So, in summary, when trying to tag an article I'll ask (in order):
- What article was I working on?
- What topic was I thinking about?
- What (of the above two) does it most remind me of?