As PeOfEo said, but I would like to add some. Certain things that defrag the drives more then others: games (the computer my brother uses for games often gets the most fragged), loading and deleting programs (your adding/deleting huge chunks that need ordering basically), um... I believe video work is supposed to as well, but i cant back this up with experience.
And to further on what Peo said about doing it often and it taking less time. I do it every week because it only takes a couple seconds then. If you rarely do it then it can take FOREVER
Last thing. Dont fill your drive up too much. This will also make it harder to defrag and it slows the computer down a lot. From what i have been told about 15-20% of your drive should be free.
Hope it helped some