If your prop is badly damaged, you may have damaged the gearbox due to the excess vibration. Similarly, if the prop is damaged it must have hit something. There could be internal damage to the gearbox as a result of this, and/or you may have a bent prop shaft which could also be causing further damage/vibration etc.
Posting a pic of the prop would help to indicate how bad the problem is likely to be. Also check and see if the prop shaft is bent by rotating it with the engine in neutral (and switched off of course!)
I had a similar problem once. Hit something in the water. One blade on the prop was damaged. Limped home with it vibrating quite a lot, cost £2000 to fix :-(