I agree, Priory Bay, on a sunny day is superb. But there is a bar which you can get caught behind so be careful with that. East Head inside Chichester Harbour is also lovely, but sometimes the tide gets very strong indeed (strong enough for anchored boats to make a 'wake')
Edit: Sorry, just noticed the on the IOW bit. East Head still worth a visit tho'!