Yup, regularly go round Portland bill to the dive sites on the East side. Normally skim round the bill within 30 yards of the shore.
The dangers are swimmers
actually spear fishermen after bass, pots and other boats including charter fishing boats drifting, oh and idiots shore fishing casts of 150 yards with 6 oz lead weights.
Keep an eye ont he echo sounder as you round the bill, it drops to 100 plus meters which would be a great dive
The bank to the East, Shambles I think should be avoided but its marked with a cardinal at each end. It comes very close to the surface (3m) and waves will break across it. Its just single so not much cop for diving unless you want a ride in the fast lane underwater.
It would have to be blowing a good 5 before I would consider a different approach.