I know everything there is to know about Yarmouth, nowt about Ventnor (its over the "other side"
Anyway. Mill road garage is the one. Its not the cheapest but cheaper than the quay. The quay is about 98p a litre (last summer) and the garage is maybe 85p if you're lucky. So on a 25 litre can you'll save just over 3 quid. However, you might get stung for harbour dues if you tie up and walk to get fuel, you won't have to pay harbour dues if you fuel up at the berth and then leave.
The garage is, as said before a 5 (more like 3 with lanky legs like me) minute walk. This is longer than it sounds carrying a full tank though! if you moor on the quay, walk over the nearby round about and towards the carpark. The road bends round to the left and then take the first right by the school. Walk up that road and the garage is at the end.
Opening times. from memory I think they are open sat am but closed sat pm and sun. I think they are open during the week. Best give em a ring.
Another thing, there is an outboard repair bloke at the garage which could be useful for someone to know.
Hope that helps Nick!