The easiest road to higher power output would be to use one of the ham band VHFs which are 50-55w and can be opened up to the marine band though that won't be waterproof so no use on an open boat ... I have a collection of ham sets which we use on marine band because the Mrs requires it for her job though of course I always make sure we are transmitting on the middle power rating not on high power
The ones I have are Icom IC-2000, a few IC2100 and an IC2200 plus an Alinco DR140 and an old Icom IC229H at work, they all operate happily in the marine band for transmit and receive, and all except the last one are 50w+
3DB gain on the aerial is worth twice the power output so may be an easier solution? Conversely some crap coax or a dodgy aerial or connections will cut the performance in half just like that.