The cost of petrol, or diesel for that matter, certainly focuses the mind but it's the cost of keeping a boat that doesn't get a lot of use that really hurts.
Small and cheap is good. No storage costs, minimal maintenance and insurance costs, negligable depreciation. If it only gets used a few times a year because of other commitments then no problem.
There's plently of fun to be had mucking around in little boats - if I had something bigger I'd want to be doing more ambitious trips, which means more time and money. It's good to try different things, if only to fill in the gap before the next RIB