I'm also a D3 owner, but have been lucky enough to do a bit of travelling in an X5.
It certainly helps that you can fix the D3 yourself as I certainly couldn't afford to keep mine if I was paying Mr LR £120/hr every time it's gone wrong.
It does of course come down to what you're going to do with the car.
I find the D3 great to cruise the motorway with, in fact more comfortable than the X5, but at 70mph not 80(on a private road etc......). And of course it's using more fuel than the X5 at those speeds.
On A roads the X5 is the better handling car if the road is full of corners, but again slowing down slightly in the D3 brings it closer to the X5, but it will never be as good in that respect.
However, the X5 will obviously never be as good as the D3 in any sort of situation that isn't normal on road driving, and will reach it's limit in Snow, Mud or on wet grass whilst the D3 goes sailing past. BUT, it always depends on the tyres fitted. I run my D3 on GG ATs which I believe are the best all round tyres I've used for the mixture of situations I find myself in. They are NOT noisy on the road, are great on wet slimy slipways and brilliant on wet grass and deep mud. They are also very good on wet roads.
I've not towed with the X5, but there is a reason the D3/D4 has been voted best tow car in it's category for years on end.
People at work admire my D3 and ask what I think about them getting one.
I always answer that compared to the Japanese 4x4s I've had in the past it's the best car I've ever owned, but also the one I've spent more time fixing. I advise them to proceed only if they can afford to pay Mr LR to fix it, or be prepared to fix it themselves.
I understand that a lot of the D3 issues have been fixed on the D4.
Despite everything, if I came into some money I'd be straight down the LR dealers for a D4. I'd love a new style Range Rover, but I need the 7 seats and load space which the RR just doesn't have, although it does have the glorious
Oh, and with the all new D5 coming out soon, I'd wait until all the D5 owners have done the development testing for LR and buy a D6.