The hailer speaker connection is for using the vhf as a hailer - ie shouting at people through your mic. Theres another wire for connecting an external speaker, says its yellow in the manual but on my icom its black (inner wire +, outer sheath -).
I could be very wrong but don't think you could connect this to an rca input, even if you could you wouldnt be able listen to the vhf and other input sources at the same time as you can only select 1 input source at a time.
You could connect a stereo speaker direct to the vhf but then you'd have 50 cent shouting over the top of somebodys mayday message
Also when an external speaker is connected to the vhf the internal speaker is automatically switched off.
Another thing to consider is that stereo speakers are designed to reproduce a wide range of sound frequencies whereas the proper job is designed specially for reproducing voices, makes a big difference in clarity.
Everything on my console is flush mounted leaving the top clear but I ran out of room for a vhf speaker. I really really didnt want to, but had no choice I had to bolt a big ugly speaker up there. Needs must.