Since you didn't say what your tubes are made of, the Weaver universal stuff should work. P/N 3001 (listed as for PVC, but works on both.) If they are hypalon tubes, any hypalon adhesive should work as well (Clifton, Shore, Weaver, Bostik, etc.)
Yes, you should clean the old adhesive off, if possible. Gentle power sanding may do it (if it's a large area); hand sanding would be my preference (for smaller areas.) Solvent (MEK for hypalon) may help to soften the old glue a bit. Make sure you only remove the old glue and lightly rough up the hypalon surface. Do not sand down through the hypalon to the fabric layer. Roughen up the rub rail in the same way. Degrease and clean with an MEK moistened rag (both types of fabric.)
Follow directions for the adhesive of choice to the letter (generally, two or three thin coats, with drying time between, allow last coat to tack up, and apply as much pressure as you can get on the join for a day or so.)
A Google search for Hypalon repair (or PVC boat repair if your tubes are PVC) should get you more detailed instructions, as well as adhesive sources.