Good fishing spots are easy - just mark good spots with a waypoint, and set it up to record trails. Most have a fish logo in the "add waypoint" screen as one of the options for that very reason!
If cheap is your main consideration, you should be able to pick up a used waterproof garmin handheld for around £50. The old GPS12 of early 1990s vintage is very basic, but does what you need and usually goes for around £40 on Ebay. B&W screen mapping ones (EG GPS 76map or GPS-V) will be slighly more expensive. Just be aware that some of the garmin <number> series had aviation related sofftware, and have basic aviation maps built in, which are obviously not much use on the water... the 12/72/76/II/III/V are all fine for boating use.
I'm not sure what you mean by "track you" - if you mean record where you have been so you have a "breadcrumb trail" to get back they will all do that. If you mean so that your fishing buddies can find you out on the water you'll need a DSC radio hooked up to the GPS, or if you fish in an area with a strong mobile phone signal, there are plenty of free apps to do that.
Be aware for the radio scenario that not all handhelds have an NMEA out. (the data signal the VHF needs to broadcast your position). Post back if that's what you mean & we can explain further.