All the SIBs i have owned have had varnished transoms, so I can't help with specific paint details. Most boat paints tend to be gloss or semi gloss (satin) rather than matt.
Is it just damage to the paint you want to touch up, or has it affected the underlying wood?
If it is purely cosmetic, then a touch up with something like Blake's Multicoat should be fine.
High opacity satin topcoat and primer for all areas, including decks, bilges & lockers. Ideal for all areas above the waterline where a semi-gloss durable topcoat is required. Also can be used as a complete coating system onto new or bare wood. Coverage 11m2 Thinners/Brushwash No I Brush/Spray
If it has gone deeper, than fill with an epoxy filler first.