Personally I'd just replace the trunk. Easier and a backup.
If filling hole and wanting a floorless finish then polyester is the simplest repair. Downside is it doesn't stick as good as my preferred epoxy and it also doesn't stick to wood at all. So if you wish to use wood in the repair polyester is out.
How thick is transom there? If be tempted to grind a bezel or taper both sides and fill the hole with successive layers of polyester resin and chopped strand. Once hole is filled then layer outward to fill the taper. typically it's quoted as 12:1 bezel so for each inch of depth you go out 12. Might cover ur hole transom if you use those ratios.
If you could work inside out, you could add a backing plate to outside of hole. Then gelcoat, then grp till hole is full, then a layer on inside to whole area and around it to tie it in.
Some gelcoat mixed with cabosil would probably do it thought if it was tapered inside and out, plus a lure of chopped strand inside.
If boat is painted just fill it with epoxy filler or epoxy mixed with cabosil and call it done. A layer of paint and you will be very pleased