If you are repairing the transom always use a fiberglass roving. The weight of the roving should be between 12-18 oz, depending on the horsepower that you are installing you would need a minimum of 3 layers of roving on either side of the plywood. If the transom cracked due to a heavy engine you may want to add a few more layers of roving and also make fiberglass knees reinforcement from transom to the hull bottom. The knees should be placed on the outboard side the engine mounting bolts.
Please make sure that you grind the existing transom to bare fiberglass (all gelcoat must be removed) in order for the new resin/ frp to adhere properly to the old transom.