Inchkeith is well worth a visit but it is relatively overgrown and has crumbling ruins so take care. Will attach some photos but the old Victorian fort / buildings and underground tunnels are well worth an explore (take a torch and use your head - if it looks unsafe stay out). The WW1 and WW2 gun emplacements have artillery stores beneath and many again are reached through tunnels.
Isle of May is beautiful, it has a small natural harbour and visitor centre and is great for wildlife - seals, porpoise (the odd Orca) and birdlife (puffins, tern, kittiwake, razorbills etc). Good snorkelling and diving off the southern / SW side.
Bass Rock is pretty amazing to see with the '000s of Gannets but is does smell somewhat
Good fishing for Pollock and Cod off the south side towards Tantalon castle. Good relatively shallow snorkel and dive on the western side and a tunnel that you can walk through at spring lows.
Craigleith is festooned with Puffins although these leave in the next few weeks, and some seals - good fishing for mackerel.
Lamb has hexagonal basalt columns similar to the giants causeway and again worth a look around if snorkelling or diving - lots of life in the cracks and crevasses.
I always think Fidra is overshadowed by its neighbours but it was reputed to be the inspiration for Treasure Island which Stevenson wrote at the nearby Marine Villa.