Head back to the Crescent Isle Hideout (by landing your airship directly onto the pad, of course), and go into the library. There, on the table, you should be able to find a note from Cid and Mid talking about how they are going to return the adamantite to where you found it, so follow them there!
You should be able to pick up from here.