The Divine King's Tower is part of a quest chain you have actually already started, but you lack a required character to move it on. It is fully optional however, though, I'd argue, is pretty worth doing, as long as you can deal with the Red Dragon there, since it is harder than the actual boss of the tower.

Forneus is the hardest of the four fiends, but his dungeon is super easy to access and has some decent loot with no sub-bosses guarding them. The Meteor Fragment is only a few screens in, hidden in a side room along the right side of one of the walls, and alone is worth running in to get, as it makes a good items in the workshop.

The mountain fiend cannot be fought until one of the other three are taken down, so you aren't doing anything wrong with that flag or missing a conversation anywhere or anything.

The locked door a the bottom of the castle isn't end game and the earth fiend is probably the best one to start with. His dungeon has several mini-bosses, but all can be skipped, and the ring itself is a pretty decent accessory I'd recommend picking up as soon as you can even if you want to put off fighting Arakes himself.

Dobis and Master Dobis, those little goblin archers, are stupid nasty, but you can find a shield called a Wonder Bangle that offers something like 99% evade against bow attacks; I believe one is in Arakes Dungeon, which is the one I usually pick up.

As I recall, only 4 characters in the game are especially good with greatswords, and since two basically don't count due to limited recruitment, Ward and Paul are your options for that.