A complete list of rare items would be daunting to post but I'll start with the ones you requested:

Stone Gun: Poach a Dark Behometh (you need to breed one).

Mace of Zeus: Move-Find on the Delta level of Deep Dungeon.

Defender: Poach a Taiju (you can encounter them in Deep Dungeon, but it's easier to breed one).

It doesn't work for everyone, but it's worth a try, especially with Excalibur. Go to a store that sells swords, equip a Knight with Two Swords, and put the sword and a shield in the character's hands, but the wrong ones (i.e. sword in the left and shield in the right). Go to the shop and choose 'best fit'. It should offer to equip your character with that weapon, but brand new and in the opposite hand. Once you've bought it, you can de-equip it and repeat as necessary.

An alternative: I've heard (but never tried) that you can duplicate a weapon by having a ninja throw it to a character who can catch while a Mime repeats the action to a final character, also with catch.

The ninjas in Deep Dungeon will sometimes throw some nice swords at you, but it's sporadic.

P.S. I created an Excel spreadsheet of every item, armor, weapon in the game and how to get them. Should I post that?