I hate Sorcerers/Mind Flayers. If they ambush your party, they'll most likely use their paralyze attack on everyone, and then they keep poking the party members for 1-2 damage that randomly causes instant death. Mages in the Ice Cave are another nightmare. When they ambush you, they'll kill everyone with Fire3 and Bane/Poison before you get a single turn (it has happened to me!).

Ghosts in the Sea Shrine are also quite annoying. They appear in groups of 3-5, they often ambush you and their attacks do a load of damage. 9 undead monsters with paralyzing capabilities and ambushing the party also annoys me. Clearly, I don't like being ambushed. WarMech is also a bit annoying in the PSX version because he's turned into WarChicken (he doesn't dare to face your party when you want to run into him).