Terrible follower AI. I think of this because I just did the Temple of Darkness in Tales of Symphonia, but I also got permanently stuck emulating Secret of Mana very early in the game because I went around a wall, my follower didn't, and the screen wouldn't let me go back. (And I state saved. Okay, that was my fault.)

Also, sneak levels and/or stolen-weapons levels. No. Let me use my freaking powerful weapons and run straight into the face of death!