SMT: Nocturne.

Etrian Odyssey: Any of them played above easy mode.

Dark Souls, especially Online Mode.

SMT: Devil Survivor broke a few friends of mine... though that may have been the strategy game element.

BoF: Dragon Quarter. Like everything else listed here I enjoyed Dragon Quarter, but I admit certain mechanics are a kick in the teeth for newbies.