I think it's something about unprecedented quality in the series, deeper characters, better writing...

Seriously, did you really play the other games? The amount of depth in both the characters in the plot far exceeds what's in FFIV and V, let alone the older games. Because I don't see how you can play a game with a villain like X-Death and then think Kefka is poorly done.