These are the hardest of the main FF games I've played:

IV - I had A LOT of difficulty with Final Fantasy IV on Playstation my first time around.

V - I was fine until endgame when it kicked my butt so hard I still feel the pain.

I - The first game is painful. Even after the required hours of grinding, the game still has many cheap enemies (Ice Cave). Also, spells are almost useless so mages slow me down in the game. Luckily they fixed the mage problem afterward and made them useful in future games.

If we're counting spin-offs:

Dirge of Cerberus - I was never good at shooters. Or action games in general. I like them, but I'm terrible.

Tactics - I don't think I've ever beat the first battle... Yeah. If we're counting all the FF games, this is the winner for me.