Originally posted by Rude
Hardest boss in the game is the Lost Number, as when he goes into his full magic or full attack state toward the end of the battle he substantially stronger than the level your party would normally be at by that stage in the game.

Of course, he is probably considered an optional boss.... so I don't know if he counts as a valid choice.
I think you can prevent him from going to only one form if you even out your attacks between physical and magic. I have betten him quite a few times without him ever changeing which is why I think this holds true.