Well... to start off with... DO NOT give up just because you lose the 2-3 first times you fight him.
Remember that he's the hardest FF boss ever!

1) You should get the celestial weapons for everyone that you use in this battle.
2) You should get better armour. Auto haste is the most important ability for this battle. Auto protect is recomended but you can do it without auto protect (I did).
3) Go for the arms. Kill the arms before you go for Penance and once the arms get back... kill them again.
4) Use Quick hit.
5) In Penance second form (when the lower part dissapears)... summon an aeon if Penance is getting a turn while both arms are alive because he will use JUDGEMENTS DAY!!! Which will do 99999 dammage and take away all your MP.