3 Major counterpoints about the Crystal Armor

1) Crystal gear hasn't been considered "ultimate gear" since FFIV and even in THAT game there is still better armor.

2) The armor you speak of can only be equipped by a handful of characters and most of those characters actually have better equipment options than Crystal Gear. Let's face it, if they are using Crystal Armor its because someone else is equipped with the better gear not because its the best.

3) Have a high defense stat does not mean its the best armor. The Red Jacket, Black Garb, Minerva Bustier, and Genji Equipment are all better in terms of raising other stats and granting bonuses over the cost of what amounts to 5 or 10 defense points which can easily be made up with the Esper system. Even then, the best defensive armor in that game (Snow Scarf) is only useable by three characters (Umaro, Gau, and Mog) and all these extra equipment are not so easily found as you say it is. Most of the best gear is obtained in the Cyan's Dream World and Phoenix Cave and I don't exactly consider those dungeons to be walks in the park despite this being VI.