He was talking about the evil organisation (the Empire in FFVI, Shin-Ra in FFVII), not the castle itself.

If you want to get technical, it's actually called the Imperial Palace.