You don't really need all the characters. Also, sidelined characters won't ever be horribly underleveled, as they get a percentage of the exp that the main party gets. If you play only with three specific characters until level 60, the rest of the crew will be around lv50 anyway. The only thing they'll lack is developed limit breaks, but that's not very time consuming to sort out if you read up on how to do it most efficiently.