Well you can still face him with a level one party. Just make sure the guys you won't be using in the final battle get the experience.