If you cast berserk on him and used either twin or blink on your own characters, his physical attacks would probably miss. That way you can avoid the powerful physical attacks. simple.