I fought him in Origins...more than once. I think it's because you gave to be secretely equipped. Remove all your armour and weapons and try again.