I had Full Draconic Armor, 100 in both Smithing and Enchanting, 100 Conjuration, and around a 70 one Handed Skill, plus I was a Brenton with an Item that boosted my Magic Resistance even more.

Yeah, he was barely able to hurt me with any attacks on either fight with him.