If you're not picking a single combat skill as a dominant skill, then it's your own fault. Just because the mechanic is broken, it doesn't mean that players making bad decisions don't deserve some of the blame for their having a rough time. What skills are you picking such that you find yourself having to reduce the difficulty?
On a different note, Oblivion has the best soundtrack out of the three most recent installments, I think.