Essentially there should be emotinoal attachment to the villain. You should WANT to defeat them.
I agree with this OR someone you DON'T want to defeat. Then its like 'why are we fighting pls stahp " and that also has a good emotional impact. Kinda like what you described in your post, I was just being more specific here :P