Like Pike, it depends on the game. I'll usually change the difficulty partway through as well, as I learn what it is about the game I'm enjoying. Some games work best, in my eyes, under certain conditions, so I pretty much always play X-Com and stuff on Ironman for example.