I think it would be much more comfortable being an Officer of the Empire than a servant of Sauron's. Mordor seems like a much more cruel place than the Empire. I think also you can justify what the Empire is doing a lot more than Mordor - they're bringing order to the galaxy, whereas Mordor's goal is just "destroy earth, live in orc filth".