If you decide to make changes, and people just make fun of you, then you need to kick them out.
In all seriousness, you're way too soft about stuff like this. Put your foot down for once.
If you need help in laying down the law, then I'll be around for backup. T'is my job after all.