And he would have regretted it. We're talking about someone who was eventually forced to kill his best friend for the good of the group. He may not like having to kill people a whole lot, but he's obviously moved on from the whole "I don't want to have to kill people so I'll keep giving you one more chance," to "I don't want to have to kill you, but god damn it, if you keep threatening everyone, acting like an idiot who's going to get someone killed, and throw a zombie in my face, you're done."
As for the other guy, yeah, he probably could have let him live, but in the heat of the moment he sided with the psycho prisoner and for all they know might have tried to kill them later on or cause some other kind of trouble. I probably would have done the same (except I might have at least done him the courtesy of putting a bullet in his head). But I'm a bit of a cold hearted bastard and if it's the zombie apocalypse it is not the time to take chances, get messy, and make mistakes Miss Frizzle.