It looked like he was contemplating taking it off because he wants to move on. It is weird that they are basically making him Shane but these are different circumstances. He legit thought Rick was dead. This is actually worse because he wanted to shoot that dude before he even knew if he was abusive then he was all "sike nah, lemme go listen to this zombie".