There is no real deterrent for murder. Most murders are momentary lapses of judgment due to anger, hatred, greed, fear, et cetera, where the consequences are ignored. Planned out murders are due to arrogance about avoiding the consequences. Just because prison doesn't deter murder doesn't mean murderers shouldn't be punished, however.

Now, prison is a deterrent for other crimes: theft, for example. Breaking-and-entering. Rape, even. Most other crimes are conceivably deterred by the threat of punishment.

Also, I don't know about most prisons, but maximum security prisons are not a nice place to be.