"It's not for me but I don't mind if other people do it" is not nearly the same as it being against your very moral code.

You don't use words like "moral issue" and "guilt" to say that something is ok, but not your cup of tea.