The correct answer would've been, "If I had actually been going out with Sarah, I would've regretted never being intimate with her. But since you and I have each other, there's nothing to be sad about in the least, and Sarah never crosses my mind."
Thinking about what you said before... to your girlfriend, it would've sounded like you were saying, "I'm sad that I'm going out with you, because I'd rather be having sex with Sarah. At this very minute."