I do that all the time too, its so awkward when you realize your looking at someone or behind someone, because they're like "what?" or even worse "ooh yeah..."

Its worse when I do it while someone is talking to me =/