If you're inhibiting or interrupting someone else's ability to have fun then you're being a jerk!

As long as you don't cross that line, then you're good.