In all seriousness, I consider a video game to simply be anything with some form of interactive gameplay which is played on a tv screen/monitor/whatever. This would include most visual novels, but maybe not all of them. I feel like there are some visual novels that probably don't have any real choice in them or anything, and those I would maybe argue against being a game.

The distinction between this and channel surfing would be creator intent I guess? Games have someone (or many someones) who created them with the intent of creating a fun, interactive experience for the player. Simple channel surfing doesn't have that. Then again, my roommates and I have created "games" out of similar experiences with picking music on spotify. Hmm.