It depends on what the game is focused on.

If the game has a clear intention from the start to focus on gameplay, I judge it by the gameplay. If the game has a clear intention from the start to focus on writing, story and characters, I judge it by the writing, story and characters. I split those three things up because it is very easy to confuse one of these things (story) with the other two (writing and characters). There are games that I feel have a terrible characters but a great story, and vice versa. Some have a crap story and characters, but the writing makes it worth pushing through because it's hilarious.

I highly doubt people judge Pacman, Sonic the Hedgehog, Tekken or Formula One Grand Prix based on story. If they do, power to them, but you get what I mean. I'm confident that someone who values story highly can still enjoy these games if they enjoy the gameplay.

Likewise, I highly doubt that people judge most point and click games on the gameplay. Many point and click games are just small puzzles to help you learn more about the story. Hell, even RPGs are often this way - the gameplay can be minimal but the story can be so deep that you just can't help yourself and keep chipping away.

In the end, it's just individual reaction. You either like something about a game for whatever reason or you don't for whatever reason. Thankfully I enjoy almost every game I play. I actually feel a little sorry for people who dislike games I enjoy. I feel they are missing out but it's not really their fault, it's just that they don't happen to like something. They are the losers in this situation. Not losers as an insult, but they are missing out on a source of enjoyment, so I consider myself a winner there. Likewise, I consider myself a loser when it comes to food. I dislike a large number of foods (well, not huge, but enough of them) and this means that I don't get to enjoy as many foods as others do. I can't help that, it's not my fault, but it's the way it is and I miss out on a source of enjoyment because I don't like mushrooms or fried tomato.