It depends.

If it's The Elder Scrolls, then I play it mostly for the lore.

If it's Deus Ex or Fallout, then I play it mostly for story/atmosphere.

If it's just about any other RPG, then I probably play it mostly for the gameplay.