Having all the side quests at the end of the game doesn't matter to me one bit. If the plot is interesting I should be constantly excited to see what comes next. Wanting to screw around as you move through the game is indicative of a weak story to me. (Not knocking games where you can do that, just saying in most environments like that plot isn't up super high on the list of selling points.)