Yeah... I don't remember having to do both in one visit. They're independent of each other, I always thought.

Just don't go inside the pagoda until you want to.