Have you done the sidequest with Dantro's wife? When you hunted the Rogue Tomato, you should have gotten a flower that he told you to take to her back at their village. Basically you need to do this side mission in order to get a key to the Estersand entrance to Barheim Passage.