There is a really easy way to find him. 1-- make sure you're not looking for him. 2-- Have a weak party with low HP, preferably with a member or two already dead. 3-- Make sure you haven't saved since three sidequests ago. Especially if one of those quests is the magic tower.

Otherwise, the best thing you can do is be moving as close to due North as possible without actually being right on due North, hold down the A button, and read a book. I recommend Anna Karenina or Atlas Shrugged.