He didn't even visit Ashe in person to give the ring back, he just sent Vaan to do it.

I think that all though he's a ladies-man and he flirts around with Penelo and Ashe, that by the end it clear that there is only one girl for him, and thats Fran.
She gets him and he gets her and no amount of pretty girls can come between a bond like that. and come on Fran is gorgeous,