My mother drove a Yaris for a while when her car was in for servicing, and she hated it. I couldn't honestly say that the Prius is any better, but I imagine the build quality would be slightly better (an assumption that I'm basing on nothing in particular). Still, I'd shop around a bit more if I were you.

Oh, and both cars are ugly.