Yeah, Yojimbo got on my nerves a bit too, but he also saved my ass a few times when fighting the dark aeons by doing zanmato.
I don't really have a least favourite aeon- I liked them all, but I guess if I had to choose i'd say ixion as he did the least damage of all my aeons.