Because he's having second thoughts after he joins you, and decides not to join because he's a coward and FuSoYa scares him.

Speaking of FuSoYa, again, since Evastio missed my question after we answered the same one almost simultaneously... What's up with FuSoYa's long, gravity-defying eyebrows?