You keep your horse unless you tie the new one to a post. Except in one story situation in which you are forced to get a better yellow horse near the beginning