If you stay until after the credits in the first Pirates movie, you'll be treated to a little 1-2 minute video of Jack the Monkey swimming up out of the water and stealing a coin out of the box. He has the curse and since he was the only one present at the time of the theft, he was the only one cursed by it.