Both Cap & Thor are under contract for 3 more movies, meaning they'll appear in their own sequel's + the 3rd Avengers two-parter. Bucky is most likely to replace Steve Rogers as Cap, considering he has something like an 8 movie contract. I think Hulk has a few more movies under his contract.

But that doesn't mean they can't sign up for more movies. The actors seem to be having fun making these movies. However, Chris Evans reported that the next two Avengers flicks will be filmed simultaneously (considering they're Part I & II, it makes sense) over a 9 month period. There's always a risk of burn-out.

Ant-Man should be good. Whedon gave the script a big thumbs up in the press & what I've seen from the trailers/promos looks great. It's also got a great cast. I'm hoping to see Ant-Man & Wasp in the next installment of Avengers.