Does it throw you out of the experience when someone you don't recognize just appears in place of an actor you're used to?
Game of Thrones example:
When Ed Skrein gets replaced by Michiel Huisman as Daario I actually liked the change more.
I wasn't bothered with the Tommen change either because he was a minor character that didn't really have dialogue until he came back anyway.
I also think Sense8 did a good job of replacing Capheus. They made a joke about it right away (new haircut?) and I got used to him within a few episodes.
Can you think of examples that were really bad?