Indeed, not waiting for Shadow results in getting him out of your party permanently.


It also makes you go to where you would normally find Shadow in the WoR to find Relm, THEN doing Relm's sidequest to get her.

So not waiting is pretty much one of the dumbest ideas ever, unless you want to see Relm's dream or something.