Eh, wait for better parties. There's no use blasting your head off over something like this, it sounded pretty simple you should have left before you deleveled.

Through all my jobs, I usualy just waited around for an hour or so in Valkurm to make sure I got a good party, and I've never had any troubles there.