When I start the game I keep four "commoners" for my main party and dump everyone else. Then I keep any special characters that I pick up along the way. I also keep monster(s) if I am mining their offspring for items from poaching.