I keep them all that are given to you, so the troops Ramza starts with and the three who join after the flashback is over. Other than that, I never really recruit characters cause I try to make room for the named characters but luckily I only like four of them which gives me plenty of flexibility to mix and match my teams. It helps that I rarely level up the named characters beyond their initial jobs cause why bother when their starting classes are so useful?