The only thing you save by leaving them alone is a guilty conscience.

The game creates a strange incentive to kill them because you get extra information in the Bestiary data. As far as I know, it has no effect whatsoever on how the Garif tribe views your party.