I'm fine with either way, but I guess I slightly prefer a party of four. Big fan of even numbers.

Furthermore, I think you have a bit more options and opportunities in a battle with four party members, e.g. you can easily use one party member for healing only while the other three can attack, which is more effective than having only two attackers, of course. But as I said, I don't mind having a party of three, either.