All of the Sword Saint abilities are physical.
Monks are only good if you are looking to boost HP. IMO HP is one of the least useful stats in the game. By lv99 most enemies will do a ton of damage to you, or inflict some nasty status effects. The best way to go is to avoid taking damage at all by a combination of reaction abilities, staying out of range, and making sure to kill anything that comes in range before it has a chance to attack.

If you have a lv50 character with a 'raw' PA of 12 and a Speed of 9
Leveled as a Mime: PA 18.9 | Speed 11.9
Leveled as a Knight: PA 18.5 | Speed 11.9
Leveled as a Ninja: PA 18.4 | Speed 12.4
Leveled as a Monk: PA 18 | Speed 11.9

So the over all difference isn't massive.

Fun Fact: Priests have as much HP growth as knights.