Yes, Shadow Hearts: Covenant is a direct sequel to Shadow Hearts, Shadow Hearts: From the new World is a bit related to Covenant, but not much, I would say it more about references than anything else.

In my opinion, both first games are just a wonderful wonderful experience, with Covenant being more polished than Shadow Hearts. But From the New World is a waste of money and time; I hated it.

The good thing of playing them in order is that you will get a better grasp of the three stories and you will get the references.
The bad thing of playing them in order is that, after playing I & II, III possibly won't compare favorably in the least.
But, it's a matter of opinion, of course; you may end up loving From the New World no matter what.

And then, if you like them, you can start hunting for Koudelka which is the very first game that starts it all, but I warn you: if I & II end up being hard/expensive to get, Koudelka will be much harder.