I'm going to echo Bolivar and say its a toss up between Lady Ashe and Terra Bradford for having it the worst. Garnet gets an honorable mention.

Terra was raised by an Empire that saw her as an object, was most likely experimented on, had her free will stripped from her and forced to perform some atrocious acts. She spends the majority of the game questioning whether she can honestly live with humanity, and even the Rebels she joins don't make it a secret they want to use her their own personal gain. Her life kind of sucks.

Ashe on the other hand loses everything she cares about in the first ten minutes of the game, is forced to retreat into the shadows while her kingdom is manipulated by Archades and her own journey is her being emotionally manipulated by the Occurians for their own ends. Her whole story is basically her trying to decide to give into her despair and feelings to seek revenge or to move above it and do what is best for her people. Like Bolivar says, she's one of the few characters where you really get to see how her life really affected her.