The argument works the other way around as well, we don't really know if the Senate is really good either. Though the game and Ultimania's point out that the Senate once ruled Archades until the military forced a imperial rule system, and that the Senate has largely been powerless since, with said current Senate hoping to use Larsa as a means of restoring their own political might. So I wouldn't say they are entirely innocent, as I said before, I wouldn't be surprised if Vayne actually had to plant evidence that they were thinking of taking out Gramis, and they certainly wanted Vayne dead and tried to have him executed over baseless charges. So I would argue the Senate was going to do to Vayne what Vayne did to them.
Gramis himself is an interesting figure who is not a lunatic, power hungry certainly but there is a reason behind his actions even if they were not always good. When we the players meet him though, he's in the twilight of his lifetime and starting to regret his actions. He wants Larsa to rule in his stead cause he recognizes that his actions that caused the wars in Nabradia and Dalmasca were going to ruin the empire. I don't feel he's a lunatic, just a man trying to protect his legacy and his country. This doesn't necessarily make his past sins forgiven but his actions do have some justification to them. Whether you agree largely hinges on where you stand morally.
What's to stop Larsa from doing to Vayne what Vayne did to his older brothers? Also, Gramis himself gets to choose the heir, its not the standard "eldest male heir" system. If Vayne really wanted to secure his rule, he would have killed any potential heir. Otherwise you run into conflicts like the other Ivalice title, FFTactics. You wipe out all heirs if you really want to be safe from legitimate threat of usurpation. Larsa is a political threat, yet Vayne pretty much coddles him throughout the game, not really deciding to try and kill him and even after Larsa turns on him he simply knocks him out and orders Gabranth to protect Larsa.Also, it's worth pointing out that Vayne had no reason to kill Larsa, because Larsa wouldn't inherit the throne until Vayne is out of the way, most likely dead. Succession passes from oldest to youngest, remember? Funny how only his older brothers wound up dead.
Before all of that, Vayne mentions to his father Gramis that he would protect his innocent brother (which he does until the final battle) and in the discussion over who should be the next ruler, Vayne argues that Archades in its current state could not support Larsa as a ruler and ruthlessness would be needed to make Larsa's rule sustainable for the Empire. As I mentioned before, in terms of Archades and Larsa, Vayne seems to have left both in a better position than they were before the game started. Vayne largely makes good on his talk with his father Gramis.
I think the big thing to understand here is that the initial story is Matsuno, which means nothing is black and white and considering the track record of his other games, I feel its safe to say the Senate is most likely corrupt and the antagonist are not evil monsters (well except the Lucavi) but people who simply live by "the end justifies the means".