There's little to judge about inactives and nothing gained in killing all inactives if there is one mafia pretending to be inert. The net loss is too high.
Then why go after a one-voter? Besides, a large amount of inactives can be extremely dangerous for the town. Towards the endgame, an inactive town will be destroyed if the mafia has the voting power due to the town not being able to establish any sort of bloc. While we shouldn't slay inactive indiscriminately, it's certainly worth trying to weed them out if possible.

An inactive town is still one more person for mafia to kill whether they do it at night or not. It doesnt matter if they're here or not but that they need to be dead for mafia to win.
Of course. But this applies to everyone. Active people can at least use their powers.