Except killing him really wouldn't have solved anything. Not until the Woodberry militia were turned towards peace, too. If she'd killed the Governor then, and it was obvious someone had assassinated him, there would have been even worse consequences. That's why she had to run. And, the fact that he followed her, meant that along the way she could try to kill him, but would only dare do so through the walkers. Otherwise, if anyone found the body, or his walking corpse, with a bullet hole or stab wound in it, again, they'd know he was assassinated.