Quite possibly they didn't have such a system because before the events of this film they'd never been attacked in such a fashion (let's not forget that there appear to be no major safeguards preventing the terrorist attacks at the start of the film either). Not to mention that a system that prevents objects from entering the atmosphere wouldn't be able to distinguish between presumably benign objects and malicious objects. The Vengeance was a Starfleet vessel. Even if there was a system in place for preventing malicious objects from entering the Earth's atmosphere, there would be no reason for said system to reject the Vengeance. How would it know the Vengeance was being piloted by a madman intent on killing as many people as possible? Even if there were military defences, by the time it was apparent the Vengeance was going to crash into San Francisco, it would probably have been too late for them to respond.