Definitely echoing what Night Fury is saying about teaching. Maths is not interesting as a subject to be taught, relatively speaking. Most other subjects can be engaging in a natural way - even the other sciences can be made interesting with more hands on things to do for learning. With maths... you're going to be staring at a bit of paper with numbers on it. There's not going to be any huge amount of talking through problems. So it can be (and probably often is) taught in a very dry, unengaging manner.

Then on top of that, a lot of maths requires understanding of more basic concepts to be built upon to get a full understanding of more complex concepts. Heck, I personally find that if I understand why I'm doing something in a particular problem, I will be able to do it better. So, if someone is lacking a piece of understanding, they're going to struggle to pick up the next bit and the next bit, leaving them probably not wanting to keep going because they just don't get it. A bit more time and attention would probably help immensely, but that cannot always be guaranteed, unfortunately.