I 'think less' of non-readers, but it's more just that I have a hard time relating and being friends with someone who doesn't read. It's really only because I read SO MUCH and it's such a big part of my life that if I can't discuss books with someone, it's difficult.
But nah, life is busy and a lot of people who love to read can't just because they don't have the time.