Who you're attracted to absolutely can be racist if you've never addressed your inherent biases and prejudices having been raised as a white person with privilege. Not being attracted to an entire race usually has a lot more going on there than simply "I don't like the way they look", because beautiful people of all races exist. If you're just like, "Eh, I just don't get what's attractive about black people, but hey, it's just a preference", that's absolutely racist.

Sorry, but it's true.